Merged PR 116: Update documentation
## Summary * Update observation space information and standardise formatting of code blocks in that section * Remove non-ascii quotation characters * Update custom blue agent page to match new AgentSession classes. * Introduce glossary * Provide a first draft of migration guide for 1.2 to 2.0 (probably not comprehensive) ## Test process Sphinx is able to build the documentation as checked on my local machine ## Checklist - [x] This PR is linked to a **work item** - [x] I have performed **self-review** of the code - [ ] I have written **tests** for any new functionality added with this PR - [x] I have updated the **documentation** if this PR changes or adds functionality - [x] I have run **pre-commit** checks for code style Related work items: #1602
This commit is contained in:
Marek Wolan
@@ -38,6 +38,8 @@ The best place to start is :ref:`about`
|
||||
PrimAITE API <source/_autosummary/primaite>
|
||||
PrimAITE Tests <source/_autosummary/tests>
|
||||
source/dependencies
|
||||
source/glossary
|
||||
source/migration_1.2_-_2.0
|
||||
|
||||
.. toctree::
|
||||
:caption: Project Links:
|
||||
|
||||
@@ -10,11 +10,11 @@ PrimAITE provides the following features:
|
||||
|
||||
* A flexible network / system laydown based on the Python networkx framework
|
||||
* Nodes and links (edges) host Python classes in order to present attributes and methods (and hence, a more representative model of a platform / system)
|
||||
* A ‘green agent’ Information Exchange Requirement (IER) function allows the representation of traffic (protocols and loading) on any / all links. Application of IERs is based on the status of node operating systems and services
|
||||
* A ‘green agent’ node Pattern-of-Life (PoL) function allows the representation of core behaviours on nodes (e.g. Hardware state, Software State, Service state, File System state)
|
||||
* A 'green agent' Information Exchange Requirement (IER) function allows the representation of traffic (protocols and loading) on any / all links. Application of IERs is based on the status of node operating systems and services
|
||||
* A 'green agent' node Pattern-of-Life (PoL) function allows the representation of core behaviours on nodes (e.g. changing the Hardware state, Software State, Service state, or File System state)
|
||||
* An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP, destination IP, protocol and port). Application of IERs adheres to any ACL restrictions
|
||||
* Presents an OpenAI Gym interface to the environment, allowing integration with any OpenAI Gym compliant defensive agents
|
||||
* Red agent activity based on ‘red’ IERs and ‘red’ PoL
|
||||
* Red agent activity based on 'red' IERs and 'red' PoL
|
||||
* Defined reward function for use with RL agents (based on nodes status, and green / red IER success)
|
||||
* Fully configurable (network / system laydown, IERs, node PoL, ACL, episode step period, episode max steps) and repeatable to suit the training requirements of agents. Therefore, not bound to a representation of any particular platform, system or technology
|
||||
* Full capture of discrete metrics relating to agent training (full system state, agent actions taken, average reward)
|
||||
@@ -201,7 +201,7 @@ An example observation space is provided below:
|
||||
* -
|
||||
- ID
|
||||
- Hardware State
|
||||
- SoftwareState
|
||||
- Software State
|
||||
- File System State
|
||||
- Service / Protocol A
|
||||
- Service / Protocol B
|
||||
@@ -250,48 +250,35 @@ An example observation space is provided below:
|
||||
|
||||
For the nodes, the following values are represented:
|
||||
|
||||
* ID
|
||||
* Hardware State:
|
||||
.. code-block::
|
||||
|
||||
* 1 = ON
|
||||
* 2 = OFF
|
||||
* 3 = RESETTING
|
||||
* 4 = SHUTTING_DOWN
|
||||
* 5 = BOOTING
|
||||
|
||||
* SoftwareState:
|
||||
|
||||
* 1 = GOOD
|
||||
* 2 = PATCHING
|
||||
* 3 = COMPROMISED
|
||||
|
||||
* Service State:
|
||||
|
||||
* 1 = GOOD
|
||||
* 2 = PATCHING
|
||||
* 3 = COMPROMISED
|
||||
* 4 = OVERWHELMED
|
||||
|
||||
* File System State:
|
||||
|
||||
* 1 = GOOD
|
||||
* 2 = CORRUPT
|
||||
* 3 = DESTROYED
|
||||
* 4 = REPAIRING
|
||||
* 5 = RESTORING
|
||||
[
|
||||
ID
|
||||
Hardware State (1=ON, 2=OFF, 3=RESETTING, 4=SHUTTING_DOWN, 5=BOOTING)
|
||||
Operating System State (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
File System State (0=none, 1=GOOD, 2=CORRUPT, 3=DESTROYED, 4=REPAIRING, 5=RESTORING)
|
||||
Service1/Protocol1 state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
Service2/Protocol2 state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
]
|
||||
|
||||
(Note that each service available in the network is provided as a column, although not all nodes may utilise all services)
|
||||
|
||||
For the links, the following statuses are represented:
|
||||
|
||||
* ID
|
||||
* Hardware State = N/A
|
||||
* SoftwareState = N/A
|
||||
* Protocol = loading in bits/s
|
||||
.. code-block::
|
||||
|
||||
[
|
||||
ID
|
||||
Hardware State (0=not applicable)
|
||||
Operating System State (0=not applicable)
|
||||
File System State (0=not applicable)
|
||||
Service1/Protocol1 state (Traffic load from this protocol on this link)
|
||||
Service2/Protocol2 state (Traffic load from this protocol on this link)
|
||||
]
|
||||
|
||||
NodeStatus component
|
||||
----------------------
|
||||
This is a MultiDiscrete observation space that can be though of as a one-dimensional vector of discrete states, represented by integers.
|
||||
This is a MultiDiscrete observation space that can be though of as a one-dimensional vector of discrete states.
|
||||
The example above would have the following structure:
|
||||
|
||||
.. code-block::
|
||||
@@ -307,9 +294,9 @@ Each ``node_info`` contains the following:
|
||||
.. code-block::
|
||||
|
||||
[
|
||||
hardware_state (0=none, 1=ON, 2=OFF, 3=RESETTING, 4=SHUTTING_DOWN, 5=BOOTING)
|
||||
hardware_state (0=none, 1=ON, 2=OFF, 3=RESETTING, 4=SHUTTING_DOWN, 5=BOOTING)
|
||||
software_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
file_system_state (0=none, 1=GOOD, 2=CORRUPT, 3=DESTROYED, 4=REPAIRING, 5=RESTORING)
|
||||
file_system_state (0=none, 1=GOOD, 2=CORRUPT, 3=DESTROYED, 4=REPAIRING, 5=RESTORING)
|
||||
service1_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
service2_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||
]
|
||||
@@ -320,10 +307,18 @@ In a network with three nodes and two services, the full observation space would
|
||||
|
||||
gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4])
|
||||
|
||||
.. note::
|
||||
NodeStatus observation component provides information only about nodes. Links are not considered.
|
||||
|
||||
LinkTrafficLevels
|
||||
-----------------
|
||||
This component is a MultiDiscrete space showing the traffic flow levels on the links in the network, after applying a threshold to convert it from a continuous to a discrete value.
|
||||
The number of bins can be customised with 5 being the default. It has the following strucutre:
|
||||
There are two configurable parameters:
|
||||
* ``quantisation_levels`` determines how many discrete bins to use for converting the continuous traffic value to discrete (default is 5).
|
||||
* ``combine_service_traffic`` determines whether to separately output traffic use for each network protocol or whether to combine them into an overall value for the link. (default is ``True``)
|
||||
|
||||
For example, with default parameters and a network with three links, the structure of this component would be:
|
||||
|
||||
.. code-block::
|
||||
|
||||
[
|
||||
@@ -337,16 +332,13 @@ Each ``link_status`` is a number from 0-4 representing the network load in relat
|
||||
.. code-block::
|
||||
|
||||
0 = No traffic (0%)
|
||||
1 = low traffic (<33%)
|
||||
2 = medium traffic (<66%)
|
||||
3 = high traffic (<100%)
|
||||
1 = low traffic (1%-33%)
|
||||
2 = medium traffic (33%-66%)
|
||||
3 = high traffic (66%-99%)
|
||||
4 = max traffic/ overwhelmed (100%)
|
||||
|
||||
If the network has three links, the full observation space would have 3 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example:
|
||||
Using ``gym`` notation, the shape of the obs space is: ``gym.spaces.MultiDiscrete([5,5,5])``.
|
||||
|
||||
.. code-block::
|
||||
|
||||
gym.spaces.MultiDiscrete([5,5,5])
|
||||
|
||||
Action Spaces
|
||||
**************
|
||||
|
||||
@@ -2,64 +2,137 @@
|
||||
=============
|
||||
|
||||
|
||||
**Integrating a user defined blue agent**
|
||||
Integrating a user defined blue agent
|
||||
*************************************
|
||||
|
||||
Integrating a blue agent with PrimAITE requires some modification of the code within the main.py file. The main.py file
|
||||
consists of a number of functions, each of which will invoke training for a particular agent. These are:
|
||||
.. note::
|
||||
|
||||
* Generic (run_generic)
|
||||
* Stable Baselines 3 PPO (:func:`~primaite.main.run_stable_baselines3_ppo)
|
||||
* Stable Baselines 3 A2C (:func:`~primaite.main.run_stable_baselines3_a2c)
|
||||
If you are planning to implement custom RL agents into PrimAITE, you must use the project as a repository. If you install PrimAITE as a python package from wheel, custom agents are not supported.
|
||||
|
||||
The selection of which agent type to use is made via the training config file. In order to train a user generated agent,
|
||||
the run_generic function should be selected, and should be modified (typically) to be:
|
||||
PrimAITE has integration with Ray RLLib and StableBaselines3 agents. All agents interface with PrimAITE through an :py:class:`primaite.agents.agent.AgentSessionABC<Agent Session>` which provides Input/Output of agent savefiles, as well as capturing and plotting performance metrics during training and evaluation. If you wish to integrate a custom blue agent, it is recommended to create a subclass of the :py:class:`primaite.agents.agent.AgentSessionABC` and implement the ``__init__()``, ``_setup()``, ``_save_checkpoint()``, ``learn()``, ``evaluate()``, ``_get_latest_checkpoint``, ``load()``, and ``save()`` methods.
|
||||
|
||||
Below is a barebones example of a custom agent implementation:
|
||||
|
||||
.. code:: python
|
||||
|
||||
agent = MyAgent(environment, num_steps)
|
||||
for episode in range(0, num_episodes):
|
||||
agent.learn()
|
||||
env.close()
|
||||
save_agent(agent)
|
||||
# src/primaite/agents/my_custom_agent.py
|
||||
|
||||
Where:
|
||||
from primaite.agents.agent import AgentSessionABC
|
||||
from primaite.common.enums import AgentFramework, AgentIdentifier
|
||||
|
||||
* *MyAgent* is the user created agent
|
||||
* *environment* is the :class:`~primaite.environment.primaite_env.Primaite` environment
|
||||
* *num_episodes* is the number of episodes in the session, as defined in the training config file
|
||||
* *num_steps* is the number of steps in an episode, as defined in the training config file
|
||||
* the *.learn()* function should be defined in the user created agent
|
||||
* the *env.close()* function is defined within PrimAITE
|
||||
* the *save_agent()* assumes that a *save()* function has been defined in the user created agent. If not, this line can
|
||||
be ommitted (although it is encouraged, since it will allow the agent to be saved and ported)
|
||||
class CustomAgent(AgentSessionABC):
|
||||
def __init__(self, training_config_path, lay_down_config_path):
|
||||
super().__init__(training_config_path, lay_down_config_path)
|
||||
assert self._training_config.agent_framework == AgentFramework.CUSTOM
|
||||
assert self._training_config.agent_identifier == AgentIdentifier.MY_AGENT
|
||||
self._setup()
|
||||
|
||||
The code below provides a suggested format for the learn() function within the user created agent.
|
||||
It's important to include the *self.environment.reset()* call within the episode loop in order that the
|
||||
environment is reset between episodes. Note that the example below should not be considered exhaustive.
|
||||
def _setup(self):
|
||||
super()._setup()
|
||||
self._env = Primaite(
|
||||
training_config_path=self._training_config_path,
|
||||
lay_down_config_path=self._lay_down_config_path,
|
||||
session_path=self.session_path,
|
||||
timestamp_str=self.timestamp_str,
|
||||
)
|
||||
self._agent = ... # your code to setup agent
|
||||
|
||||
.. code:: python
|
||||
def _save_checkpoint(self):
|
||||
checkpoint_num = self._training_config.checkpoint_every_n_episodes
|
||||
episode_count = self._env.episode_count
|
||||
save_checkpoint = False
|
||||
if checkpoint_num:
|
||||
save_checkpoint = episode_count % checkpoint_num == 0
|
||||
# saves checkpoint if the episode count is not 0 and save_checkpoint flag was set to true
|
||||
if episode_count and save_checkpoint:
|
||||
...
|
||||
# your code to save checkpoint goes here.
|
||||
# The path should start with self.checkpoints_path and include the episode number.
|
||||
|
||||
def learn(self) :
|
||||
def learn(self):
|
||||
...
|
||||
# call your agent's learning function here.
|
||||
|
||||
# pre-reqs
|
||||
super().learn() # this will finalise learning and output session metadata
|
||||
self.save()
|
||||
|
||||
# reset the environment
|
||||
self.environment.reset()
|
||||
done = False
|
||||
def evaluate(self):
|
||||
...
|
||||
# call your agent's evaluation function here.
|
||||
|
||||
for step in range(max_steps):
|
||||
# calculate the action
|
||||
action = ...
|
||||
self._env.close()
|
||||
super().evaluate()
|
||||
|
||||
# execute the environment step
|
||||
new_state, reward, done, info = self.environment.step(action)
|
||||
def _get_latest_checkpoint(self):
|
||||
...
|
||||
# Load an agent from file.
|
||||
|
||||
# algorithm updates
|
||||
...
|
||||
@classmethod
|
||||
def load(cls, path):
|
||||
...
|
||||
# Create a CustomAgent object which loads model weights from file.
|
||||
|
||||
# update to our new state
|
||||
state = new_state
|
||||
def save(self):
|
||||
...
|
||||
# Call your agent's function that saves it to a file
|
||||
|
||||
# if done, finish episode
|
||||
if done == True:
|
||||
break
|
||||
|
||||
You will also need to modify :py:class:`primaite.primaite_session.PrimaiteSession<PrimaiteSession>` and :py:mod:`primaite.common.enums` to capture your new agent identifiers.
|
||||
|
||||
.. code-block:: python
|
||||
:emphasize-lines: 17, 18
|
||||
|
||||
# src/primaite/common/enums.py
|
||||
|
||||
class AgentIdentifier(Enum):
|
||||
"""The Red Agent algo/class."""
|
||||
A2C = 1
|
||||
"Advantage Actor Critic"
|
||||
PPO = 2
|
||||
"Proximal Policy Optimization"
|
||||
HARDCODED = 3
|
||||
"The Hardcoded agents"
|
||||
DO_NOTHING = 4
|
||||
"The DoNothing agents"
|
||||
RANDOM = 5
|
||||
"The RandomAgent"
|
||||
DUMMY = 6
|
||||
"The DummyAgent"
|
||||
CUSTOM_AGENT = 7
|
||||
"Your custom agent"
|
||||
|
||||
.. code-block:: python
|
||||
:emphasize-lines: 3, 11, 12
|
||||
|
||||
# src/primaite_session.py
|
||||
|
||||
from primaite.agents.my_custom_agent import CustomAgent
|
||||
|
||||
# ...
|
||||
|
||||
def setup(self):
|
||||
"""Performs the session setup."""
|
||||
if self._training_config.agent_framework == AgentFramework.CUSTOM:
|
||||
_LOGGER.debug(f"PrimaiteSession Setup: Agent Framework = {AgentFramework.CUSTOM}")
|
||||
if self._training_config.agent_identifier == AgentIdentifier.CUSTOM_AGENT:
|
||||
self._agent_session = CustomAgent(self._training_config_path, self._lay_down_config_path)
|
||||
if self._training_config.agent_identifier == AgentIdentifier.HARDCODED:
|
||||
_LOGGER.debug(f"PrimaiteSession Setup: Agent Identifier =" f" {AgentIdentifier.HARDCODED}")
|
||||
if self._training_config.action_type == ActionType.NODE:
|
||||
# Deterministic Hardcoded Agent with Node Action Space
|
||||
self._agent_session = HardCodedNodeAgent(self._training_config_path, self._lay_down_config_path)
|
||||
|
||||
Finally, specify your agent in your training config.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# ~/primaite/config/path/to/your/config_main.yaml
|
||||
|
||||
# Training Config File
|
||||
|
||||
agent_framework: CUSTOM
|
||||
agent_identifier: CUSTOM_AGENT
|
||||
random_red_agent: False
|
||||
# ...
|
||||
|
||||
Now you can :ref:`run a primaite session<run a primaite session>` with your custom agent by passing in the custom ``config_main``.
|
||||
|
||||
77
docs/source/glossary.rst
Normal file
77
docs/source/glossary.rst
Normal file
@@ -0,0 +1,77 @@
|
||||
Glossary
|
||||
=============
|
||||
|
||||
.. glossary::
|
||||
:sorted:
|
||||
|
||||
Network
|
||||
The network in primaite is a logical representation of a computer network containing :term:`Nodes<Node>` and :term:`Links<Link>`.
|
||||
|
||||
Node
|
||||
A Node represents a network endpoint. For example a computer, server, switch, or an actuator.
|
||||
|
||||
Link
|
||||
A Link represents the connection between two Nodes. For example, a physical wire between a computer and a switch or a wireless connection.
|
||||
|
||||
Protocol
|
||||
Protocols are used by links to separate different types of network traffic. Common examples would be HTTP, TCP, and UDP.
|
||||
|
||||
Service
|
||||
A service represents a piece of software that is installed on a node, such as a web server or a database.
|
||||
|
||||
Access Control List
|
||||
PrimAITE blocks or allows certain traffic on the network by simulating firewall rules, which are defined in the Access Control List.
|
||||
|
||||
Agent
|
||||
An agent is a representation of a user of the network. Typically this would be a user that is using one of the computer nodes, though it could be an autonomous agent.
|
||||
|
||||
Green agent
|
||||
Simulates typical benign activity on the network, such as real users using computers and servers.
|
||||
|
||||
Red Agent
|
||||
An agent that is aiming to attack the network in some way, for example by executing a Denial-Of-Service attack or stealing data.
|
||||
|
||||
Blue Agent
|
||||
A defensive agent that protects the network from Red Agent attacks to minimise disruption to green agents and protect data.
|
||||
|
||||
Information Exchange Requirement (IER)
|
||||
Simulates network traffic by sending data from one network node to another via links for a specified amount of time. IERs can be part of green agent behaviour or red agent behaviour. PrimAITE can be configured to apply a penalty for green agents' IERs being blocked and a reward for red agents' IERs being blocked.
|
||||
|
||||
Pattern-of-Life (PoL)
|
||||
PoLs allow agents to change the current hardware, OS, file system, or service statuses of nodes during the course of an episode. For example, a green agent may restart a server node to represent scheduled maintainance. A red agent's Pattern-of-Life can be used to attack nodes by changing their states to CORRUPTED or COMPROMISED.
|
||||
|
||||
Reward
|
||||
The reward is a single number used by the blue agent to understand whether it's performing well or poorly. RL agents change their behaviour in an attempt to increase the expected reward each episode. The reward is generated based on the current states of the environment / :term:`reference environment` and is impacted positively by things like green IERS running successfully and negatively by things like nodes being compromised.
|
||||
|
||||
Observation
|
||||
An observation is a representation of the current state of the environment that is given to the learning agent so it can decide on which action to perform. If the environment is 'fully observable', the observation contains information about every possible aspect of the environment. More commonly, the environment is 'partially observable' which means the learning agent has to make decisions without knowing every detail of the current environment state.
|
||||
|
||||
Action
|
||||
The learning agent decides on an action to take on every step in the simulation. The action has the chance to positively or negatively impact the environment state. Over time, the agent aims to learn which actions to take when to maximise the expected reward.
|
||||
|
||||
Training
|
||||
During training, an RL agent is placed in the simulated network and it learns which actions to take in which scenarios to obtain maximum reward.
|
||||
|
||||
Evaluation
|
||||
During evaluation, an RL agent acts on the simulated network but it is not allowed to update it's behaviour. Evaluation is used to assess how successful agents are at defending the network.
|
||||
|
||||
Step
|
||||
The agents can only act in the environment at discrete intervals. The time step is the basic unit of time in the simulation. At each step, the RL agent has an opportunity to observe the state of the environment and decide an action. Steps are also used for updating states for time-dependent activities such as rebooting a node.
|
||||
|
||||
Episode
|
||||
When an episode starts, the network simulation is reset to an initial state. The agents take actions on each step of the episode until it reaches a terminal state, which usually happens after a predetermined number of steps. After the terminal state is reached, a new episode starts and the RL agent has another opportunity to protect the network.
|
||||
|
||||
Reference environment
|
||||
While the network simulation is unfolding, a parallel simulation takes place which is identical to the main one except that blue and red agent actions are not applied. This reference environment essentially shows what would be happening to the network if there had been no cyberattack or defense. The reference environment is used to calculate rewards.
|
||||
|
||||
Transaction
|
||||
PrimAITE records the decisions of the learning agent by saving its observation, action, and reward at every time step. During each session, this data is saved to disk to allow for full inspection.
|
||||
|
||||
Laydown
|
||||
The laydown is a file which defines the training scenario. It contains the network topology, firewall rules, services, protocols, and details about green and red agent behaviours.
|
||||
|
||||
Gym
|
||||
PrimAITE uses the Gym reinforcement learning framework API to create a training environment and interface with RL agents. Gym defines a common way of creating observations, actions, and rewards.
|
||||
|
||||
User data directory
|
||||
PrimAITE supports upgrading software version while retaining user data. The user data directory is where configs, notebooks, and results are stored, this location is `~/primaite` on linux/darwin and `C:\Users\<username>\primaite` on Windows.
|
||||
51
docs/source/migration_1.2_-_2.0.rst
Normal file
51
docs/source/migration_1.2_-_2.0.rst
Normal file
@@ -0,0 +1,51 @@
|
||||
v1.2 to v2.0 Migration guide
|
||||
============================
|
||||
|
||||
**1. Installing PrimAITE**
|
||||
|
||||
Like before, you can install primaite from the repository by running ``pip install -e .``. But, there is now an additional setup step which does several things, like setting up user directories, copy default configs and notebooks, etc. Once you have installed PrimAITE to your virtual environment, run this command to finalise setup.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
primaite setup
|
||||
|
||||
**2. Running a training session**
|
||||
|
||||
In version 1.2 of PrimAITE, the main entry point for training or evaluating agents was the ``src/primaite/main.py`` file. v2.0.0 introduced managed 'sessions' which are responsible for reading configuration files, performing training, and writing outputs.
|
||||
|
||||
``main.py`` file still runs a training session but it now uses the new `PrimaiteSession`, and it now requires you to provide the path to your config files.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python src/primaite/main.py --tc path/to/training-config.yaml --ldc path/to/laydown-config.yaml
|
||||
|
||||
Alternatively, the session can be invoked via the commandline by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
primaite session --tc path/to/training-config.yaml --ldc path/to/laydown-config.yaml
|
||||
|
||||
**3. Location of configs**
|
||||
|
||||
In version 1.2, training configs and laydown configs were all stored in the project repository under ``src/primaite/config``. Version 2.0.0 introduced user data directories, and now when you install and setup PrimAITE, config files are stored in your user data location. On Linux/OSX, this is stored in ``~/primaite/config``. On Windows, this is stored in ``C:\Users\<your username>\primaite\configs``. Upon first setup, the configs folder is populated with some default yaml files. It is recommended that you store all your custom configuration files here.
|
||||
|
||||
**4. Contents of configs**
|
||||
|
||||
Some things that were previously part of the laydown config are now part of the traning config.
|
||||
|
||||
* Actions
|
||||
|
||||
If you have custom configs which use these, you will need to adapt them by moving the configuration from the laydown config to the training config.
|
||||
|
||||
Also, there are new configurable items in the training config:
|
||||
|
||||
* Observations
|
||||
* Agent framework
|
||||
* Agent
|
||||
* Deep learning framework
|
||||
* random red agents
|
||||
* seed
|
||||
* deterministic
|
||||
* hard coded agent view
|
||||
|
||||
Each of these items have default values which are designed so that PrimAITE has the same behaviour as it did in 1.2.0, so you do not have to specify them.
|
||||
@@ -1,3 +1,5 @@
|
||||
.. _run a primaite session:
|
||||
|
||||
Run a PrimAITE Session
|
||||
======================
|
||||
|
||||
@@ -78,9 +80,9 @@ PrimAITE automatically creates two sets of results from each session:
|
||||
* Timestamp
|
||||
* Episode number
|
||||
* Step number
|
||||
* Initial observation space (what the blue agent observed when it decided its action)
|
||||
* Reward value
|
||||
* Action taken (as presented by the blue agent on this step). Individual elements of the action space are presented in the format AS_X
|
||||
* Initial observation space (what the blue agent observed when it decided its action)
|
||||
|
||||
**Diagrams**
|
||||
|
||||
|
||||
Reference in New Issue
Block a user