Committed the v1.1.0 code provided by James Short. Had to add setuptools==66 to setup.py as older versions of Gym are uninstallable with setuptools>=67
This commit is contained in:
Chris McCarthy
@@ -1,4 +1,4 @@
|
||||
.. _about:
|
||||
.. _about:
|
||||
|
||||
About PrimAITE
|
||||
==============
|
||||
@@ -11,7 +11,7 @@ PrimAITE provides the following features:
|
||||
* A flexible network / system laydown based on the Python networkx framework
|
||||
* Nodes and links (edges) host Python classes in order to present attributes and methods (and hence, a more representative model of a platform / system)
|
||||
* A ‘green agent’ Information Exchange Requirement (IER) function allows the representation of traffic (protocols and loading) on any / all links. Application of IERs is based on the status of node operating systems and services
|
||||
* A ‘green agent’ node Pattern-of-Life (PoL) function allows the representation of core behaviours on nodes (e.g. Operating state, Operating System state, Service state)
|
||||
* A ‘green agent’ node Pattern-of-Life (PoL) function allows the representation of core behaviours on nodes (e.g. Operating state, Operating System state, Service state, File System state)
|
||||
* An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP, destination IP, protocol and port). Application of IERs adheres to any ACL restrictions
|
||||
* Presents an OpenAI Gym interface to the environment, allowing integration with any OpenAI Gym compliant defensive agents
|
||||
* Red agent activity based on ‘red’ IERs and ‘red’ PoL
|
||||
@@ -37,6 +37,7 @@ Active Nodes also have the following attributes (Class: Active Node):
|
||||
|
||||
* IP Address
|
||||
* Operating System State (GOOD, PATCHING, COMPROMISED - enumeration)
|
||||
* File System State (GOOD, CORRUPT, DESTROYED, REPAIRING, RESTORING - enumeration)
|
||||
|
||||
Service Nodes also have the following attributes (Class: Service Node):
|
||||
|
||||
@@ -114,6 +115,14 @@ The status changes that can be made to a node are as follows:
|
||||
* PATCHING - when a status of patching is entered, the node will automatically exit this state after a number of steps (as defined by the osPatchingDuration configuration item) after which it returns to a GOOD state
|
||||
* COMPROMISED
|
||||
|
||||
* File System State:
|
||||
|
||||
* GOOD
|
||||
* CORRUPT (can be resolved by repair or restore)
|
||||
* DESTROYED (can be resolved by restore only)
|
||||
* REPAIRING - when a status of repairing is entered, the node will automatically exit this state after a number of steps (as defined by the fileSystemRepairingLimit configuration item) after which it returns to a GOOD state
|
||||
* RESTORING - when a status of repairing is entered, the node will automatically exit this state after a number of steps (as defined by the fileSystemRestoringLimit configuration item) after which it returns to a GOOD state
|
||||
|
||||
* Service Nodes only:
|
||||
|
||||
* Service State (for any associated service):
|
||||
@@ -123,6 +132,20 @@ The status changes that can be made to a node are as follows:
|
||||
* COMPROMISED
|
||||
* OVERWHELMED
|
||||
|
||||
Red agent pattern-of-life has an additional feature not found in the green pattern-of-life. This is the ability to influence the state of the attributes of a node via a number of different conditions:
|
||||
|
||||
* DIRECT:
|
||||
|
||||
The pattern-of-life described by the configuration file item will be applied regardless of any other conditions in the network. This is particularly useful for direct red agent entry into the network.
|
||||
|
||||
* IER:
|
||||
|
||||
The pattern-of-life described by the configuration file item will be applied to the service on the node, only if there is an IER of the same protocol / service type incoming at the specified timestep.
|
||||
|
||||
* SERVICE:
|
||||
|
||||
The pattern-of-life described by the configuration file item will be applied to the node based on the state of a service. The service can either be on the same node, or a different node within the network.
|
||||
|
||||
Access Control List modelling
|
||||
*****************************
|
||||
|
||||
@@ -162,19 +185,20 @@ Observation Spaces
|
||||
|
||||
The OpenAI Gym observation space provides the status of all nodes and links across the whole system:
|
||||
|
||||
* Nodes (in terms of operating state, operating system state, and services state)
|
||||
* Nodes (in terms of operating state, operating system state, file system state and services state)
|
||||
* Links (in terms of current loading for each service/protocol)
|
||||
|
||||
An example observation space is provided below:
|
||||
|
||||
.. list-table:: Observation Space example
|
||||
:widths: 25 25 25 25 25 25
|
||||
:widths: 25 25 25 25 25 25 25
|
||||
:header-rows: 1
|
||||
|
||||
* -
|
||||
- ID
|
||||
- Operating State
|
||||
- O/S State
|
||||
- File System State
|
||||
- Service / Protocol A
|
||||
- Service / Protocol B
|
||||
* - Node A
|
||||
@@ -183,38 +207,44 @@ An example observation space is provided below:
|
||||
- 1
|
||||
- 1
|
||||
- 1
|
||||
- 1
|
||||
* - Node B
|
||||
- 2
|
||||
- 1
|
||||
- 3
|
||||
- 1
|
||||
- 1
|
||||
- 1
|
||||
* - Node C
|
||||
- 3
|
||||
- 2
|
||||
- 1
|
||||
- 1
|
||||
- 3
|
||||
- 2
|
||||
* - Link 1
|
||||
- 5
|
||||
- 1
|
||||
- 1
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
- 10000
|
||||
* - Link 2
|
||||
- 6
|
||||
- 1
|
||||
- 1
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
- 10000
|
||||
* - Link 3
|
||||
- 7
|
||||
- 1
|
||||
- 1
|
||||
- 0
|
||||
- 0
|
||||
- 0
|
||||
- 5000
|
||||
- 0
|
||||
|
||||
The observation space is a 6 x 5 Box type (OpenAI Gym Space) in this example. This is made up from the node and link information detailed below.
|
||||
The observation space is a 6 x 6 Box type (OpenAI Gym Space) in this example. This is made up from the node and link information detailed below.
|
||||
|
||||
For the nodes, the following values are represented:
|
||||
|
||||
@@ -238,6 +268,14 @@ For the nodes, the following values are represented:
|
||||
* 3 = COMPROMISED
|
||||
* 4 = OVERWHELMED
|
||||
|
||||
* File System State:
|
||||
|
||||
* 1 = GOOD
|
||||
* 2 = CORRUPT
|
||||
* 3 = DESTROYED
|
||||
* 4 = REPAIRING
|
||||
* 5 = RESTORING
|
||||
|
||||
(Note that each service available in the network is provided as a column, although not all nodes may utilise all services)
|
||||
|
||||
For the links, the following statuses are represented:
|
||||
@@ -262,8 +300,8 @@ The choice of action space used during a training session is determined in the c
|
||||
The agent is able to influence the status of nodes by switching them off, resetting, or patching operating systems and services. In this instance, the action space is an OpenAI Gym multidiscrete type, as follows:
|
||||
|
||||
* [0, num nodes] - Node ID (0 = nothing, node ID)
|
||||
* [0, 3] - What property it's acting on (0 = nothing, 1 = state, 2 = O/S state, 3 = service state)
|
||||
* [0, 3] - Action on property (0 = nothing, 1 = on, 2 = off, 3 = reset / patch)
|
||||
* [0, 4] - What property it's acting on (0 = nothing, 1 = state, 2 = O/S state, 3 = service state, 4 = file system state)
|
||||
* [0, 3] - Action on property (0 = nothing, 1 = on / scan, 2 = off / repair, 3 = reset / patch / restore)
|
||||
* [0, num services] - Resolves to service ID (0 = nothing, resolves to service)
|
||||
|
||||
**Access Control List**
|
||||
@@ -305,4 +343,3 @@ The PrimAITE project has an ambition to include the following enhancements in fu
|
||||
* Integration with a suitable standardised framework to allow multi-agent integration
|
||||
* Integration with external threat emulation tools, either using off-line data, or integrating at runtime
|
||||
* Provision of data such that agents can construct alternative observation spaces (as an alternative to the default PrimAITE observation space)
|
||||
* Introduction of a testing phase (post training) to evaluate the effectiveness of the training
|
||||
|
||||
@@ -5,8 +5,8 @@ The Config Files Explained
|
||||
|
||||
PrimAITE uses two configuration files for its operation:
|
||||
|
||||
* config_main.yaml - used to define the top-level settings of the PrimAITE environment, and the training session that is to be run.
|
||||
* config_[name].yaml - used to define the low-level settings of a training session, including the network laydown, green / red agent information exchange requirements (IERSs), Access Control Rules, Action Space type, and the number of steps in each episode.
|
||||
* config_main.yaml - used to define the top-level settings of the PrimAITE environment, and the session that is to be run.
|
||||
* config_[name].yaml - used to define the low-level settings of a session, including the network laydown, green / red agent information exchange requirements (IERSs), Access Control Rules, Action Space type, and the number of steps in each episode.
|
||||
|
||||
config_main.yaml:
|
||||
*****************
|
||||
@@ -17,7 +17,7 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
* **agentIdentifier** [enum]
|
||||
|
||||
This identifies the agent to use for the training session. Select from one of the following:
|
||||
This identifies the agent to use for the session. Select from one of the following:
|
||||
|
||||
* GENERIC - Where a user developed agent is to be used
|
||||
* STABLE_BASELINES3_PPO - Use a SB3 PPO agent
|
||||
@@ -25,15 +25,27 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
* **numEpisodes** [int]
|
||||
|
||||
This defines the number of episodes that the agent will train over. Each episode consists of a number of steps (with step number defined in the config_[name].yaml file)
|
||||
This defines the number of episodes that the agent will train or be evaluated over. Each episode consists of a number of steps (with step number defined in the config_[name].yaml file)
|
||||
|
||||
* **timeDelay** [int]
|
||||
|
||||
The time delay (in milliseconds) to take between each step when training a GENERIC agent
|
||||
The time delay (in milliseconds) to take between each step when running a GENERIC agent session
|
||||
|
||||
* **configFilename** [filename]
|
||||
|
||||
The name of the config_[name].yaml file to use for this training session
|
||||
The name of the config_[name].yaml file to use for this session
|
||||
|
||||
* **sessionType** [text]
|
||||
|
||||
Type of session to be run (TRAINING or EVALUATION)
|
||||
|
||||
* **loadAgent** [bool]
|
||||
|
||||
Determine whether to load an agent from file
|
||||
|
||||
* **agentLoadFile** [text]
|
||||
|
||||
File path and file name of agent if you're loading one in
|
||||
|
||||
* **observationSpaceHighValue** [int]
|
||||
|
||||
@@ -69,6 +81,10 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
The score to give when the node should be off, but is resetting
|
||||
|
||||
* **Node Operating State [resetting]** [int]
|
||||
|
||||
The score to give when the node is resetting
|
||||
|
||||
* **Node Operating System or Service State [goodShouldBePatching]** [int]
|
||||
|
||||
The score to give when the state should be patching, but is good
|
||||
@@ -93,6 +109,10 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
The score to give when the state should be overwhelmed, but is patching
|
||||
|
||||
* **Node Operating System or Service State [patching]** [int]
|
||||
|
||||
The score to give when the state is patching
|
||||
|
||||
* **Node Operating System or Service State [compromisedShouldBeGood]** [int]
|
||||
|
||||
The score to give when the state should be good, but is compromised
|
||||
@@ -125,6 +145,106 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
The score to give when the state is overwhelmed
|
||||
|
||||
* **Node File System State [goodShouldBeRepairing]** [int]
|
||||
|
||||
The score to give when the state should be repairing, but is good
|
||||
|
||||
* **Node File System State [goodShouldBeRestoring]** [int]
|
||||
|
||||
The score to give when the state should be restoring, but is good
|
||||
|
||||
* **Node File System State [goodShouldBeCorrupt]** [int]
|
||||
|
||||
The score to give when the state should be corrupt, but is good
|
||||
|
||||
* **Node File System State [goodShouldBeDestroyed]** [int]
|
||||
|
||||
The score to give when the state should be destroyed, but is good
|
||||
|
||||
* **Node File System State [repairingShouldBeGood]** [int]
|
||||
|
||||
The score to give when the state should be good, but is repairing
|
||||
|
||||
* **Node File System State [repairingShouldBeRestoring]** [int]
|
||||
|
||||
The score to give when the state should be restoring, but is repairing
|
||||
|
||||
* **Node File System State [repairingShouldBeCorrupt]** [int]
|
||||
|
||||
The score to give when the state should be corrupt, but is repairing
|
||||
|
||||
* **Node File System State [repairingShouldBeDestroyed]** [int]
|
||||
|
||||
The score to give when the state should be destroyed, but is repairing
|
||||
|
||||
* **Node File System State [repairing]** [int]
|
||||
|
||||
The score to give when the state is repairing
|
||||
|
||||
* **Node File System State [restoringShouldBeGood]** [int]
|
||||
|
||||
The score to give when the state should be good, but is restoring
|
||||
|
||||
* **Node File System State [restoringShouldBeRepairing]** [int]
|
||||
|
||||
The score to give when the state should be repairing, but is restoring
|
||||
|
||||
* **Node File System State [restoringShouldBeCorrupt]** [int]
|
||||
|
||||
The score to give when the state should be corrupt, but is restoring
|
||||
|
||||
* **Node File System State [restoringShouldBeDestroyed]** [int]
|
||||
|
||||
The score to give when the state should be destroyed, but is restoring
|
||||
|
||||
* **Node File System State [restoring]** [int]
|
||||
|
||||
The score to give when the state is restoring
|
||||
|
||||
* **Node File System State [corruptShouldBeGood]** [int]
|
||||
|
||||
The score to give when the state should be good, but is corrupt
|
||||
|
||||
* **Node File System State [corruptShouldBeRepairing]** [int]
|
||||
|
||||
The score to give when the state should be repairing, but is corrupt
|
||||
|
||||
* **Node File System State [corruptShouldBeRestoring]** [int]
|
||||
|
||||
The score to give when the state should be restoring, but is corrupt
|
||||
|
||||
* **Node File System State [corruptShouldBeDestroyed]** [int]
|
||||
|
||||
The score to give when the state should be destroyed, but is corrupt
|
||||
|
||||
* **Node File System State [corrupt]** [int]
|
||||
|
||||
The score to give when the state is corrupt
|
||||
|
||||
* **Node File System State [destroyedShouldBeGood]** [int]
|
||||
|
||||
The score to give when the state should be good, but is destroyed
|
||||
|
||||
* **Node File System State [destroyedShouldBeRepairing]** [int]
|
||||
|
||||
The score to give when the state should be repairing, but is destroyed
|
||||
|
||||
* **Node File System State [destroyedShouldBeRestoring]** [int]
|
||||
|
||||
The score to give when the state should be restoring, but is destroyed
|
||||
|
||||
* **Node File System State [destroyedShouldBeCorrupt]** [int]
|
||||
|
||||
The score to give when the state should be corrupt, but is destroyed
|
||||
|
||||
* **Node File System State [destroyed]** [int]
|
||||
|
||||
The score to give when the state is destroyed
|
||||
|
||||
* **Node File System State [scanning]** [int]
|
||||
|
||||
The score to give when the state is scanning
|
||||
|
||||
* **IER Status [redIerRunning]** [int]
|
||||
|
||||
The score to give when a red agent IER is permitted to run
|
||||
@@ -147,6 +267,18 @@ The config_main.yaml file consists of the following attributes:
|
||||
|
||||
The number of steps to take when patching a service
|
||||
|
||||
* **fileSystemRepairingLimit** [int]:
|
||||
|
||||
The number of steps to take when repairing the file system
|
||||
|
||||
* **fileSystemRestoringLimit** [int]
|
||||
|
||||
The number of steps to take when restoring the file system
|
||||
|
||||
* **fileSystemScanningLimit** [int]
|
||||
|
||||
The number of steps to take when scanning the file system
|
||||
|
||||
config_[name].yaml:
|
||||
*******************
|
||||
|
||||
@@ -154,19 +286,19 @@ The config_[name].yaml file consists of the following attributes:
|
||||
|
||||
* **itemType: ACTIONS** [enum]
|
||||
|
||||
Determines whether a NODE or ACL action space format is adopted for the training session
|
||||
Determines whether a NODE or ACL action space format is adopted for the session
|
||||
|
||||
* **itemType: STEPS** [int]
|
||||
|
||||
Determines the number of steps to run in each episode of the training session
|
||||
Determines the number of steps to run in each episode of the session
|
||||
|
||||
* **itemType: PORTS** [int]
|
||||
|
||||
Provides a list of ports modelled in this training session
|
||||
Provides a list of ports modelled in this session
|
||||
|
||||
* **itemType: SERVICES** [freetext]
|
||||
|
||||
Provides a list of services modelled in this training session
|
||||
Provides a list of services modelled in this session
|
||||
|
||||
* **itemType: NODE**
|
||||
|
||||
@@ -180,6 +312,7 @@ The config_[name].yaml file consists of the following attributes:
|
||||
* **hardwareState** [enum]: The initial hardware state of the node. Can be one of ON, OFF or RESETTING
|
||||
* **ipAddress** [IP address]: The IP address of the component in format xxx.xxx.xxx.xxx
|
||||
* **softwareState** [enum]: The intial state of the node operating system. Can be GOOD, PATCHING or COMPROMISED
|
||||
* **fileSystemState** [enum]: The initial state of the node file system. Can be GOOD, CORRUPT, DESTROYED, REPAIRING or RESTORING
|
||||
* **services**: For each service associated with the node:
|
||||
|
||||
* **name** [freetext]: Free-text name of the service, but must match one of the services defined for the system in the services list
|
||||
@@ -231,7 +364,7 @@ The config_[name].yaml file consists of the following attributes:
|
||||
* **id** [int]: Unique ID for this YAML item
|
||||
* **startStep** [int]: The start step (in the episode) for this PoL to begin
|
||||
* **endStep** [int]: Not currently used. Default to same as start step
|
||||
* **node** [int]: The ID of the node to apply the PoL to
|
||||
* **nodeId** [int]: The ID of the node to apply the PoL to
|
||||
* **type** [enum]: The type of PoL to apply. Can be one of OPERATING, OS or SERVICE
|
||||
* **protocol** [freetext]: The protocol to be affected if SERVICE type is chosen. Must match a value in the services list
|
||||
* **state** [enuum]: The state to apply to the node (which represents the PoL change). Can be one of ON, OFF or RESETTING (for node state) or GOOD, PATCHING or COMPROMISED (for operating system state) or GOOD, PATCHING, COMPROMISED or OVERWHELMED (for service state)
|
||||
@@ -243,11 +376,14 @@ The config_[name].yaml file consists of the following attributes:
|
||||
* **id** [int]: Unique ID for this YAML item
|
||||
* **startStep** [int]: The start step (in the episode) for this PoL to begin
|
||||
* **endStep** [int]: Not currently used. Default to same as start step
|
||||
* **node** [int]: The ID of the node to apply the PoL to
|
||||
* **targetNodeId** [int]: The ID of the node to apply the PoL to
|
||||
* **initiator** [enum]: What initiates the PoL. Can be DIRECT, IER or SERVICE
|
||||
* **type** [enum]: The type of PoL to apply. Can be one of OPERATING, OS or SERVICE
|
||||
* **protocol** [freetext]: The protocol to be affected if SERVICE type is chosen. Must match a value in the services list
|
||||
* **state** [enum]: The state to apply to the node (which represents the PoL change). Can be one of ON, OFF or RESETTING (for node state) or GOOD, PATCHING or COMPROMISED (for operating system state) or GOOD, PATCHING, COMPROMISED or OVERWHELMED (for service state)
|
||||
* **isEntryNode** [bool]: Defines whether the node affected is an entry node to the system
|
||||
* **state** [enum]: The state to apply to the node (which represents the PoL change). Can be one of ON, OFF or RESETTING (for node state) or GOOD, PATCHING or COMPROMISED (for operating system state) or GOOD, PATCHING, COMPROMISED or OVERWHELMED (for service state) or GOOD, CORRUPT, DESTROYED, REPAIRING or RESTORING (for file system state)
|
||||
* **sourceNodeId** [int] The ID of the source node containing the service to check (used for SERVICE initiator)
|
||||
* **sourceNodeService** [freetext]: The service on the source node to check (used for SERVICE initiator). Must match a value in the services list for this node
|
||||
* **sourceNodeServiceState** [enum]: The state of the source node service to check (used for SERVICE initiator). Can be one of GOOD, PATCHING, COMPROMISED or OVERWHELMED
|
||||
|
||||
* **itemType: ACL_RULE**
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ What is PrimAITE?
|
||||
PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. It incorporates the functionality required of a Primary-level environment, as specified in the Dstl ARCD Training Environment Matrix document:
|
||||
|
||||
* The ability to model a relevant platform / system context;
|
||||
* The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, services and processes;
|
||||
* The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes;
|
||||
* Operates at machine-speed to enable fast training cycles.
|
||||
|
||||
PrimAITE aims to evolve into an ARCD environment that could be used as the follow-on from Reception level approaches (e.g. YAWNING TITAN), and help bridge the Sim-to-Real gap into Secondary level environments (e.g. IMAGINARY YAK).
|
||||
@@ -38,5 +38,5 @@ The best place to start is :ref:`about`
|
||||
about
|
||||
dependencies
|
||||
config
|
||||
training
|
||||
session
|
||||
results
|
||||
|
||||
@@ -16,9 +16,9 @@ Logging can be found in the *[Install Directory]\\PRIMAITE\\PRIMAITE\\logs* dire
|
||||
|
||||
**Outputs - Results**
|
||||
|
||||
PrimAITE automatically creates two sets of results from each training session, and stores them in the *Results* folder:
|
||||
PrimAITE automatically creates two sets of results from each session, and stores them in the *Results* folder:
|
||||
|
||||
* Average reward per episode - a csv file listing the average reward for each episode of the training session. This provides an indication of the change, over a training session, of the reward value
|
||||
* Average reward per episode - a csv file listing the average reward for each episode of the session. This provides, for example, an indication of the change over a training session of the reward value
|
||||
* All transactions - a csv file listing the following values for every step of every episode:
|
||||
|
||||
* Timestamp
|
||||
@@ -31,11 +31,11 @@ PrimAITE automatically creates two sets of results from each training session, a
|
||||
|
||||
**Outputs - Diagrams**
|
||||
|
||||
For each training run, PrimAITE automatically creates a visual of the system / network laydown configuration, and stores it in the *Diagrams* folder.
|
||||
For each session, PrimAITE automatically creates a visualisation of the system / network laydown configuration, and stores it in the *Diagrams* folder.
|
||||
|
||||
**Outputs - Saved agents**
|
||||
|
||||
For each training run, assuming the agent being trained implements the *save()* function and this function is called by the code, PrimAITE automatically saves the agent state and stores it in the *agents* folder.
|
||||
For each training session, assuming the agent being trained implements the *save()* function and this function is called by the code, PrimAITE automatically saves the agent state and stores it in the *agents* folder.
|
||||
|
||||
**Logging**
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
.. _training:
|
||||
.. _session:
|
||||
|
||||
Running a PrimAITE Training Session
|
||||
===================================
|
||||
Running a PrimAITE Training or Evaluation Session
|
||||
=================================================
|
||||
|
||||
A PrimAITE training session will usually be associated with a "Training Use Case Profile". This document will present:
|
||||
The application will determine whether a Training or Evaluation session is being executed via the 'sessionType' value in the config_mail.yaml file. A PrimAITE session will usually be associated with a "Use Case Profile"; this document will present:
|
||||
|
||||
* The Use Case name, default number of steps in a training episode and default number of episodes in a training session. The number of steps and episodes can be modified in the configuration files
|
||||
* The Use Case name, default number of steps in an episode and default number of episodes in a session. The number of steps and episodes can be modified in the configuration files
|
||||
* The system laydown being modelled
|
||||
* The objectives of the session (steady-state), the red agent and the blue agent (in a defensive role)
|
||||
* The green agent pattern-of-life profile
|
||||
@@ -40,7 +40,7 @@ Where:
|
||||
* *MyAgent* is the user created agent
|
||||
* *environment* is the PrimAITE environment
|
||||
* *max_steps* is the number of steps in an episode, as defined in the config_[name].yaml file
|
||||
* *num_episodes* is the number of episodes in the training session, as defined in the config_main.yaml file
|
||||
* *num_episodes* is the number of episodes in the session, as defined in the config_main.yaml file
|
||||
* the *.learn()* function should be defined in the user created agent
|
||||
* the *env.close()* function is defined within PrimAITE
|
||||
* the *save_agent()* assumes that a *save()* function has been defined in the user created agent. If not, this line can be ommitted (although it is encouraged, since it will allow the agent to be saved and ported)
|
||||
@@ -76,13 +76,13 @@ environment is reset between episodes. Note that the example below should not be
|
||||
if done == True:
|
||||
break
|
||||
|
||||
**Running the training session**
|
||||
**Running the session**
|
||||
|
||||
In order to execute a training session, carry out the following steps:
|
||||
In order to execute a session, carry out the following steps:
|
||||
|
||||
1. Navigate to "[Install directory]\\PRIMAITE\\PRIMAITE\\”
|
||||
2. Start a console window (type “CMD” in path window, or start a console window first and navigate to “[Install Directory]\\PRIMAITE\\PRIMAITE\\”)
|
||||
3. Type “python main.py”
|
||||
4. Training will start with an output indicating the current episode, and average reward value for the episode
|
||||
4. The session will start with an output indicating the current episode, and average reward value for the episode
|
||||
|
||||
|
||||
Reference in New Issue
Block a user