Update docs page on observations
This commit is contained in:
@@ -182,16 +182,13 @@ All ACL rules are considered when applying an IER. Logic follows the order of ru
|
|||||||
|
|
||||||
Observation Spaces
|
Observation Spaces
|
||||||
******************
|
******************
|
||||||
|
The observation space provides the blue agent with information about the current status of nodes and links.
|
||||||
|
|
||||||
The OpenAI Gym observation space provides the status of all nodes and links across the whole system:
|
PrimAITE builds on top of Gym Spaces to create an observation space that is easily configurable for users. It's made up of components which are managed by the :py:class:`primaite.environment.observations.ObservationHandler`. Each training scenario can define its own observation space, and the user can choose which information to inlude, and how it should be formatted.
|
||||||
|
|
||||||
* Nodes (in terms of hardware state, Software State, file system state and services state)
|
NodeLinkTable component
|
||||||
* Links (in terms of current loading for each service/protocol)
|
-----------------------
|
||||||
|
For example, the :py:class:`primaite.environment.observations.NodeLinkTable` component represents the status of nodes and links as a ``gym.spaces.Box`` with an example format shown below:
|
||||||
The observation space can be configured as a ``gym.spaces.Box`` or ``gym.spaces.MultiDiscrete``, by setting the ``OBSERVATIONS`` parameter in the laydown config.
|
|
||||||
|
|
||||||
Box-type observation space
|
|
||||||
--------------------------
|
|
||||||
|
|
||||||
An example observation space is provided below:
|
An example observation space is provided below:
|
||||||
|
|
||||||
@@ -249,8 +246,6 @@ An example observation space is provided below:
|
|||||||
- 5000
|
- 5000
|
||||||
- 0
|
- 0
|
||||||
|
|
||||||
The observation space is a 6 x 6 Box type (OpenAI Gym Space) in this example. This is made up from the node and link information detailed below.
|
|
||||||
|
|
||||||
For the nodes, the following values are represented:
|
For the nodes, the following values are represented:
|
||||||
|
|
||||||
* ID
|
* ID
|
||||||
@@ -290,9 +285,9 @@ For the links, the following statuses are represented:
|
|||||||
* SoftwareState = N/A
|
* SoftwareState = N/A
|
||||||
* Protocol = loading in bits/s
|
* Protocol = loading in bits/s
|
||||||
|
|
||||||
MultiDiscrete-type observation space
|
NodeStatus component
|
||||||
------------------------------------
|
----------------------
|
||||||
The MultiDiscrete observation space can be though of as a one-dimensional vector of discrete states, represented by integers.
|
This is a MultiDiscrete observation space that can be though of as a one-dimensional vector of discrete states, represented by integers.
|
||||||
The example above would have the following structure:
|
The example above would have the following structure:
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
@@ -301,9 +296,6 @@ The example above would have the following structure:
|
|||||||
node1_info
|
node1_info
|
||||||
node2_info
|
node2_info
|
||||||
node3_info
|
node3_info
|
||||||
link1_status
|
|
||||||
link2_status
|
|
||||||
link3_status
|
|
||||||
]
|
]
|
||||||
|
|
||||||
Each ``node_info`` contains the following:
|
Each ``node_info`` contains the following:
|
||||||
@@ -318,7 +310,25 @@ Each ``node_info`` contains the following:
|
|||||||
service2_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
service2_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED)
|
||||||
]
|
]
|
||||||
|
|
||||||
Each ``link_status`` is just a number from 0-4 representing the network load in relation to bandwidth.
|
In a network with three nodes and two services, the full observation space would have 15 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example:
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4])
|
||||||
|
|
||||||
|
LinkTrafficLevels
|
||||||
|
-----------------
|
||||||
|
This component is a MultiDiscrete space showing the traffic flow levels on the links in the network, after applying a threshold to convert it from a continuous to a discrete value.
|
||||||
|
The number of bins can be customised with 5 being the default. It has the following strucutre:
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
[
|
||||||
|
link1_status
|
||||||
|
link2_status
|
||||||
|
link3_status
|
||||||
|
]
|
||||||
|
|
||||||
|
Each ``link_status`` is a number from 0-4 representing the network load in relation to bandwidth.
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
|
|
||||||
@@ -328,12 +338,11 @@ Each ``link_status`` is just a number from 0-4 representing the network load in
|
|||||||
3 = high traffic (<100%)
|
3 = high traffic (<100%)
|
||||||
4 = max traffic/ overwhelmed (100%)
|
4 = max traffic/ overwhelmed (100%)
|
||||||
|
|
||||||
The full observation space would have 15 node-related elements and 3 link-related elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example:
|
If the network has three links, the full observation space would have 3 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example:
|
||||||
|
|
||||||
.. code-block::
|
.. code-block::
|
||||||
|
|
||||||
gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4,5,5,5])
|
gym.spaces.MultiDiscrete([5,5,5])
|
||||||
|
|
||||||
|
|
||||||
Action Spaces
|
Action Spaces
|
||||||
**************
|
**************
|
||||||
|
|||||||
Reference in New Issue
Block a user