From ac31c996a740ceb3b9314fe4c6b9493285bdc98a Mon Sep 17 00:00:00 2001 From: Marek Wolan Date: Thu, 1 Jun 2023 21:42:34 +0100 Subject: [PATCH] Update docs page on observations --- docs/source/about.rst | 49 +++++++++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 20 deletions(-) diff --git a/docs/source/about.rst b/docs/source/about.rst index 8cc08b13..ee84d880 100644 --- a/docs/source/about.rst +++ b/docs/source/about.rst @@ -182,16 +182,13 @@ All ACL rules are considered when applying an IER. Logic follows the order of ru Observation Spaces ****************** +The observation space provides the blue agent with information about the current status of nodes and links. -The OpenAI Gym observation space provides the status of all nodes and links across the whole system:​ +PrimAITE builds on top of Gym Spaces to create an observation space that is easily configurable for users. It's made up of components which are managed by the :py:class:`primaite.environment.observations.ObservationHandler`. Each training scenario can define its own observation space, and the user can choose which information to inlude, and how it should be formatted. -* Nodes (in terms of hardware state, Software State, file system state and services state) ​ -* Links (in terms of current loading for each service/protocol) - -The observation space can be configured as a ``gym.spaces.Box`` or ``gym.spaces.MultiDiscrete``, by setting the ``OBSERVATIONS`` parameter in the laydown config. - -Box-type observation space --------------------------- +NodeLinkTable component +----------------------- +For example, the :py:class:`primaite.environment.observations.NodeLinkTable` component represents the status of nodes and links as a ``gym.spaces.Box`` with an example format shown below: An example observation space is provided below: @@ -249,8 +246,6 @@ An example observation space is provided below: - 5000 - 0 -The observation space is a 6 x 6 Box type (OpenAI Gym Space) in this example. This is made up from the node and link information detailed below. - For the nodes, the following values are represented: * ID @@ -290,9 +285,9 @@ For the links, the following statuses are represented: * SoftwareState = N/A * Protocol = loading in bits/s -MultiDiscrete-type observation space ------------------------------------- -The MultiDiscrete observation space can be though of as a one-dimensional vector of discrete states, represented by integers. +NodeStatus component +---------------------- +This is a MultiDiscrete observation space that can be though of as a one-dimensional vector of discrete states, represented by integers. The example above would have the following structure: .. code-block:: @@ -301,9 +296,6 @@ The example above would have the following structure: node1_info node2_info node3_info - link1_status - link2_status - link3_status ] Each ``node_info`` contains the following: @@ -318,7 +310,25 @@ Each ``node_info`` contains the following: service2_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED) ] -Each ``link_status`` is just a number from 0-4 representing the network load in relation to bandwidth. +In a network with three nodes and two services, the full observation space would have 15 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example: + +.. code-block:: + + gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4]) + +LinkTrafficLevels +----------------- +This component is a MultiDiscrete space showing the traffic flow levels on the links in the network, after applying a threshold to convert it from a continuous to a discrete value. +The number of bins can be customised with 5 being the default. It has the following strucutre: +.. code-block:: + + [ + link1_status + link2_status + link3_status + ] + +Each ``link_status`` is a number from 0-4 representing the network load in relation to bandwidth. .. code-block:: @@ -328,12 +338,11 @@ Each ``link_status`` is just a number from 0-4 representing the network load in 3 = high traffic (<100%) 4 = max traffic/ overwhelmed (100%) -The full observation space would have 15 node-related elements and 3 link-related elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example: +If the network has three links, the full observation space would have 3 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example: .. code-block:: - gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4,5,5,5]) - + gym.spaces.MultiDiscrete([5,5,5]) Action Spaces **************