oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

#2257: massive docs addition for config file

Browse Source
This commit is contained in:
Czar Echavez
2024-02-21 14:49:59 +00:00
parent 2e2d83c3e9
commit deb7a3aa9d
28 changed files with 1101 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/_static/firewall_acl.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

BIN
docs/_static/switched_p2p_network.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.0 KiB

19
docs/source/config.rst
View File

@@ -8,6 +8,25 @@ Primaite v3 config
PrimAITE uses a single configuration file to define everything needed to train and evaluate an RL policy in a custom cybersecurity scenario. This includes the configuration of the network, the scripted or trained agents that interact with the network, as well as settings that define how to perform training in Stable Baselines 3 or Ray RLLib.
The entire config is used by the ``PrimaiteSession`` object for users who wish to let PrimAITE handle the agent definition and training. If you wish to define custom agents and control the training loop yourself, you can use the config with the ``PrimaiteGame``, and ``PrimaiteGymEnv`` objects instead. That way, only the network configuration and agent setup parts of the config are used, and the training section is ignored.
Example Configuration Hierarchy
###############################
The top level configuration items in a configuration file is as follows
.. code-block:: yaml
training_config:
...
io_settings:
...
game:
...
agents:
...
simulation:
...
These are expanded upon in the Configurable items section below
Configurable items
##################

181
docs/source/configuration/agents.rst
View File

@@ -7,39 +7,168 @@
==========
Agents can be scripted (deterministic and stochastic), or controlled by a reinforcement learning algorithm. Not to be confused with an RL agent, the term agent here is used to refer to an entity that sends requests to the simulated network. In this part of the config, each agent's action space, observation space, and reward function can be defined. All three are defined in a modular way.
**type**: Specifies which class should be used for the agent. ``ProxyAgent`` is used for agents that receive instructions from an RL algorithm. Scripted agents like ``RedDatabaseCorruptingAgent`` and ``GreenWebBrowsingAgent`` generate their own behaviour.
``agents`` hierarchy
--------------------
**team**: Specifies if the agent is malicious (RED), benign (GREEN), or defensive (BLUE). Currently this value is not used for anything.
.. code-block:: yaml
**observation space:**
* ``type``: selects which python class from the ``primaite.game.agent.observation`` module is used for the overall observation structure.
* ``options``: allows configuring the chosen observation type. The ``UC2BlueObservation`` should be used for RL Agents.
* ``num_services_per_node``, ``num_folders_per_node``, ``num_files_per_folder``, ``num_nics_per_node`` all define the shape of the observation space. The size and shape of the obs space must remain constant, but the number of files, folders, ACL rules, and other components can change within an episode. Therefore padding is performed and these options set the size of the obs space.
* ``nodes``: list of nodes that will be present in this agent's observation space. The ``node_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config. Each node can also be configured with services, and files that should be monitored.
* ``links``: list of links that will be present in this agent's observation space. The ``link_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config.
* ``acl``: configure how the agent reads the access control list on the router in the simulation. ``router_node_ref`` is for selecting which router's ACL table should be used. ``ip_address_order`` sets the encoding of ip addresses as integers within the observation space.
agents:
- ref: red_agent_example
...
- ref: blue_agent_example
...
- ref: green_agent_example
team: GREEN
type: GreenWebBrowsingAgent
observation_space:
type: UC2GreenObservation
action_space:
action_list:
- type: DONOTHING
- type: NODE_APPLICATION_EXECUTE
options:
nodes:
- node_name: client_2
applications:
- application_name: WebBrowser
max_folders_per_node: 1
max_files_per_folder: 1
max_services_per_node: 1
max_applications_per_node: 1
reward_function:
reward_components:
- type: DUMMY
agent_settings:
start_settings:
start_step: 5
frequency: 4
variance: 3
flatten_obs: False
``ref``
-------
The reference to be used for the given agent.
``team``
--------
Specifies if the agent is malicious (``RED``), benign (``GREEN``), or defensive (``BLUE``). Currently this value is not used for anything other than for human readability in the configuration file.
``type``
--------
Specifies which class should be used for the agent. ``ProxyAgent`` is used for agents that receive instructions from an RL algorithm. Scripted agents like ``RedDatabaseCorruptingAgent`` and ``GreenWebBrowsingAgent`` generate their own behaviour.
Available agent types:
- ``GreenWebBrowsingAgent``
- ``ProxyAgent``
- ``RedDatabaseCorruptingAgent``
``observation_space``
---------------------
Defines the observation space of the agent.
``type``
^^^^^^^^
selects which python class from the :py:mod:`primaite.game.agent.observation` module is used for the overall observation structure.
``options``
^^^^^^^^^^^
Allows configuration of the chosen observation type. These are optional.
* ``num_services_per_node``, ``num_folders_per_node``, ``num_files_per_folder``, ``num_nics_per_node`` all define the shape of the observation space. The size and shape of the obs space must remain constant, but the number of files, folders, ACL rules, and other components can change within an episode. Therefore padding is performed and these options set the size of the obs space.
* ``nodes``: list of nodes that will be present in this agent's observation space. The ``node_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config. Each node can also be configured with services, and files that should be monitored.
* ``links``: list of links that will be present in this agent's observation space. The ``link_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config.
* ``acl``: configure how the agent reads the access control list on the router in the simulation. ``router_node_ref`` is for selecting which router's ACL table should be used. ``ip_address_order`` sets the encoding of ip addresses as integers within the observation space.
For more information see :py:mod:`primaite.game.agent.observations`
``action_space``
----------------
**action space:**
The action space is configured to be made up of individual action types. Once configured, the agent can select an action type and some optional action parameters at every step. For example: The ``NODE_SERVICE_SCAN`` action takes the parameters ``node_id`` and ``service_id``.
Description of configurable items:
* ``action_list``: a list of action modules. The options are listed in the ``primaite.game.agent.actions`` module.
* ``action_map``: (optional). Restricts the possible combinations of action type / action parameter values to reduce the overall size of the action space. By default, every possible combination of actions and parameters will be assigned an integer for the agent's ``MultiDiscrete`` action space. Instead, the ``action_map`` allows you to list the actions corresponding to each integer in the ``MultiDiscrete`` space.
* ``options``: Options that apply too all action components.
* ``nodes``: list the nodes that the agent can act on, the order of this list defines the mapping between nodes and ``node_id`` integers.
* ``max_folders_per_node``, ``max_files_per_folder``, ``max_services_per_node``, ``max_nics_per_node``, ``max_acl_rules`` all are used to define the size of the action space.
``action_list``
^^^^^^^^^^^^^^^
**reward function:**
Similar to action space, this is defined as a list of components.
A list of action modules. The options are listed in the :py:mod:`primaite.game.agent.actions.ActionManager.act_class_identifiers` module.
Description of configurable items:
* ``reward_components`` a list of reward components from the ``primaite.game.agent.reward`` module.
* ``weight``: relative importance of this reward component. The total reward for a step is a weighted sum of all reward components.
* ``options``: list of options passed to the reward component during initialisation, the exact options required depend on the reward component.
``action_map``
^^^^^^^^^^^^^^
**agent_settings**:
Settings passed to the agent during initialisation. These depend on the agent class.
Restricts the possible combinations of action type / action parameter values to reduce the overall size of the action space. By default, every possible combination of actions and parameters will be assigned an integer for the agent's ``MultiDiscrete`` action space. Instead, the ``action_map`` allows you to list the actions corresponding to each integer in the ``MultiDiscrete`` space.
Reinforcement learning agents use the ``ProxyAgent`` class, they accept these agent settings:
This is Optional.
**flatten_obs**: If true, gymnasium flattening will be performed on the observation space before sending to the agent. Set this to true if your agent does not support nested observation spaces.
``options``
^^^^^^^^^^^
Options that apply to all action components. These are optional.
* ``nodes``: list the nodes that the agent can act on, the order of this list defines the mapping between nodes and ``node_id`` integers.
* ``max_folders_per_node``, ``max_files_per_folder``, ``max_services_per_node``, ``max_nics_per_node``, ``max_acl_rules`` all are used to define the size of the action space.
For more information see :py:mod:`primaite.game.agent.actions`
``reward_function``
-------------------
Similar to action space, this is defined as a list of components from the :py:mod:`primaite.game.agent.rewards` module.
``reward_components``
^^^^^^^^^^^^^^^^^^^^^
A list of reward types from :py:mod:`primaite.game.agent.rewards.RewardFunction.rew_class_identifiers`
e.g.
.. code-block:: yaml
reward_components:
- type: DUMMY
- type: DATABASE_FILE_INTEGRITY
``agent_settings``
------------------
Settings passed to the agent during initialisation. Determines how the agent will behave during training.
e.g.
.. code-block:: yaml
agent_settings:
start_settings:
start_step: 25
frequency: 20
variance: 5
``start_step``
^^^^^^^^^^^^^^
Optional. Default value is ``5``.
The timestep where the agent begins performing actions.
``frequency``
^^^^^^^^^^^^^
Optional. Default value is ``5``.
The number of timesteps the agent will wait before performing another action.
``variance``
^^^^^^^^^^^^
Optional. Default value is ``0``.
The amount of timesteps that the frequency can randomly change.
``flatten_obs``
---------------
If ``True``, gymnasium flattening will be performed on the observation space before sending to the agent. Set this to ``True`` if your agent does not support nested observation spaces.

38
docs/source/configuration/game.rst
View File

@@ -6,3 +6,41 @@
``game``
========
This section defines high-level settings that apply across the game, currently it's used to help shape the action and observation spaces by restricting which ports and internet protocols should be considered. Here, users can also set the maximum number of steps in an episode.
``game`` hierarchy
------------------
.. code-block:: yaml
game:
max_episode_length: 256
ports:
- ARP
- DNS
- HTTP
- POSTGRES_SERVER
protocols:
- ICMP
- TCP
- UDP
``max_episode_length``
----------------------
Optional. Default value is ``256``.
The maximum number of episodes a Reinforcement Learning agent(s) can be trained for.
``ports``
---------
A list of ports that the Reinforcement Learning agent(s) are able to see in the observation space.
See :ref:`List of Ports <List of Ports>` for a list of ports.
``protocols``
-------------
A list of protocols that the Reinforcement Learning agent(s) are able to see in the observation space.
See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.

81
docs/source/configuration/io_settings.rst
View File

@@ -7,20 +7,83 @@
===============
This section configures how PrimAITE saves data during simulation and training.
**save_final_model**: Only used if training with PrimaiteSession, if true, the policy will be saved after the final training iteration.
``io_settings`` hierarchy
-------------------------
**save_checkpoints**: Only used if training with PrimaiteSession, if true, the policy will be saved periodically during training.
.. code-block:: yaml
**checkpoint_interval**: Only used if training with PrimaiteSession and if ``save_checkpoints`` is true. Defines how often to save the policy during training.
io_settings:
save_final_model: True
save_checkpoints: False
checkpoint_interval: 10
# save_logs: True
# save_transactions: False
# save_tensorboard_logs: False
save_step_metadata: False
save_pcap_logs: False
save_sys_logs: False
**save_logs**: *currently unused*.
``save_final_model``
--------------------
**save_transactions**: *currently unused*.
Optional. Default value is ``True``.
**save_tensorboard_logs**: *currently unused*.
Only used if training with PrimaiteSession.
If ``True``, the policy will be saved after the final training iteration.
**save_step_metadata**: Whether to save the RL agents' action, environment state, and other data at every single step.
**save_pcap_logs**: Whether to save pcap files of all network traffic during the simulation.
``save_checkpoints``
--------------------
**save_sys_logs**: Whether to save system logs from all nodes during the simulation.
Optional. Default value is ``False``.
Only used if training with PrimaiteSession.
If ``True``, the policy will be saved periodically during training.
``checkpoint_interval``
-----------------------
Optional. Default value is ``10``.
Only used if training with PrimaiteSession and if ``save_checkpoints`` is ``True``.
Defines how often to save the policy during training.
``save_logs``
-------------
*currently unused*.
``save_transactions``
---------------------
*currently unused*.
``save_tensorboard_logs``
-------------------------
*currently unused*.
``save_step_metadata``
----------------------
Optional. Default value is ``False``.
If ``True``, The RL agent(s) actions, environment states and other data will be saved at every single step.
``save_pcap_logs``
------------------
Optional. Default value is ``False``.
If ``True``, then the pcap files which contain all network traffic during the simulation will be saved.
``save_sys_logs``
-----------------
Optional. Default value is ``False``.
If ``True``, then the log files which contain all node actions during the simulation will be saved.

89
docs/source/configuration/simulation.rst
View File

@@ -9,6 +9,17 @@ In this section the network layout is defined. This part of the config follows a
At the top level of the network are ``nodes`` and ``links``.
e.g.
.. code-block:: yaml
simulation:
network:
nodes:
...
links:
...
**nodes:**
* ``type``: one of ``router``, ``switch``, ``computer``, or ``server``, this affects what other sub-options should be defined.
* ``hostname`` - a non-unique name used for logging and outputs.
@@ -19,9 +30,75 @@ At the top level of the network are ``nodes`` and ``links``.
* ``applications`` (computer and servers only): Similar to services. A list of application to install on the node.
* ``network_interfaces`` (computers and servers only): If the node has multiple networking devices, the second, third, fourth, etc... must be defined here with an ``ip_address`` and ``subnet_mask``.
**links:**
* ``ref``: unique identifier for this link
* ``endpoint_a_ref``: Reference to the node at the first end of the link
* ``endpoint_a_port``: The ethernet port or switch port index of the second node
* ``endpoint_b_ref``: Reference to the node at the second end of the link
* ``endpoint_b_port``: The ethernet port or switch port index on the second node
``nodes``
---------
This is where the list of nodes are defined. Some items will differ according to the node type, however, there will be common items such as a node's reference (which is used by the agent), the node's ``type`` and ``hostname``
To see the configuration for these nodes, refer to the following:
.. toctree::
:maxdepth: 1
simulation/nodes/computer.rst
simulation/nodes/firewall.rst
simulation/nodes/router.rst
simulation/nodes/server.rst
simulation/nodes/switch.rst
``links``
---------
This is where the links between the nodes are formed.
e.g.
In order to recreate the network below, we will need to create 2 links:
- a link from computer_1 to the switch
- a link from computer_2 to the switch
.. image:: ../../_static/switched_p2p_network.png
this results in:
.. code-block:: yaml
links:
- ref: computer_1___switch
endpoint_a_ref: computer_1
endpoint_a_port: 1 # port 1 on computer_1
endpoint_b_ref: switch
endpoint_b_port: 1 # port 1 on switch
- ref: computer_2___switch
endpoint_a_ref: computer_2
endpoint_a_port: 1 # port 1 on computer_2
endpoint_b_ref: switch
endpoint_b_port: 2 # port 2 on switch
``ref``
^^^^^^^
The human readable name for the link. Not used in code, however is useful for a human to understand what the link is for.
``endpoint_a_ref``
^^^^^^^^^^^^^^^^^^
The name of the node which must be connected.
``endpoint_a_port``
^^^^^^^^^^^^^^^^^^^
The port on ``endpoint_a_ref`` which is to be connected to ``endpoint_b_port``.
This accepts an integer value e.g. if port 1 is to be connected, the configuration should be ``endpoint_a_port: 1``
``endpoint_b_ref``
^^^^^^^^^^^^^^^^^^
The name of the node which must be connected.
``endpoint_b_port``
^^^^^^^^^^^^^^^^^^^
The port on ``endpoint_b_ref`` which is to be connected to ``endpoint_a_port``.
This accepts an integer value e.g. if port 1 is to be connected, the configuration should be ``endpoint_b_port: 1``

41
docs/source/configuration/simulation/nodes/common/common_host_node_attributes.rst Normal file
View File

@@ -0,0 +1,41 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``ip_address``
^^^^^^^^^^^^^^
The IP address of the |NODE| in the network.
``subnet_mask``
^^^^^^^^^^^^^^^
Optional. Default value is ``255.255.255.0``.
The subnet mask for the |NODE| to use.
``default_gateway``
^^^^^^^^^^^^^^^^^^^
The IP address that the |NODE| will use as the default gateway. Typically, this is the IP address of the closest router that the |NODE| is connected to.
``dns_server``
^^^^^^^^^^^^^^
Optional. Default value is ``None``
The IP address of the node which holds an instance of the DNS server. Some applications may use a domain name e.g. the WebBrowser (TODO: WebBrowser page)
``applications``
^^^^^^^^^^^^^^^^
A list of applications which are not considered system software that need to be installed on the |NODE|.
See :ref:`Applications <applications_config>`
``services``
^^^^^^^^^^^^
A list of services which are not considered system software that need to be installed on the |NODE|.
See :ref:`Services <services_config>`

49
docs/source/configuration/simulation/nodes/common/common_network_node_attributes.rst Normal file
View File

@@ -0,0 +1,49 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``routes``
----------
A list of routes which tells the |NODE| where to forward the packet to depending on the target IP address.
e.g.
.. code-block:: yaml
nodes:
- ref: node
...
routes:
- address: 192.168.0.10
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.1
metric: 0
``address``
"""""""""""
The target IP address for the route. If the packet destination IP address matches this, the router will route the packet according to the ``next_hop_ip_address``.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the route.
``next_hop_ip_address``
"""""""""""""""""""""""
The IP address of the next hop IP address that the packet will follow if the address matches the packet's destination IP address.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``metric``
""""""""""
Optional. Default value is ``0``. This value accepts floats.
The cost or distance of a route. The higher the value, the more cost or distance is attributed to the route.

13
docs/source/configuration/simulation/nodes/common/common_node_attributes.rst Normal file
View File

@@ -0,0 +1,13 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``ref``
-------
Human readable name used as reference for the |NODE|. Not used in code.
``hostname``
------------
The hostname of the |NODE|. This will be used to reference the |NODE|.

18
docs/source/configuration/simulation/nodes/common/node_type_list.rst Normal file
View File

@@ -0,0 +1,18 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``type``
--------
The type of node to add.
Available options are:
- ``computer``
- ``firewall``
- ``router``
- ``server``
- ``switch``
To create a |NODE|, type must be |NODE_TYPE|.

39
docs/source/configuration/simulation/nodes/computer.rst Normal file
View File

@@ -0,0 +1,39 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _computer_configuration:
``computer``
============
A basic representation of a computer within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.host.computer.Computer`
example computer
----------------
.. code-block:: yaml
nodes:
- ref: client_1
hostname: client_1
type: computer
ip_address: 192.168.0.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.0.1
dns_server: 192.168.1.10
applications:
...
services:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
.. include:: common/common_host_node_attributes.rst
.. |NODE| replace:: computer
.. |NODE_TYPE| replace:: ``computer``

258
docs/source/configuration/simulation/nodes/firewall.rst Normal file
View File

@@ -0,0 +1,258 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _firewall_configuration:
``firewall``
============
A basic representation of a network router within the simulation.
The firewall is similar to how :ref:`Router <router_configuration>` works, with the difference being how firewall has specific ACL rules for inbound and outbound traffic as well as firewall being limited to 3 ports.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.firewall.Firewall`
example firewall
----------------
.. code-block:: yaml
nodes:
- ref: firewall
hostname: firewall
type: firewall
start_up_duration: 0
shut_down_duration: 0
ports:
external_port: # port 1
ip_address: 192.168.20.1
subnet_mask: 255.255.255.0
internal_port: # port 2
ip_address: 192.168.1.2
subnet_mask: 255.255.255.0
dmz_port: # port 3
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
acl:
internal_inbound_acl:
...
internal_outbound_acl:
...
dmz_inbound_acl:
...
dmz_outbound_acl:
...
external_inbound_acl:
...
external_outbound_acl:
...
routes:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``ports``
---------
The firewall node only has 3 ports. These specifically are:
- ``external_port`` (port 1)
- ``internal_port`` (port 2)
- ``dmz_port`` (port 3) (can be optional)
The ports should be defined with an ip address and subnet mask e.g.
.. code-block:: yaml
nodes:
- ref: firewall
...
ports:
external_port: # port 1
ip_address: 192.168.20.1
subnet_mask: 255.255.255.0
internal_port: # port 2
ip_address: 192.168.1.2
subnet_mask: 255.255.255.0
dmz_port: # port 3
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
``ip_address``
""""""""""""""
The IP address for the given port. This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the port.
``acl``
-------
There are 6 ACLs that can be defined for a firewall
- ``internal_inbound_acl`` for traffic going towards the internal network
- ``internal_outbound_acl`` for traffic coming from the internal network
- ``dmz_inbound_acl`` for traffic going towards the dmz network
- ``dmz_outbound_acl`` for traffic coming from the dmz network
- ``external_inbound_acl`` for traffic coming from the external network
- ``external_outbound_acl`` for traffic going towards the external network
.. image:: ../../../../_static/firewall_acl.png
By default, ``external_inbound_acl`` and ``external_outbound_acl`` will permit any traffic through.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.AccessControlList`
See :ref:`List of Ports <List of Ports>` for a list of ports.
``internal_inbound_acl``
""""""""""""""""""""""""
ACL rules for packets that have a destination IP address in what is considered the internal network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
internal_inbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``internal_outbound_acl``
"""""""""""""""""""""""""
ACL rules for packets that have a source IP address in what is considered the internal network and is going towards the DMZ network or the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
internal_outbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``dmz_inbound_acl``
"""""""""""""""""""
ACL rules for packets that have a destination IP address in what is considered the DMZ network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
dmz_inbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``dmz_outbound_acl``
""""""""""""""""""""
ACL rules for packets that have a source IP address in what is considered the DMZ network and is going towards the internal network or the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
dmz_outbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``external_inbound_acl``
""""""""""""""""""""""""
Optional. By default, this will allow any traffic through.
ACL rules for packets that have a destination IP address in what is considered the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
external_inbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``external_outbound_acl``
"""""""""""""""""""""""""
Optional. By default, this will allow any traffic through.
ACL rules for packets that have a source IP address in what is considered the external network and is going towards the DMZ network or the internal network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
external_outbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
.. include:: common/common_network_node_attributes.rst
.. |NODE| replace:: firewall
.. |NODE_TYPE| replace:: ``firewall``

125
docs/source/configuration/simulation/nodes/router.rst Normal file
View File

@@ -0,0 +1,125 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _router_configuration:
``router``
==========
A basic representation of a network router within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.Router`
example router
--------------
.. code-block:: yaml
nodes:
- ref: router_1
hostname: router_1
type: router
num_ports: 5
ports:
...
acl:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``num_ports``
-------------
Optional. Default value is ``5``.
The number of ports the router will have.
``ports``
---------
Sets up the router's ports with an IP address and a subnet mask.
Example of setting ports for a router with 2 ports:
.. code-block:: yaml
nodes:
- ref: router_1
...
ports:
1:
ip_address: 192.168.1.1
subnet_mask: 255.255.255.0
2:
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
``ip_address``
""""""""""""""
The IP address for the given port. This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the port.
``acl``
-------
Sets up the ACL rules for the router.
e.g.
.. code-block:: yaml
nodes:
- ref: router_1
...
acl:
1:
action: PERMIT
src_port: ARP
dst_port: ARP
2:
action: PERMIT
protocol: ICMP
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.AccessControlList`
See :ref:`List of Ports <List of Ports>` for a list of ports.
``action``
""""""""""
Available options are
- ``PERMIT`` : Allows the specified ``protocol`` or ``src_port`` and ``dst_port`` pairs
- ``DENY`` : Blocks the specified ``protocol`` or ``src_port`` and ``dst_port`` pairs
``src_port``
""""""""""""
Is used alongside ``dst_port``. Specifies the port where a packet originates. Used by the ACL Rule to determine if a packet with a specific source port is allowed to pass through the network node.
``dst_port``
""""""""""""
Is used alongside ``src_port``. Specifies the port where a packet is destined to arrive. Used by the ACL Rule to determine if a packet with a specific destination port is allowed to pass through the network node.
``protocol``
""""""""""""
Specifies which protocols are allowed by the ACL Rule to pass through the network node.
See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.
.. include:: common/common_network_node_attributes.rst
.. |NODE| replace:: router
.. |NODE_TYPE| replace:: ``router``

39
docs/source/configuration/simulation/nodes/server.rst Normal file
View File

@@ -0,0 +1,39 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _server_configuration:
``server``
==========
A basic representation of a server within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.host.server.Server`
example server
--------------
.. code-block:: yaml
nodes:
- ref: server_1
hostname: server_1
type: server
ip_address: 192.168.10.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
dns_server: 192.168.1.10
applications:
...
services:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
.. include:: common/common_host_node_attributes.rst
.. |NODE| replace:: server
.. |NODE_TYPE| replace:: ``server``

37
docs/source/configuration/simulation/nodes/switch.rst Normal file
View File

@@ -0,0 +1,37 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _switch_configuration:
``switch``
==========
A basic representation of a network switch within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.switch.Switch`
example switch
--------------
.. code-block:: yaml
nodes:
- ref: switch_1
hostname: switch_1
type: switch
num_ports: 8
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``num_ports``
-------------
Optional. Default value is ``8``.
The number of ports the switch will have.
.. |NODE| replace:: switch
.. |NODE_TYPE| replace:: ``switch``

10
docs/source/configuration/simulation/software/applications.rst Normal file
View File

@@ -0,0 +1,10 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _applications_config:
``applications``
================
apps

10
docs/source/configuration/simulation/software/services.rst Normal file
View File

@@ -0,0 +1,10 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _services_config:
``services``
============
services

50
docs/source/configuration/training_config.rst
View File

@@ -4,6 +4,22 @@
``training_config``
===================
Configuration items relevant to how the Reinforcement Learning agent(s) will be trained.
``training_config`` hierarchy
-----------------------------
.. code-block:: yaml
training_config:
rl_framework: SB3 # or RLLIB_single_agent or RLLIB_multi_agent
rl_algorithm: PPO # or A2C
n_learn_episodes: 5
max_steps_per_episode: 200
n_eval_episodes: 1
deterministic_eval: True
seed: 123
``rl_framework``
----------------
@@ -23,3 +39,37 @@ Options available are:
- ``PPO`` (Proximal Policy Optimisation)
- ``A2C`` (Advantage Actor Critic)
``n_learn_episodes``
--------------------
The number of episodes to train the agent(s).
This should be an integer value above ``0``
``max_steps_per_episode``
-------------------------
The number of steps each episode will last for.
This should be an integer value above ``0``.
``n_eval_episodes``
-------------------
Optional. Default value is ``0``.
The number of evaluation episodes to run the trained agent for.
This should be an integer value above ``0``.
``deterministic_eval``
----------------------
Optional. By default this value is ``False``.
If this is set to ``True``, the agents will act deterministically instead of stochastically.
``seed``
--------
Optional.
The seed is used (alongside ``deterministic_eval``) to reproduce a previous instance of training and evaluation of an RL agent.
The seed should be an integer value.
Useful for debugging.

20
src/primaite/config/_package_data/example_config.yaml
View File

@@ -583,8 +583,8 @@ simulation:
nodes:
- ref: router_1
type: router
hostname: router_1
type: router
num_ports: 5
ports:
1:
@@ -619,18 +619,18 @@ simulation:
protocol: ICMP
- ref: switch_1
type: switch
hostname: switch_1
type: switch
num_ports: 8
- ref: switch_2
type: switch
hostname: switch_2
type: switch
num_ports: 8
- ref: domain_controller
type: server
hostname: domain_controller
type: server
ip_address: 192.168.1.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -642,8 +642,8 @@ simulation:
arcd.com: 192.168.1.12 # web server
- ref: web_server
type: server
hostname: web_server
type: server
ip_address: 192.168.1.12
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -658,8 +658,8 @@ simulation:
- ref: database_server
type: server
hostname: database_server
type: server
ip_address: 192.168.1.14
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -673,8 +673,8 @@ simulation:
type: FTPClient
- ref: backup_server
type: server
hostname: backup_server
type: server
ip_address: 192.168.1.16
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -684,8 +684,8 @@ simulation:
type: FTPServer
- ref: security_suite
type: server
hostname: security_suite
type: server
ip_address: 192.168.1.110
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -696,8 +696,8 @@ simulation:
subnet_mask: 255.255.255.0
- ref: client_1
type: computer
hostname: client_1
type: computer
ip_address: 192.168.10.21
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
@@ -719,8 +719,8 @@ simulation:
type: DNSClient
- ref: client_2
type: computer
hostname: client_2
type: computer
ip_address: 192.168.10.22
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1

4
src/primaite/game/agent/actions.py
View File

@@ -572,7 +572,7 @@ class NetworkNICDisableAction(NetworkNICAbstractAction):
class ActionManager:
"""Class which manages the action space for an agent."""
_act_class_identifiers: Dict[str, type] = {
act_class_identifiers: Dict[str, type] = {
"DONOTHING": DoNothingAction,
"NODE_SERVICE_SCAN": NodeServiceScanAction,
"NODE_SERVICE_STOP": NodeServiceStopAction,
@@ -753,7 +753,7 @@ class ActionManager:
# and `options` is an optional dict of options to pass to the init method of the action class
act_type = act_spec.get("type")
act_options = act_spec.get("options", {})
self.actions[act_type] = self._act_class_identifiers[act_type](self, **global_action_args, **act_options)
self.actions[act_type] = self.act_class_identifiers[act_type](self, **global_action_args, **act_options)
self.action_map: Dict[int, Tuple[str, Dict]] = {}
"""

5
src/primaite/game/agent/rewards.py
View File

@@ -245,12 +245,13 @@ class WebpageUnavailablePenalty(AbstractReward):
class RewardFunction:
"""Manages the reward function for the agent."""
__rew_class_identifiers: Dict[str, Type[AbstractReward]] = {
rew_class_identifiers: Dict[str, Type[AbstractReward]] = {
"DUMMY": DummyReward,
"DATABASE_FILE_INTEGRITY": DatabaseFileIntegrity,
"WEB_SERVER_404_PENALTY": WebServer404Penalty,
"WEBPAGE_UNAVAILABLE_PENALTY": WebpageUnavailablePenalty,
}
"""List of reward class identifiers."""
def __init__(self):
"""Initialise the reward function object."""
@@ -297,7 +298,7 @@ class RewardFunction:
for rew_component_cfg in config["reward_components"]:
rew_type = rew_component_cfg["type"]
weight = rew_component_cfg.get("weight", 1.0)
rew_class = cls.__rew_class_identifiers[rew_type]
rew_class = cls.rew_class_identifiers[rew_type]
rew_instance = rew_class.from_config(config=rew_component_cfg.get("options", {}))
new.register_component(component=rew_instance, weight=weight)
return new

10
src/primaite/game/game.py
View File

@@ -231,24 +231,24 @@ class PrimaiteGame:
new_node = Computer(
hostname=node_cfg["hostname"],
ip_address=node_cfg["ip_address"],
subnet_mask=node_cfg["subnet_mask"],
subnet_mask=IPv4Address(node_cfg.get("subnet_mask", "255.255.255.0")),
default_gateway=node_cfg["default_gateway"],
dns_server=node_cfg["dns_server"],
dns_server=node_cfg.get("dns_server", None),
operating_state=NodeOperatingState.ON,
)
elif n_type == "server":
new_node = Server(
hostname=node_cfg["hostname"],
ip_address=node_cfg["ip_address"],
subnet_mask=node_cfg["subnet_mask"],
subnet_mask=IPv4Address(node_cfg.get("subnet_mask", "255.255.255.0")),
default_gateway=node_cfg["default_gateway"],
dns_server=node_cfg.get("dns_server"),
dns_server=node_cfg.get("dns_server", None),
operating_state=NodeOperatingState.ON,
)
elif n_type == "switch":
new_node = Switch(
hostname=node_cfg["hostname"],
num_ports=node_cfg.get("num_ports"),
num_ports=int(node_cfg.get("num_ports", "8")),
operating_state=NodeOperatingState.ON,
)
elif n_type == "router":

12
src/primaite/simulator/network/hardware/nodes/network/firewall.py
View File

@@ -506,22 +506,24 @@ class Firewall(Router):
# configure internal port
new.configure_internal_port(
ip_address=IPV4Address(internal_port.get("ip_address")),
subnet_mask=IPV4Address(internal_port.get("subnet_mask")),
subnet_mask=IPV4Address(internal_port.get("subnet_mask", "255.255.255.0")),
)
# configure external port
new.configure_external_port(
ip_address=IPV4Address(external_port.get("ip_address")),
subnet_mask=IPV4Address(external_port.get("subnet_mask")),
subnet_mask=IPV4Address(external_port.get("subnet_mask", "255.255.255.0")),
)
# configure dmz port
new.configure_dmz_port(
ip_address=IPV4Address(dmz_port.get("ip_address")), subnet_mask=IPV4Address(dmz_port.get("subnet_mask"))
ip_address=IPV4Address(dmz_port.get("ip_address")),
subnet_mask=IPV4Address(dmz_port.get("subnet_mask", "255.255.255.0")),
)
if "acl" in cfg:
# acl rules for internal_inbound_acl
if cfg["acl"]["internal_inbound_acl"]:
new.internal_inbound_acl.max_acl_rules
new.internal_inbound_acl._default_config = cfg["acl"]["internal_inbound_acl"]
new.internal_inbound_acl._reset_rules_to_default()
@@ -553,8 +555,8 @@ class Firewall(Router):
for route in cfg.get("routes"):
new.route_table.add_route(
address=IPv4Address(route.get("address")),
subnet_mask=IPv4Address(route.get("subnet_mask")),
subnet_mask=IPv4Address(route.get("subnet_mask", "255.255.255.0")),
next_hop_ip_address=IPv4Address(route.get("next_hop_ip_address")),
metric=float(route.get("metric")),
metric=float(route.get("metric", 0)),
)
return new

8
src/primaite/simulator/network/hardware/nodes/network/router.py
View File

@@ -1482,7 +1482,7 @@ class Router(NetworkNode):
"""
new = Router(
hostname=cfg["hostname"],
num_ports=cfg.get("num_ports"),
num_ports=int(cfg.get("num_ports", "5")),
operating_state=NodeOperatingState.ON,
)
if "ports" in cfg:
@@ -1490,7 +1490,7 @@ class Router(NetworkNode):
new.configure_port(
port=port_num,
ip_address=port_cfg["ip_address"],
subnet_mask=port_cfg["subnet_mask"],
subnet_mask=IPv4Address(port_cfg.get("subnet_mask", "255.255.255.0")),
)
if "acl" in cfg:
new.acl._default_config = cfg["acl"] # save the config to allow resetting
@@ -1499,8 +1499,8 @@ class Router(NetworkNode):
for route in cfg.get("routes"):
new.route_table.add_route(
address=IPv4Address(route.get("address")),
subnet_mask=IPv4Address(route.get("subnet_mask")),
subnet_mask=IPv4Address(route.get("subnet_mask", "255.255.255.0")),
next_hop_ip_address=IPv4Address(route.get("next_hop_ip_address")),
metric=float(route.get("metric")),
metric=float(route.get("metric", 0)),
)
return new

9
src/primaite/simulator/network/transmission/network_layer.py
View File

@@ -9,11 +9,18 @@ _LOGGER = getLogger(__name__)
class IPProtocol(Enum):
"""Enum representing transport layer protocols in IP header."""
"""
Enum representing transport layer protocols in IP header.
.. _List of IPProtocols:
"""
TCP = "tcp"
"""Transmission Control Protocol."""
UDP = "udp"
"""User Datagram Protocol."""
ICMP = "icmp"
"""Internet Control Message Protocol."""
class Precedence(Enum):

6
src/primaite/simulator/network/transmission/transport_layer.py
View File

@@ -5,7 +5,11 @@ from pydantic import BaseModel
class Port(Enum):
"""Enumeration of common known TCP/UDP ports used by protocols for operation of network applications."""
"""
Enumeration of common known TCP/UDP ports used by protocols for operation of network applications.
.. _List of Ports:
"""
NONE = 0
"Place holder for a non-port."

2
tests/assets/configs/dmz_network.yaml
View File

@@ -20,7 +20,7 @@
# . ---------------- -------------- -------------- .
# . | dmz_server |------| switch_2 |------| firewall | .
# . ---------------- -------------- -------------- .
# . (Computer) | .
# . (Server) | .
# ........................................................|...................
# |
# External Network |