Merged PR 280: Router Routes and Configuration documentation updates

## Summary

- Updated Sphinx 6.1.3 => 7.1.2
- Updated furo 2023.3.27 => 2024.01.29
- Added tests to check that firewall, routers and nodes are properly added via config (config parser tests)
- Added some reference points in code comments for Sphinx documentation to reference
- Created a list of System Software so it can be referenced in docs and by code
- Added default values to config within game.py (The defaults are pulled from example_config.yaml)
- Created a section for creating a session via config files:
    - set up a lot of stuff so documentation is easier to maintain
    - Template for things that are repeated in places
    - added how to create nodes via config
    - added how to install applications and services via config
- diagrams to make it easier to understand some stuff e.g. ACL rules for firewall

see https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/CHANGELOG.md

## Test process
- Firewall:
  - Created a DMZ Network example config
  - Tested the creation of Firewall
  - Tested the ACL Rules of Firewall

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_firewall_config.py

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_router_config.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2257
This commit is contained in:
Czar Echavez
2024-02-29 16:22:55 +00:00
82 changed files with 3546 additions and 652 deletions

View File

@@ -83,6 +83,19 @@ SessionManager.
- `AirSpace` class to simulate wireless communications, managing wireless interfaces and facilitating the transmission of frames within specified frequencies.
- `AirSpaceFrequency` enum for defining standard wireless frequencies, including 2.4 GHz and 5 GHz bands, to support realistic wireless network simulations.
- `WirelessRouter` class, extending the `Router` class, to incorporate wireless networking capabilities alongside traditional wired connections. This class allows the configuration of wireless access points with specific IP settings and operating frequencies.
- Documentation Updates:
- Examples include how to set up PrimAITE session via config
- Examples include how to create nodes and install software via config
- Examples include how to set up PrimAITE session via Python
- Examples include how to create nodes and install software via Python
- Added missing ``DoSBot`` documentation page
- Added diagrams where needed to make understanding some things easier
- Templated parts of the documentation to prevent unnecessary repetition and for easier maintaining of documentation
- Separated documentation pages of some items i.e. client and server software were on the same pages - which may make things confusing
- Configuration section at the bottom of the software pages specifying the configuration options available (and which ones are optional)
- Ability to add ``Firewall`` node via config
- Ability to add ``Router`` routes via config
- Ability to add ``Router``/``Firewall`` ``ACLRule`` via config
- NMNE capturing capabilities to `NetworkInterface` class for detecting and logging Malicious Network Events.
- New `nmne_config` settings in the simulation configuration to enable NMNE capturing and specify keywords such as "DELETE".

BIN
docs/_static/firewall_acl.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

BIN
docs/_static/switched_p2p_network.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.0 KiB

View File

@@ -1,3 +1,5 @@
:orphan:
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK

View File

@@ -15,7 +15,6 @@ import furo # noqa
sys.path.insert(0, os.path.abspath("../"))
# -- Project information -----------------------------------------------------
year = datetime.datetime.now().year
project = "PrimAITE"
@@ -28,6 +27,11 @@ with open("../src/primaite/VERSION", "r") as file:
# The full version, including alpha/beta/rc tags
release = version
# set global variables
rst_prolog = f"""
.. |VERSION| replace:: {release}
"""
html_title = f"{project} v{release} docs"
# -- General configuration ---------------------------------------------------
@@ -45,13 +49,17 @@ extensions = [
"sphinx_copybutton", # Adds a copy button to code blocks
]
templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
exclude_patterns = [
"_build",
"Thumbs.db",
".DS_Store",
]
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
html_theme = "furo"
html_static_path = ["_static"]
html_theme_options = {"globaltoc_collapse": True, "globaltoc_maxdepth": 2}
html_copy_source = False

View File

@@ -1,102 +1,40 @@
Primaite v3 config
******************
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
PrimAITE |VERSION| Configuration
********************************
PrimAITE uses a single configuration file to define everything needed to train and evaluate an RL policy in a custom cybersecurity scenario. This includes the configuration of the network, the scripted or trained agents that interact with the network, as well as settings that define how to perform training in Stable Baselines 3 or Ray RLLib.
The entire config is used by the ``PrimaiteSession`` object for users who wish to let PrimAITE handle the agent definition and training. If you wish to define custom agents and control the training loop yourself, you can use the config with the ``PrimaiteGame``, and ``PrimaiteGymEnv`` objects instead. That way, only the network configuration and agent setup parts of the config are used, and the training section is ignored.
Example Configuration Hierarchy
###############################
The top level configuration items in a configuration file is as follows
.. code-block:: yaml
training_config:
...
io_settings:
...
game:
...
agents:
...
simulation:
...
These are expanded upon in the Configurable items section below
Configurable items
==================
##################
``training_config``
-------------------
This section allows selecting which training framework and algorithm to use, and set some training hyperparameters.
.. toctree::
:maxdepth: 1
``io_settings``
---------------
This section configures how PrimAITE saves data during simulation and training.
**save_final_model**: Only used if training with PrimaiteSession, if true, the policy will be saved after the final training iteration.
**save_checkpoints**: Only used if training with PrimaiteSession, if true, the policy will be saved periodically during training.
**checkpoint_interval**: Only used if training with PrimaiteSession and if ``save_checkpoints`` is true. Defines how often to save the policy during training.
**save_logs**: *currently unused*.
**save_transactions**: *currently unused*.
**save_tensorboard_logs**: *currently unused*.
**save_step_metadata**: Whether to save the RL agents' action, environment state, and other data at every single step.
**save_pcap_logs**: Whether to save pcap files of all network traffic during the simulation.
**save_sys_logs**: Whether to save system logs from all nodes during the simulation.
``game``
--------
This section defines high-level settings that apply across the game, currently it's used to help shape the action and observation spaces by restricting which ports and internet protocols should be considered. Here, users can also set the maximum number of steps in an episode.
``agents``
----------
Agents can be scripted (deterministic and stochastic), or controlled by a reinforcement learning algorithm. Not to be confused with an RL agent, the term agent here is used to refer to an entity that sends requests to the simulated network. In this part of the config, each agent's action space, observation space, and reward function can be defined. All three are defined in a modular way.
**type**: Specifies which class should be used for the agent. ``ProxyAgent`` is used for agents that receive instructions from an RL algorithm. Scripted agents like ``RedDatabaseCorruptingAgent`` and ``GreenWebBrowsingAgent`` generate their own behaviour.
**team**: Specifies if the agent is malicious (RED), benign (GREEN), or defensive (BLUE). Currently this value is not used for anything.
**observation space:**
* ``type``: selects which python class from the ``primaite.game.agent.observation`` module is used for the overall observation structure.
* ``options``: allows configuring the chosen observation type. The ``UC2BlueObservation`` should be used for RL Agents.
* ``num_services_per_node``, ``num_folders_per_node``, ``num_files_per_folder``, ``num_nics_per_node`` all define the shape of the observation space. The size and shape of the obs space must remain constant, but the number of files, folders, ACL rules, and other components can change within an episode. Therefore padding is performed and these options set the size of the obs space.
* ``nodes``: list of nodes that will be present in this agent's observation space. The ``node_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config. Each node can also be configured with services, and files that should be monitored.
* ``links``: list of links that will be present in this agent's observation space. The ``link_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config.
* ``acl``: configure how the agent reads the access control list on the router in the simulation. ``router_node_ref`` is for selecting which router's ACL table should be used. ``ip_address_order`` sets the encoding of ip addresses as integers within the observation space.
**action space:**
The action space is configured to be made up of individual action types. Once configured, the agent can select an action type and some optional action parameters at every step. For example: The ``NODE_SERVICE_SCAN`` action takes the parameters ``node_id`` and ``service_id``.
Description of configurable items:
* ``action_list``: a list of action modules. The options are listed in the ``primaite.game.agent.actions`` module.
* ``action_map``: (optional). Restricts the possible combinations of action type / action parameter values to reduce the overall size of the action space. By default, every possible combination of actions and parameters will be assigned an integer for the agent's ``MultiDiscrete`` action space. Instead, the ``action_map`` allows you to list the actions corresponding to each integer in the ``MultiDiscrete`` space.
* ``options``: Options that apply too all action components.
* ``nodes``: list the nodes that the agent can act on, the order of this list defines the mapping between nodes and ``node_id`` integers.
* ``max_folders_per_node``, ``max_files_per_folder``, ``max_services_per_node``, ``max_nics_per_node``, ``max_acl_rules`` all are used to define the size of the action space.
**reward function:**
Similar to action space, this is defined as a list of components.
Description of configurable items:
* ``reward_components`` a list of reward components from the ``primaite.game.agent.reward`` module.
* ``weight``: relative importance of this reward component. The total reward for a step is a weighted sum of all reward components.
* ``options``: list of options passed to the reward component during initialisation, the exact options required depend on the reward component.
**agent_settings**:
Settings passed to the agent during initialisation. These depend on the agent class.
Reinforcement learning agents use the ``ProxyAgent`` class, they accept these agent settings:
**flatten_obs**: If true, gymnasium flattening will be performed on the observation space before sending to the agent. Set this to true if your agent does not support nested observation spaces.
``simulation``
--------------
In this section the network layout is defined. This part of the config follows a hierarchical structure. Almost every component defines a ``ref`` field which acts as a human-readable unique identifier, used by other parts of the config, such as agents.
At the top level of the network are ``nodes`` and ``links``.
**nodes:**
* ``type``: one of ``router``, ``switch``, ``computer``, or ``server``, this affects what other sub-options should be defined.
* ``hostname`` - a non-unique name used for logging and outputs.
* ``num_ports`` (optional, routers and switches only): number of network interfaces present on the device.
* ``ports`` (optional, routers and switches only): configuration for each network interface, including IP address and subnet mask.
* ``acl`` (Router only): Define the ACL rules at each index of the ACL on the router. the possible options are: ``action`` (PERMIT or DENY), ``src_port``, ``dst_port``, ``protocol``, ``src_ip``, ``dst_ip``. Any options left blank default to none which usually means that it will apply across all options. For example leaving ``src_ip`` blank will apply the rule to all IP addresses.
* ``services`` (computers and servers only): a list of services to install on the node. They must define a ``ref``, ``type``, and ``options`` that depend on which ``type`` was selected.
* ``applications`` (computer and servers only): Similar to services. A list of application to install on the node.
* ``network_interfaces`` (computers and servers only): If the node has multiple networking devices, the second, third, fourth, etc... must be defined here with an ``ip_address`` and ``subnet_mask``.
**links:**
* ``ref``: unique identifier for this link
* ``endpoint_a_ref``: Reference to the node at the first end of the link
* ``endpoint_a_port``: The ethernet port or switch port index of the second node
* ``endpoint_b_ref``: Reference to the node at the second end of the link
* ``endpoint_b_port``: The ethernet port or switch port index on the second node
configuration/training_config.rst
configuration/io_settings.rst
configuration/game.rst
configuration/agents.rst
configuration/simulation.rst

View File

@@ -0,0 +1,174 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``agents``
==========
Agents can be scripted (deterministic and stochastic), or controlled by a reinforcement learning algorithm. Not to be confused with an RL agent, the term agent here is used to refer to an entity that sends requests to the simulated network. In this part of the config, each agent's action space, observation space, and reward function can be defined. All three are defined in a modular way.
``agents`` hierarchy
--------------------
.. code-block:: yaml
agents:
- ref: red_agent_example
...
- ref: blue_agent_example
...
- ref: green_agent_example
team: GREEN
type: GreenWebBrowsingAgent
observation_space:
type: UC2GreenObservation
action_space:
action_list:
- type: DONOTHING
- type: NODE_APPLICATION_EXECUTE
options:
nodes:
- node_name: client_2
applications:
- application_name: WebBrowser
max_folders_per_node: 1
max_files_per_folder: 1
max_services_per_node: 1
max_applications_per_node: 1
reward_function:
reward_components:
- type: DUMMY
agent_settings:
start_settings:
start_step: 5
frequency: 4
variance: 3
flatten_obs: False
``ref``
-------
The reference to be used for the given agent.
``team``
--------
Specifies if the agent is malicious (``RED``), benign (``GREEN``), or defensive (``BLUE``). Currently this value is not used for anything other than for human readability in the configuration file.
``type``
--------
Specifies which class should be used for the agent. ``ProxyAgent`` is used for agents that receive instructions from an RL algorithm. Scripted agents like ``RedDatabaseCorruptingAgent`` and ``GreenWebBrowsingAgent`` generate their own behaviour.
Available agent types:
- ``GreenWebBrowsingAgent``
- ``ProxyAgent``
- ``RedDatabaseCorruptingAgent``
``observation_space``
---------------------
Defines the observation space of the agent.
``type``
^^^^^^^^
selects which python class from the :py:mod:`primaite.game.agent.observation` module is used for the overall observation structure.
``options``
^^^^^^^^^^^
Allows configuration of the chosen observation type. These are optional.
* ``num_services_per_node``, ``num_folders_per_node``, ``num_files_per_folder``, ``num_nics_per_node`` all define the shape of the observation space. The size and shape of the obs space must remain constant, but the number of files, folders, ACL rules, and other components can change within an episode. Therefore padding is performed and these options set the size of the obs space.
* ``nodes``: list of nodes that will be present in this agent's observation space. The ``node_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config. Each node can also be configured with services, and files that should be monitored.
* ``links``: list of links that will be present in this agent's observation space. The ``link_ref`` relates to the human-readable unique reference defined later in the ``simulation`` part of the config.
* ``acl``: configure how the agent reads the access control list on the router in the simulation. ``router_node_ref`` is for selecting which router's ACL table should be used. ``ip_address_order`` sets the encoding of ip addresses as integers within the observation space.
For more information see :py:mod:`primaite.game.agent.observations`
``action_space``
----------------
The action space is configured to be made up of individual action types. Once configured, the agent can select an action type and some optional action parameters at every step. For example: The ``NODE_SERVICE_SCAN`` action takes the parameters ``node_id`` and ``service_id``.
``action_list``
^^^^^^^^^^^^^^^
A list of action modules. The options are listed in the :py:mod:`primaite.game.agent.actions.ActionManager.act_class_identifiers` module.
``action_map``
^^^^^^^^^^^^^^
Restricts the possible combinations of action type / action parameter values to reduce the overall size of the action space. By default, every possible combination of actions and parameters will be assigned an integer for the agent's ``MultiDiscrete`` action space. Instead, the ``action_map`` allows you to list the actions corresponding to each integer in the ``MultiDiscrete`` space.
This is Optional.
``options``
^^^^^^^^^^^
Options that apply to all action components. These are optional.
* ``nodes``: list the nodes that the agent can act on, the order of this list defines the mapping between nodes and ``node_id`` integers.
* ``max_folders_per_node``, ``max_files_per_folder``, ``max_services_per_node``, ``max_nics_per_node``, ``max_acl_rules`` all are used to define the size of the action space.
For more information see :py:mod:`primaite.game.agent.actions`
``reward_function``
-------------------
Similar to action space, this is defined as a list of components from the :py:mod:`primaite.game.agent.rewards` module.
``reward_components``
^^^^^^^^^^^^^^^^^^^^^
A list of reward types from :py:mod:`primaite.game.agent.rewards.RewardFunction.rew_class_identifiers`
e.g.
.. code-block:: yaml
reward_components:
- type: DUMMY
- type: DATABASE_FILE_INTEGRITY
``agent_settings``
------------------
Settings passed to the agent during initialisation. Determines how the agent will behave during training.
e.g.
.. code-block:: yaml
agent_settings:
start_settings:
start_step: 25
frequency: 20
variance: 5
``start_step``
^^^^^^^^^^^^^^
Optional. Default value is ``5``.
The timestep where the agent begins performing actions.
``frequency``
^^^^^^^^^^^^^
Optional. Default value is ``5``.
The number of timesteps the agent will wait before performing another action.
``variance``
^^^^^^^^^^^^
Optional. Default value is ``0``.
The amount of timesteps that the frequency can randomly change.
``flatten_obs``
---------------
If ``True``, gymnasium flattening will be performed on the observation space before sending to the agent. Set this to ``True`` if your agent does not support nested observation spaces.

View File

@@ -0,0 +1,46 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``game``
========
This section defines high-level settings that apply across the game, currently it's used to help shape the action and observation spaces by restricting which ports and internet protocols should be considered. Here, users can also set the maximum number of steps in an episode.
``game`` hierarchy
------------------
.. code-block:: yaml
game:
max_episode_length: 256
ports:
- ARP
- DNS
- HTTP
- POSTGRES_SERVER
protocols:
- ICMP
- TCP
- UDP
``max_episode_length``
----------------------
Optional. Default value is ``256``.
The maximum number of episodes a Reinforcement Learning agent(s) can be trained for.
``ports``
---------
A list of ports that the Reinforcement Learning agent(s) are able to see in the observation space.
See :ref:`List of Ports <List of Ports>` for a list of ports.
``protocols``
-------------
A list of protocols that the Reinforcement Learning agent(s) are able to see in the observation space.
See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.

View File

@@ -0,0 +1,89 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``io_settings``
===============
This section configures how PrimAITE saves data during simulation and training.
``io_settings`` hierarchy
-------------------------
.. code-block:: yaml
io_settings:
save_final_model: True
save_checkpoints: False
checkpoint_interval: 10
# save_logs: True
# save_transactions: False
# save_tensorboard_logs: False
save_step_metadata: False
save_pcap_logs: False
save_sys_logs: False
``save_final_model``
--------------------
Optional. Default value is ``True``.
Only used if training with PrimaiteSession.
If ``True``, the policy will be saved after the final training iteration.
``save_checkpoints``
--------------------
Optional. Default value is ``False``.
Only used if training with PrimaiteSession.
If ``True``, the policy will be saved periodically during training.
``checkpoint_interval``
-----------------------
Optional. Default value is ``10``.
Only used if training with PrimaiteSession and if ``save_checkpoints`` is ``True``.
Defines how often to save the policy during training.
``save_logs``
-------------
*currently unused*.
``save_transactions``
---------------------
*currently unused*.
``save_tensorboard_logs``
-------------------------
*currently unused*.
``save_step_metadata``
----------------------
Optional. Default value is ``False``.
If ``True``, The RL agent(s) actions, environment states and other data will be saved at every single step.
``save_pcap_logs``
------------------
Optional. Default value is ``False``.
If ``True``, then the pcap files which contain all network traffic during the simulation will be saved.
``save_sys_logs``
-----------------
Optional. Default value is ``False``.
If ``True``, then the log files which contain all node actions during the simulation will be saved.

View File

@@ -0,0 +1,93 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``simulation``
==============
In this section the network layout is defined. This part of the config follows a hierarchical structure. Almost every component defines a ``ref`` field which acts as a human-readable unique identifier, used by other parts of the config, such as agents.
At the top level of the network are ``nodes`` and ``links``.
e.g.
.. code-block:: yaml
simulation:
network:
nodes:
...
links:
...
``nodes``
---------
This is where the list of nodes are defined. Some items will differ according to the node type, however, there will be common items such as a node's reference (which is used by the agent), the node's ``type`` and ``hostname``
To see the configuration for these nodes, refer to the following:
.. toctree::
:maxdepth: 1
:glob:
simulation/nodes/*
``links``
---------
This is where the links between the nodes are formed.
e.g.
In order to recreate the network below, we will need to create 2 links:
- a link from computer_1 to the switch
- a link from computer_2 to the switch
.. image:: ../../_static/switched_p2p_network.png
:width: 500
:align: center
this results in:
.. code-block:: yaml
links:
- ref: computer_1___switch
endpoint_a_ref: computer_1
endpoint_a_port: 1 # port 1 on computer_1
endpoint_b_ref: switch
endpoint_b_port: 1 # port 1 on switch
- ref: computer_2___switch
endpoint_a_ref: computer_2
endpoint_a_port: 1 # port 1 on computer_2
endpoint_b_ref: switch
endpoint_b_port: 2 # port 2 on switch
``ref``
^^^^^^^
The human readable name for the link. Not used in code, however is useful for a human to understand what the link is for.
``endpoint_a_ref``
^^^^^^^^^^^^^^^^^^
The ``hostname`` of the node which must be connected.
``endpoint_a_port``
^^^^^^^^^^^^^^^^^^^
The port on ``endpoint_a_ref`` which is to be connected to ``endpoint_b_port``.
This accepts an integer value e.g. if port 1 is to be connected, the configuration should be ``endpoint_a_port: 1``
``endpoint_b_ref``
^^^^^^^^^^^^^^^^^^
The ``hostname`` of the node which must be connected.
``endpoint_b_port``
^^^^^^^^^^^^^^^^^^^
The port on ``endpoint_b_ref`` which is to be connected to ``endpoint_a_port``.
This accepts an integer value e.g. if port 1 is to be connected, the configuration should be ``endpoint_b_port: 1``

View File

@@ -0,0 +1,35 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _Node Attributes:
Common Attributes
#################
Node Attributes
===============
Attributes that are shared by all nodes.
.. include:: common_node_attributes.rst
.. _Network Node Attributes:
Network Node Attributes
=======================
Attributes that are shared by nodes that inherit from :py:mod:`primaite.simulator.network.hardware.nodes.network.network_node.NetworkNode`
.. include:: common_host_node_attributes.rst
.. _Host Node Attributes:
Host Node Attributes
====================
Attributes that are shared by nodes that inherit from :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.HostNode`
.. include:: common_host_node_attributes.rst
.. |NODE| replace:: node

View File

@@ -0,0 +1,26 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _common_host_node_attributes:
``ip_address``
--------------
The IP address of the |NODE| in the network.
``subnet_mask``
---------------
Optional. Default value is ``255.255.255.0``.
The subnet mask for the |NODE| to use.
``default_gateway``
-------------------
The IP address that the |NODE| will use as the default gateway. Typically, this is the IP address of the closest router that the |NODE| is connected to.
.. include:: ../software/applications.rst
.. include:: ../software/services.rst

View File

@@ -0,0 +1,51 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _common_network_node_attributes:
``routes``
----------
A list of routes which tells the |NODE| where to forward the packet to depending on the target IP address.
e.g.
.. code-block:: yaml
nodes:
- ref: node
...
routes:
- address: 192.168.0.10
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.1
metric: 0
``address``
"""""""""""
The target IP address for the route. If the packet destination IP address matches this, the |NODE| will route the packet according to the ``next_hop_ip_address``.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the route.
``next_hop_ip_address``
"""""""""""""""""""""""
The IP address of the next hop IP address that the packet will follow if the address matches the packet's destination IP address.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``metric``
""""""""""
Optional. Default value is ``0``. This value accepts floats.
The cost or distance of a route. The higher the value, the more cost or distance is attributed to the route.

View File

@@ -0,0 +1,55 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _common_node_attributes:
``ref``
-------
Human readable name used as reference for the |NODE|. Not used in code.
``hostname``
------------
The hostname of the |NODE|. This will be used to reference the |NODE|.
``operating_state``
-------------------
The initial operating state of the node.
Optional. Default value is ``ON``.
Options available are:
- ``ON``
- ``OFF``
- ``BOOTING``
- ``SHUTTING_DOWN``
Note that YAML may assume non quoted ``ON`` and ``OFF`` as ``True`` and ``False`` respectively. To prevent this, use ``"ON"`` or ``"OFF"``
See :py:mod:`primaite.simulator.network.hardware.node_operating_state.NodeOperatingState`
``dns_server``
--------------
Optional. Default value is ``None``.
The IP address of the node which holds an instance of the :ref:`DNSServer`. Some applications may use a domain name e.g. the :ref:`WebBrowser`
``start_up_duration``
---------------------
Optional. Default value is ``3``.
The number of time steps required to occur in order for the node to cycle from ``OFF`` to ``BOOTING_UP`` and then finally ``ON``.
``shut_down_duration``
----------------------
Optional. Default value is ``3``.
The number of time steps required to occur in order for the node to cycle from ``ON`` to ``SHUTTING_DOWN`` and then finally ``OFF``.

View File

@@ -0,0 +1,18 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``type``
--------
The type of node to add.
Available options are:
- ``computer``
- ``firewall``
- ``router``
- ``server``
- ``switch``
To create a |NODE|, type must be |NODE_TYPE|.

View File

@@ -0,0 +1,41 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _computer_configuration:
``computer``
============
A basic representation of a computer within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.host.computer.Computer`
example computer
----------------
.. code-block:: yaml
simulation:
network:
nodes:
- ref: client_1
hostname: client_1
type: computer
ip_address: 192.168.0.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.0.1
dns_server: 192.168.1.10
applications:
...
services:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
.. include:: common/common_host_node_attributes.rst
.. |NODE| replace:: computer
.. |NODE_TYPE| replace:: ``computer``

View File

@@ -0,0 +1,300 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _firewall_configuration:
``firewall``
============
A basic representation of a network firewall within the simulation.
The firewall is similar to how :ref:`Router <router_configuration>` works, with the difference being how firewall has specific ACL rules for inbound and outbound traffic as well as firewall being limited to 3 ports.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.firewall.Firewall`
example firewall
----------------
.. code-block:: yaml
simulation:
network:
nodes:
- ref: firewall
hostname: firewall
type: firewall
start_up_duration: 0
shut_down_duration: 0
ports:
external_port: # port 1
ip_address: 192.168.20.1
subnet_mask: 255.255.255.0
internal_port: # port 2
ip_address: 192.168.1.2
subnet_mask: 255.255.255.0
dmz_port: # port 3
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
acl:
internal_inbound_acl:
...
internal_outbound_acl:
...
dmz_inbound_acl:
...
dmz_outbound_acl:
...
external_inbound_acl:
...
external_outbound_acl:
...
routes:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``ports``
---------
The firewall node only has 3 ports. These specifically are:
- ``external_port`` (port 1)
- ``internal_port`` (port 2)
- ``dmz_port`` (port 3) (can be optional)
The ports should be defined with an ip address and subnet mask e.g.
.. code-block:: yaml
nodes:
- ref: firewall
...
ports:
external_port: # port 1
ip_address: 192.168.20.1
subnet_mask: 255.255.255.0
internal_port: # port 2
ip_address: 192.168.1.2
subnet_mask: 255.255.255.0
dmz_port: # port 3
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
``ip_address``
""""""""""""""
The IP address for the given port. This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the port.
``acl``
-------
There are 6 ACLs that can be defined for a firewall
- ``internal_inbound_acl`` for traffic going towards the internal network
- ``internal_outbound_acl`` for traffic coming from the internal network
- ``dmz_inbound_acl`` for traffic going towards the dmz network
- ``dmz_outbound_acl`` for traffic coming from the dmz network
- ``external_inbound_acl`` for traffic coming from the external network
- ``external_outbound_acl`` for traffic going towards the external network
.. image:: ../../../../_static/firewall_acl.png
:width: 500
:align: center
By default, ``external_inbound_acl`` and ``external_outbound_acl`` will permit any traffic through.
``internal_inbound_acl``, ``internal_outbound_acl``, ``dmz_inbound_acl`` and ``dmz_outbound_acl`` will deny any traffic by default, so must be configured to allow defined ``src_port`` and ``dst_port`` or ``protocol``.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.AccessControlList`
See :ref:`List of Ports <List of Ports>` for a list of ports.
``internal_inbound_acl``
""""""""""""""""""""""""
ACL rules for packets that have a destination IP address in what is considered the internal network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
internal_inbound_acl:
21: # position 21 on ACL list
action: PERMIT # allow packets that
src_port: POSTGRES_SERVER # are emitted from the POSTGRES_SERVER port
dst_port: POSTGRES_SERVER # are going towards an POSTGRES_SERVER port
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``internal_outbound_acl``
"""""""""""""""""""""""""
ACL rules for packets that have a source IP address in what is considered the internal network and is going towards the DMZ network or the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
internal_outbound_acl:
21: # position 21 on ACL list
action: PERMIT # allow packets that
src_port: POSTGRES_SERVER # are emitted from the POSTGRES_SERVER port
dst_port: POSTGRES_SERVER # are going towards an POSTGRES_SERVER port
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``dmz_inbound_acl``
"""""""""""""""""""
ACL rules for packets that have a destination IP address in what is considered the DMZ network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
dmz_inbound_acl:
19: # position 19 on ACL list
action: PERMIT # allow packets that
src_port: POSTGRES_SERVER # are emitted from the POSTGRES_SERVER port
dst_port: POSTGRES_SERVER # are going towards an POSTGRES_SERVER port
20: # position 20 on ACL list
action: PERMIT # allow packets that
src_port: HTTP # are emitted from the HTTP port
dst_port: HTTP # are going towards an HTTP port
21: # position 21 on ACL list
action: PERMIT # allow packets that
src_port: HTTPS # are emitted from the HTTPS port
dst_port: HTTPS # are going towards an HTTPS port
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``dmz_outbound_acl``
""""""""""""""""""""
ACL rules for packets that have a source IP address in what is considered the DMZ network and is going towards the internal network or the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
dmz_outbound_acl:
19: # position 19 on ACL list
action: PERMIT # allow packets that
src_port: POSTGRES_SERVER # are emitted from the POSTGRES_SERVER port
dst_port: POSTGRES_SERVER # are going towards an POSTGRES_SERVER port
20: # position 20 on ACL list
action: PERMIT # allow packets that
src_port: HTTP # are emitted from the HTTP port
dst_port: HTTP # are going towards an HTTP port
21: # position 21 on ACL list
action: PERMIT # allow packets that
src_port: HTTPS # are emitted from the HTTPS port
dst_port: HTTPS # are going towards an HTTPS port
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``external_inbound_acl``
""""""""""""""""""""""""
Optional. By default, this will allow any traffic through.
ACL rules for packets that have a destination IP address in what is considered the external network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
external_inbound_acl:
21: # position 19 on ACL list
action: DENY # deny packets that
src_port: POSTGRES_SERVER # are emitted from the POSTGRES_SERVER port
dst_port: POSTGRES_SERVER # are going towards an POSTGRES_SERVER port
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
``external_outbound_acl``
"""""""""""""""""""""""""
Optional. By default, this will allow any traffic through.
ACL rules for packets that have a source IP address in what is considered the external network and is going towards the DMZ network or the internal network.
example:
.. code-block:: yaml
nodes:
- ref: firewall
...
acl:
external_outbound_acl:
22: # position 22 on ACL list
action: PERMIT # allow packets that
src_port: ARP # are emitted from the ARP port
dst_port: ARP # are going towards an ARP port
23: # position 23 on ACL list
action: PERMIT # allow packets that
protocol: ICMP # are ICMP
.. include:: common/common_network_node_attributes.rst
.. |NODE| replace:: firewall
.. |NODE_TYPE| replace:: ``firewall``

View File

@@ -0,0 +1,127 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _router_configuration:
``router``
==========
A basic representation of a network router within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.Router`
example router
--------------
.. code-block:: yaml
simulation:
network:
nodes:
- ref: router_1
hostname: router_1
type: router
num_ports: 5
ports:
...
acl:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``num_ports``
-------------
Optional. Default value is ``5``.
The number of ports the router will have.
``ports``
---------
Sets up the router's ports with an IP address and a subnet mask.
Example of setting ports for a router with 2 ports:
.. code-block:: yaml
nodes:
- ref: router_1
...
ports:
1:
ip_address: 192.168.1.1
subnet_mask: 255.255.255.0
2:
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
``ip_address``
""""""""""""""
The IP address for the given port. This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``subnet_mask``
"""""""""""""""
Optional. Default value is ``255.255.255.0``.
The subnet mask setting for the port.
``acl``
-------
Sets up the ACL rules for the router.
e.g.
.. code-block:: yaml
nodes:
- ref: router_1
...
acl:
1:
action: PERMIT
src_port: ARP
dst_port: ARP
2:
action: PERMIT
protocol: ICMP
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.AccessControlList`
See :ref:`List of Ports <List of Ports>` for a list of ports.
``action``
""""""""""
Available options are
- ``PERMIT`` : Allows the specified ``protocol`` or ``src_port`` and ``dst_port`` pairs
- ``DENY`` : Blocks the specified ``protocol`` or ``src_port`` and ``dst_port`` pairs
``src_port``
""""""""""""
Is used alongside ``dst_port``. Specifies the port where a packet originates. Used by the ACL Rule to determine if a packet with a specific source port is allowed to pass through the network node.
``dst_port``
""""""""""""
Is used alongside ``src_port``. Specifies the port where a packet is destined to arrive. Used by the ACL Rule to determine if a packet with a specific destination port is allowed to pass through the network node.
``protocol``
""""""""""""
Specifies which protocols are allowed by the ACL Rule to pass through the network node.
See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.
.. include:: common/common_network_node_attributes.rst
.. |NODE| replace:: router
.. |NODE_TYPE| replace:: ``router``

View File

@@ -0,0 +1,41 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _server_configuration:
``server``
==========
A basic representation of a server within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.host.server.Server`
example server
--------------
.. code-block:: yaml
simulation:
network:
nodes:
- ref: server_1
hostname: server_1
type: server
ip_address: 192.168.10.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
dns_server: 192.168.1.10
applications:
...
services:
...
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
.. include:: common/common_host_node_attributes.rst
.. |NODE| replace:: server
.. |NODE_TYPE| replace:: ``server``

View File

@@ -0,0 +1,39 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _switch_configuration:
``switch``
==========
A basic representation of a network switch within the simulation.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.switch.Switch`
example switch
--------------
.. code-block:: yaml
simulation:
network:
nodes:
- ref: switch_1
hostname: switch_1
type: switch
num_ports: 8
.. include:: common/common_node_attributes.rst
.. include:: common/node_type_list.rst
``num_ports``
-------------
Optional. Default value is ``8``.
The number of ports the switch will have.
.. |NODE| replace:: switch
.. |NODE_TYPE| replace:: ``switch``

View File

@@ -0,0 +1,25 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``applications``
----------------
List of available applications that can be installed on a |NODE| can be found in :ref:`List of Applications <List of Applications>`
application in configuration
""""""""""""""""""""""""""""
Applications takes a list of applications as shown in the example below.
.. code-block:: yaml
- ref: client_1
hostname: client_1
type: computer
...
applications:
- ref: example_application
type: example_application_type
options:
# this section is different for each application

View File

@@ -0,0 +1,25 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``services``
------------
List of available services that can be installed on a |NODE| can be found in :ref:`List of Services <List of Services>`
services in configuration
"""""""""""""""""""""""""
Services takes a list of services as shown in the example below.
.. code-block:: yaml
- ref: client_1
hostname: client_1
type: computer
...
applications:
- ref: example_service
type: example_service_type
options:
# this section is different for each service

View File

@@ -0,0 +1,75 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``training_config``
===================
Configuration items relevant to how the Reinforcement Learning agent(s) will be trained.
``training_config`` hierarchy
-----------------------------
.. code-block:: yaml
training_config:
rl_framework: SB3 # or RLLIB_single_agent or RLLIB_multi_agent
rl_algorithm: PPO # or A2C
n_learn_episodes: 5
max_steps_per_episode: 200
n_eval_episodes: 1
deterministic_eval: True
seed: 123
``rl_framework``
----------------
The RL (Reinforcement Learning) Framework to use in the training session
Options available are:
- ``SB3`` (Stable Baselines 3)
- ``RLLIB_single_agent`` (Single Agent Ray RLLib)
- ``RLLIB_multi_agent`` (Multi Agent Ray RLLib)
``rl_algorithm``
----------------
The Reinforcement Learning Algorithm to use in the training session
Options available are:
- ``PPO`` (Proximal Policy Optimisation)
- ``A2C`` (Advantage Actor Critic)
``n_learn_episodes``
--------------------
The number of episodes to train the agent(s).
This should be an integer value above ``0``
``max_steps_per_episode``
-------------------------
The number of steps each episode will last for.
This should be an integer value above ``0``.
``n_eval_episodes``
-------------------
Optional. Default value is ``0``.
The number of evaluation episodes to run the trained agent for.
This should be an integer value above ``0``.
``deterministic_eval``
----------------------
Optional. By default this value is ``False``.
If this is set to ``True``, the agents will act deterministically instead of stochastically.
``seed``
--------
Optional.
The seed is used (alongside ``deterministic_eval``) to reproduce a previous instance of training and evaluation of an RL agent.
The seed should be an integer value.
Useful for debugging.

View File

@@ -5,6 +5,8 @@
.. role:: raw-html(raw)
:format: html
.. _Dependencies:
Dependencies
============

View File

@@ -18,7 +18,7 @@ Game layer
The game layer is responsible for managing agents and getting them to interface with the simulator correctly. It consists of several components:
PrimAITE Session
^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^
.. admonition:: Deprecated
:class: deprecated

View File

@@ -36,7 +36,7 @@ Technical Detail
This system was achieved by implementing two classes, :py:class:`primaite.simulator.core.RequestType`, and :py:class:`primaite.simulator.core.RequestManager`.
``RequestType``
------
---------------
The ``RequestType`` object stores a reference to a method that executes the request, for example a node could have a request type that stores a reference to ``self.turn_on()``. Technically, this can be any callable that accepts `request, context` as it's parameters. In practice, this is often defined using ``lambda`` functions within a component's ``self._init_request_manager()`` method. Optionally, the ``RequestType`` object can also hold a validator that will permit/deny the request depending on context.
@@ -60,7 +60,7 @@ A simple example without chaining can be seen in the :py:class:`primaite.simulat
*ellipses (``...``) used to omit code impertinent to this explanation*
Chaining RequestManagers
-----------------------
------------------------
A request function needs to be a callable that accepts ``request, context`` as parameters. Since the request manager resolves requests by invoking it with ``request, context`` as parameter, it is possible to use a ``RequestManager`` as a ``RequestType``.

View File

@@ -22,9 +22,9 @@ Contents
simulation_components/network/nodes/host_node
simulation_components/network/nodes/network_node
simulation_components/network/nodes/router
simulation_components/network/nodes/switch
simulation_components/network/nodes/wireless_router
simulation_components/network/nodes/firewall
simulation_components/network/switch
simulation_components/network/network
simulation_components/system/internal_frame_processing
simulation_components/system/sys_log

View File

@@ -12,34 +12,81 @@ complex, specialized hardware components inherit from and build upon.
The key elements defined in ``base.py`` are:
NetworkInterface
================
``NetworkInterface``
====================
- Abstract base class for network interfaces like NICs. Defines common attributes like MAC address, speed, MTU.
- Requires subclasses to implement ``enable()``, ``disable()``, ``send_frame()`` and ``receive_frame()``.
- Provides basic state description and request handling capabilities.
Node
====
``Node``
========
The Node class stands as a central component in ``base.py``, acting as the superclass for all network nodes within a
PrimAITE simulation.
Node Attributes
---------------
See :ref:`Node Attributes`
- **hostname**: The network hostname of the node.
- **operating_state**: Indicates the current hardware state of the node.
- **network_interfaces**: Maps interface names to NetworkInterface objects on the node.
- **network_interface**: Maps port IDs to ``NetworkInterface`` objects on the node.
- **dns_server**: Specifies DNS servers for domain name resolution.
- **start_up_duration**: The time it takes for the node to become fully operational after being powered on.
- **shut_down_duration**: The time required for the node to properly shut down.
- **sys_log**: A system log for recording events related to the node.
- **session_manager**: Manages user sessions within the node.
- **software_manager**: Controls the installation and management of software and services on the node.
.. _Node Start up and Shut down:
Node Start up and Shut down
---------------------------
Nodes are powered on and off over multiple timesteps. By default, the node ``start_up_duration`` and ``shut_down_duration`` is 3 timesteps.
Example code where a node is turned on:
.. code-block:: python
from primaite.simulator.network.hardware.base import Node
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
node = Node(hostname="pc_a")
assert node.operating_state is NodeOperatingState.OFF # By default, node is instantiated in an OFF state
node.power_on() # power on the node
assert node.operating_state is NodeOperatingState.BOOTING # node is booting up
for i in range(node.start_up_duration + 1):
# apply timestep until the node start up duration
node.apply_timestep(timestep=i)
assert node.operating_state is NodeOperatingState.ON # node is in ON state
If the node needs to be instantiated in an on state:
.. code-block:: python
from primaite.simulator.network.hardware.base import Node
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
node = Node(hostname="pc_a", operating_state=NodeOperatingState.ON)
assert node.operating_state is NodeOperatingState.ON # node is in ON state
Setting ``start_up_duration`` and/or ``shut_down_duration`` to ``0`` will allow for the ``power_on`` and ``power_off`` methods to be completed instantly without applying timesteps:
.. code-block:: python
from primaite.simulator.network.hardware.base import Node
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
node = Node(hostname="pc_a", start_up_duration=0, shut_down_duration=0)
assert node.operating_state is NodeOperatingState.OFF # node is in OFF state
node.power_on()
assert node.operating_state is NodeOperatingState.ON # node is in ON state
node.power_off()
assert node.operating_state is NodeOperatingState.OFF # node is in OFF state
Node Behaviours/Functions
-------------------------

View File

@@ -30,11 +30,11 @@ we'll use the following Network that has a client, server, two switches, and a r
.. code-block:: python
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.base import NIC
from primaite.simulator.network.hardware.nodes.computer import Computer
from primaite.simulator.network.hardware.nodes.router import Router, ACLAction
from primaite.simulator.network.hardware.nodes.server import Server
from primaite.simulator.network.hardware.nodes.switch import Switch
from primaite.simulator.network.hardware.base import NetworkInterface
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.network.hardware.nodes.network.router import Router, ACLAction
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.network.hardware.nodes.network.switch import Switch
from primaite.simulator.network.transmission.network_layer import IPProtocol
from primaite.simulator.network.transmission.transport_layer import Port

View File

@@ -13,6 +13,8 @@ facilitates modular development, enhances maintainability, and supports scalabil
allowing for focused enhancements within each layer.
.. image:: primaite_network_interface_model.png
:width: 500
:align: center
Layer Descriptions
==================

View File

@@ -229,7 +229,7 @@ To limit database server access to selected external IP addresses:
position=7
)
**Permitting DMZ Web Server Access while Blocking Specific Threats*
**Permitting DMZ Web Server Access while Blocking Specific Threats**
To authorize HTTP/HTTPS access to a DMZ-hosted web server, excluding known malicious IPs:

View File

@@ -27,7 +27,7 @@ in the transmission and routing of data within the simulated environment.
**Key Features:**
- **Frame Processing:** Central to the class is the ability to receive and process network frames, facilitating the
simulation of data flow through network devices.
simulation of data flow through network devices.
- **Abstract Methods:** Includes abstract methods such as ``receive_frame``, which subclasses must implement to specify
how devices handle incoming traffic.

View File

@@ -2,20 +2,22 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DataManipulationBot:
DataManipulationBot
===================
###################
The ``DataManipulationBot`` class provides functionality to connect to a ``DatabaseService`` and execute malicious SQL statements.
The ``DataManipulationBot`` class provides functionality to connect to a :ref:`DatabaseService` and execute malicious SQL statements.
Overview
--------
========
The bot is intended to simulate a malicious actor carrying out attacks like:
- Dropping tables
- Deleting records
- Modifying data
on a database server by abusing an application's trusted database connectivity.
The bot performs attacks in the following stages to simulate the real pattern of an attack:
@@ -27,7 +29,7 @@ The bot performs attacks in the following stages to simulate the real pattern of
Each of these stages has a random, configurable probability of succeeding (by default 10%). The bot can also be configured to repeat the attack once complete.
Usage
-----
=====
- Create an instance and call ``configure`` to set:
- Target database server IP
@@ -40,16 +42,35 @@ The bot handles connecting, executing the statement, and disconnecting.
In a simulation, the bot can be controlled by using ``DataManipulationAgent`` which calls ``run`` on the bot at configured timesteps.
Example
-------
Implementation
==============
The bot extends :ref:`DatabaseClient` and leverages its connectivity.
- Uses the Application base class for lifecycle management.
- Credentials, target IP and other options set via ``configure``.
- ``run`` handles connecting, executing statement, and disconnecting.
- SQL payload executed via ``query`` method.
- Results in malicious SQL being executed on remote database server.
Examples
========
Python
""""""
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.system.applications.red_applications.data_manipulation_bot import DataManipulationBot
client_1 = Computer(
hostname="client_1",
ip_address="192.168.10.21",
subnet_mask="255.255.255.0",
default_gateway="192.168.10.1"
default_gateway="192.168.10.1",
operating_state=NodeOperatingState.ON # initialise the computer in an ON state
)
network.connect(endpoint_b=client_1.network_interface[1], endpoint_a=switch_2.network_interface[1])
@@ -58,16 +79,16 @@ Example
data_manipulation_bot.configure(server_ip_address=IPv4Address("192.168.1.14"), payload="DELETE")
data_manipulation_bot.run()
This would connect to the database service at 192.168.1.14, authenticate, and execute the SQL statement to drop the 'users' table.
This would connect to the database service at 192.168.1.14, authenticate, and execute the SQL statement to delete database contents.
Example with ``DataManipulationAgent``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
""""""""""""""""""""""""""""""""""""""
If not using the data manipulation bot manually, it needs to be used with a data manipulation agent. Below is an example section of configuration file for setting up a simulation with data manipulation bot and agent.
.. code-block:: yaml
game_config:
game:
# ...
agents:
- ref: data_manipulation_red_bot
@@ -78,7 +99,7 @@ If not using the data manipulation bot manually, it needs to be used with a data
type: UC2RedObservation
options:
nodes:
- node_ref: client_1
- node_name: client_1
observations:
- logon_status
- operating_status
@@ -95,7 +116,7 @@ If not using the data manipulation bot manually, it needs to be used with a data
- type: NODE_APPLICATION_EXECUTE
options:
nodes:
- node_ref: client_1
- node_name: client_1
applications:
- application_ref: data_manipulation_bot
max_folders_per_node: 1
@@ -128,13 +149,51 @@ If not using the data manipulation bot manually, it needs to be used with a data
payload: "DELETE"
server_ip: 192.168.1.14
Implementation
--------------
Configuration
=============
The bot extends ``DatabaseClient`` and leverages its connectivity.
.. include:: ../common/common_configuration.rst
- Uses the Application base class for lifecycle management.
- Credentials, target IP and other options set via ``configure``.
- ``run`` handles connecting, executing statement, and disconnecting.
- SQL payload executed via ``query`` method.
- Results in malicious SQL being executed on remote database server.
.. |SOFTWARE_NAME| replace:: DataManipulationBot
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DataManipulationBot``
``server_ip``
"""""""""""""
IP address of the :ref:`DatabaseService` which the ``DataManipulationBot`` will try to attack.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``server_password``
"""""""""""""""""""
Optional. Default value is ``None``.
The password that the ``DataManipulationBot`` will use to access the :ref:`DatabaseService`.
``payload``
"""""""""""
Optional. Default value is ``DELETE``.
The payload that the ``DataManipulationBot`` will send to the :ref:`DatabaseService`.
.. include:: ../common/db_payload_list.rst
``port_scan_p_of_success``
""""""""""""""""""""""""""
Optional. Default value is ``0.1``.
The chance of the ``DataManipulationBot`` to succeed with a port scan (and therefore continue the attack).
This must be a float value between ``0`` and ``1``.
``data_manipulation_p_of_success``
""""""""""""""""""""""""""""""""""
Optional. Default value is ``0.1``.
The chance of the ``DataManipulationBot`` to succeed with a data manipulation attack.
This must be a float value between ``0`` and ``1``.

View File

@@ -0,0 +1,106 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DatabaseClient:
DatabaseClient
##############
The ``DatabaseClient`` provides a client interface for connecting to the :ref:`DatabaseService`.
Key features
============
- Connects to the :ref:`DatabaseService` via the ``SoftwareManager``.
- Handles connecting and disconnecting.
- Executes SQL queries and retrieves result sets.
Usage
=====
- Initialise with server IP address and optional password.
- Connect to the :ref:`DatabaseService` with ``connect``.
- Retrieve results in a dictionary.
- Disconnect when finished.
Implementation
==============
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Connect and disconnect methods manage sessions.
- Payloads serialised as dictionaries for transmission.
- Extends base Application class.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.system.applications.database_client import DatabaseClient
client = Computer(
hostname="client",
ip_address="192.168.10.21",
subnet_mask="255.255.255.0",
default_gateway="192.168.10.1",
operating_state=NodeOperatingState.ON # initialise the computer in an ON state
)
# install DatabaseClient
client.software_manager.install(DatabaseClient)
database_client: DatabaseClient = client.software_manager.software.get("DatabaseClient")
# Configure the DatabaseClient
database_client.configure(server_ip_address=IPv4Address("192.168.0.1")) # address of the DatabaseService
database_client.run()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_computer
hostname: example_computer
type: computer
...
applications:
- ref: database_client
type: DatabaseClient
options:
db_server_ip: 192.168.0.1
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: DatabaseClient
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DatabaseClient``
``db_server_ip``
""""""""""""""""
IP address of the :ref:`DatabaseService` that the ``DatabaseClient`` will connect to
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``server_password``
"""""""""""""""""""
Optional. Default value is ``None``.
The password that the ``DatabaseClient`` will use to access the :ref:`DatabaseService`.

View File

@@ -0,0 +1,160 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DoSBot:
DoSBot
######
The ``DoSBot`` is an implementation of a Denial of Service attack within the PrimAITE simulation. This specifically simulates a `Slow Loris attack <https://en.wikipedia.org/wiki/Slowloris_(computer_security)>`.
Key features
============
- Connects to the :ref:`DatabaseService` via the ``SoftwareManager``.
- Makes many connections to the :ref:`DatabaseService` which ends up using up the available connections.
Usage
=====
- Configure with target IP address and optional password.
- use ``run`` to run the application_loop of DoSBot to begin attacks
- DoSBot runs through different actions at each timestep
Implementation
==============
- Leverages :ref:`DatabaseClient` to create connections with :ref`DatabaseServer`.
- Extends base Application class.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.system.applications.red_applications.dos_bot import DoSBot
# Create Computer
computer = Computer(
hostname="computer",
ip_address="192.168.1.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
computer.power_on()
# Install DoSBot on computer
computer.software_manager.install(DoSBot)
dos_bot: DoSBot = computer.software_manager.software.get("DoSBot")
# Configure the DoSBot
dos_bot.configure(
target_ip_address=IPv4Address("192.168.0.10"),
payload="SPOOF DATA",
repeat=False,
port_scan_p_of_success=0.8,
dos_intensity=1.0,
max_sessions=1000
)
# run DoSBot
dos_bot.run()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_computer
hostname: example_computer
type: computer
...
applications:
- ref: dos_bot
type: DoSBot
options:
target_ip_address: 192.168.0.10
payload: SPOOF DATA
repeat: False
port_scan_p_of_success: 0.8
dos_intensity: 1.0
max_sessions: 1000
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: DoSBot
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DoSBot``
``target_ip_address``
"""""""""""""""""""""
IP address of the :ref:`DatabaseService` which the ``DataManipulationBot`` will try to attack.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``target_port``
"""""""""""""""
Optional. Default value is ``5432``.
Port of the target service.
See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.
``payload``
"""""""""""
Optional. Default value is ``None``.
The payload that the ``DoSBot`` sends as part of its attack.
.. include:: ../common/db_payload_list.rst
``repeat``
""""""""""
Optional. Default value is ``False``.
If ``True`` the ``DoSBot`` will maintain its attack.
``port_scan_p_of_success``
""""""""""""""""""""""""""
Optional. Default value is ``0.1``.
The chance of the ``DoSBot`` to succeed with a port scan (and therefore continue the attack).
This must be a float value between ``0`` and ``1``.
``dos_intensity``
"""""""""""""""""
Optional. Default value is ``1.0``.
The intensity of the Denial of Service attack. This is multiplied by the number of ``max_sessions``.
This must be a float value between ``0`` and ``1``.
``max_sessions``
""""""""""""""""
Optional. Default value is ``1000``.
The maximum number of sessions the ``DoSBot`` is able to make.
This must be an integer value equal to or greater than ``0``.

View File

@@ -0,0 +1,111 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _WebBrowser:
WebBrowser
##########
The ``WebBrowser`` provides a client interface for connecting to the :ref:`WebServer`.
Key features
============
- Connects to the :ref:`WebServer` via the ``SoftwareManager``.
- Simulates HTTP requests and HTTP packet transfer across a network
- Allows the emulation of HTTP requests between client and server:
- Automatically uses ``DNSClient`` to resolve domain names
- GET: performs an HTTP GET request from client to server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the ``WebBrowser``.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
- Execute sending an HTTP GET request with ``get_webpage``
Implementation
==============
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for making HTTP requests between an HTTP client and server.
- Extends base Service class.
Examples
========
Python
""""""
The ``WebBrowser`` utilises :ref:`DNSClient` and :ref:`DNSServer` to resolve a URL.
The :ref:`DNSClient` must be configured to use the :ref:`DNSServer`. The :ref:`DNSServer` should be configured to have the ``WebBrowser`` ``target_url`` within its ``domain_mapping``.
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.system.applications.web_browser import WebBrowser
# Create Computer
computer = Computer(
hostname="computer",
ip_address="192.168.1.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
computer.power_on()
# Install WebBrowser on computer
computer.software_manager.install(WebBrowser)
web_browser: WebBrowser = computer.software_manager.software.get("WebBrowser")
web_browser.run()
# configure the WebBrowser
web_browser.target_url = "arcd.com"
# once DNS server is configured with the correct domain mapping
# this should work
web_browser.get_webpage()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_computer
hostname: example_computer
type: computer
...
applications:
- ref: web_browser
type: WebBrowser
options:
target_url: http://arcd.com/
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: WebBrowser
.. |SOFTWARE_NAME_BACKTICK| replace:: ``WebBrowser``
``target_url``
""""""""""""""
The URL that the ``WebBrowser`` will request when ``get_webpage`` is called without parameters.
The URL can be in any format so long as the domain is within it e.g.
The domain ``arcd.com`` can be matched by
- http://arcd.com/
- http://arcd.com/users/
- arcd.com

View File

@@ -0,0 +1,18 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``ref``
=======
Human readable name used as reference for the |SOFTWARE_NAME_BACKTICK|. Not used in code.
``type``
========
The type of software that should be added. To add |SOFTWARE_NAME| this must be |SOFTWARE_NAME_BACKTICK|.
``options``
===========
The configuration options are the attributes that fall under the options for an application.

View File

@@ -0,0 +1,11 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _Database Payload List:
Available Database Payloads:
- ``SELECT``
- ``INSERT``
- ``DELETE``

View File

@@ -1,71 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Database Client Server
======================
Database Service
----------------
The ``DatabaseService`` provides a SQL database server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Creates a database file in the ``Node`` 's ``FileSystem`` upon creation.
- Handles connecting clients by maintaining a dictionary of connections mapped to session IDs.
- Authenticates connections using a configurable password.
- Simulates ``SELECT``, ``DELETE`` and ``INSERT`` SQL queries.
- Returns query results and status codes back to clients.
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Clients connect, execute queries, and disconnect.
- Service runs on TCP port 5432 by default.
Implementation
^^^^^^^^^^^^^^
- Creates the database file within the node's file system.
- Manages client connections in a dictionary by session ID.
- Processes SQL queries.
- Returns results and status codes in a standard dictionary format.
- Extends Service class for integration with ``SoftwareManager``.
Database Client
---------------
The DatabaseClient provides a client interface for connecting to the ``DatabaseService``.
Key features
^^^^^^^^^^^^
- Connects to the ``DatabaseService`` via the ``SoftwareManager``.
- Handles connecting and disconnecting.
- Executes SQL queries and retrieves result sets.
Usage
^^^^^
- Initialise with server IP address and optional password.
- Connect to the ``DatabaseService`` with ``connect``.
- Retrieve results in a dictionary.
- Disconnect when finished.
To create database backups:
- Configure the backup server on the ``DatabaseService`` by providing the Backup server ``IPv4Address`` with ``configure_backup``
- Create a backup using ``backup_database``. This fails if the backup server is not configured.
- Restore a backup using ``restore_backup``. By default, this uses the database created via ``backup_database``.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Connect and disconnect methods manage sessions.
- Payloads serialised as dictionaries for transmission.
- Extends base Application class.

View File

@@ -1,56 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
DNS Client Server
=================
DNS Server
----------
Also known as a DNS Resolver, the ``DNSServer`` provides a DNS Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates DNS requests and DNSPacket transfer across a network
- Registers domain names and the IP Address linked to the domain name
- Returns the IP address for a given domain name within a DNS Packet that a DNS Client can read
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
Implementation
^^^^^^^^^^^^^^
- DNS request and responses use a ``DNSPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
DNS Client
----------
The DNSClient provides a client interface for connecting to the ``DNSServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``DNSServer`` via the ``SoftwareManager``.
- Executes DNS lookup requests and keeps a cache of known domain name IP addresses.
- Handles connection to DNSServer and querying for domain name IP addresses.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
- Execute domain name checks with ``check_domain_exists``.
- ``DNSClient`` will automatically add the IP Address of the domain into its cache
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.

View File

@@ -1,135 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
FTP Client Server
=================
FTP Server
----------
Provides a FTP Client-Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the FTP server service.
- Service runs on FTP (command) port 21 by default. (TODO: look at in depth implementation of FTP PORT command)
Implementation
^^^^^^^^^^^^^^
- FTP request and responses use a ``FTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
FTP Client
----------
The ``FTPClient`` provides a client interface for connecting to the ``FTPServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``FTPServer`` via the ``SoftwareManager``.
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- PORT: specifies the port that server should connect to on the client (currently only uses ``Port.FTP``)
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- QUIT: disconnect from server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the FTP client service.
- Service runs on FTP (command) port 21 by default. (TODO: look at in depth implementation of FTP PORT command)
- Execute sending a file to the FTP server with ``send_file``
- Execute retrieving a file from the FTP server with ``request_file``
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to transfer files between each other.
- Extends base Service class.
Example Usage
-------------
Dependencies
^^^^^^^^^^^^
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.nodes.computer import Computer
from primaite.simulator.network.hardware.nodes.server import Server
from primaite.simulator.system.services.ftp.ftp_server import FTPServer
from primaite.simulator.system.services.ftp.ftp_client import FTPClient
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
Example peer to peer network
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
net = Network()
pc1 = Computer(
hostname="pc1",
ip_address="120.10.10.10",
subnet_mask="255.255.255.0",
operating_state=NodeOperatingState.ON # initialise the computer in an ON state
)
srv = Server(
hostname="srv",
ip_address="120.10.10.20",
subnet_mask="255.255.255.0",
operating_state=NodeOperatingState.ON # initialise the server in an ON state
)
net.connect(pc1.network_interface[1], srv.network_interface[1])
Install the FTP Server
^^^^^^^^^^^^^^^^^^^^^^
FTP Client should be pre installed on nodes
.. code-block:: python
srv.software_manager.install(FTPServer)
ftpserv: FTPServer = srv.software_manager.software['FTPServer']
Setting up the FTP Server
^^^^^^^^^^^^^^^^^^^^^^^^^
Set up the FTP Server with a file that the client will need to retrieve
.. code-block:: python
srv.file_system.create_file('my_file.png')
Check that file was retrieved
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
client.request_file(
src_folder_name='root',
src_file_name='my_file.png',
dest_folder_name='root',
dest_file_name='test.png',
dest_ip_address=IPv4Address("120.10.10.20")
)
print(client.get_file(folder_name="root", file_name="test.png"))

View File

@@ -0,0 +1,15 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. toctree::
:maxdepth: 1
:glob:
applications/*
More info :py:mod:`primaite.game.game.APPLICATION_TYPES_MAPPING`
.. include:: list_of_system_applications.rst
.. |SOFTWARE_TYPE| replace:: application

View File

@@ -0,0 +1,15 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. toctree::
:maxdepth: 1
:glob:
services/*
More info :py:mod:`primaite.game.game.SERVICE_TYPES_MAPPING`
.. include:: list_of_system_services.rst
.. |SOFTWARE_TYPE| replace:: service

View File

@@ -0,0 +1,16 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``system applications``
"""""""""""""""""""""""
Some applications are pre installed on nodes - this is similar to how some applications are included with the Operating System.
The application may not be configured as needed, in which case, see the relevant application page.
The list of applications that are considered system software are:
- ``WebBrowser``
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.HostNode.SYSTEM_SOFTWARE`

View File

@@ -0,0 +1,18 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``system services``
"""""""""""""""""""
Some services are pre installed on nodes - this is similar to how some services are included with the Operating System.
The service may not be configured as needed, in which case, see the relevant service page.
The list of services that are considered system software are:
- ``DNSClient``
- ``FTPClient``
- ``NTPClient``
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.HostNode.SYSTEM_SOFTWARE`

View File

@@ -1,54 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
NTP Client Server
=================
NTP Server
----------
The ``NTPServer`` provides a NTP Server simulation by extending the base Service class.
NTP Client
----------
The ``NTPClient`` provides a NTP Client simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates NTP requests and NTPPacket transfer across a network
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
^^^^^^^^^^^^^^
- NTP request and responses use a ``NTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
NTP Client
----------
The NTPClient provides a client interface for connecting to the ``NTPServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``NTPServer`` via the ``SoftwareManager``.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.

View File

@@ -0,0 +1,109 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DatabaseService:
DatabaseService
###############
The ``DatabaseService`` provides a SQL database server simulation by extending the base Service class.
Key capabilities
================
- Creates a database file in the ``FileSystem`` of the ``Node`` (which the ``DatabaseService`` is installed on) upon creation.
- Handles connecting clients by maintaining a dictionary of connections mapped to session IDs.
- Authenticates connections using a configurable password.
- Simulates ``SELECT``, ``DELETE`` and ``INSERT`` SQL queries.
- Returns query results and status codes back to clients.
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Clients connect, execute queries, and disconnect.
- Service runs on TCP port 5432 by default.
Implementation
==============
- Creates the database file within the node's file system.
- Manages client connections in a dictionary by session ID.
- Processes SQL queries.
- Returns results and status codes in a standard dictionary format.
- Extends Service class for integration with ``SoftwareManager``.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.database.database_service import DatabaseService
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install DatabaseService on server
server.software_manager.install(DatabaseService)
db_service: DatabaseService = server.software_manager.software.get("DatabaseService")
db_service.start()
# configure DatabaseService
db_service.configure_backup(IPv4Address("192.168.0.10"))
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: database_service
type: DatabaseService
options:
backup_server_ip: 192.168.0.10
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: DatabaseService
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DatabaseService``
``backup_server_ip``
""""""""""""""""""""
Optional. Default value is ``None``.
The IP Address of the backup server that the ``DatabaseService`` will use to create backups of the database.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
``password``
""""""""""""
Optional. Default value is ``None``.
The password that needs to be provided by connecting clients in order to create a successful connection.

View File

@@ -0,0 +1,99 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DNSClient:
DNSClient
#########
The DNSClient provides a client interface for connecting to the :ref:`DNSServer`.
Key features
============
- Connects to the :ref:`DNSServer` via the ``SoftwareManager``.
- Executes DNS lookup requests and keeps a cache of known domain name IP addresses.
- Handles connection to DNSServer and querying for domain name IP addresses.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
- Execute domain name checks with ``check_domain_exists``.
- ``DNSClient`` will automatically add the IP Address of the domain into its cache
Implementation
==============
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.dns.dns_client import DNSClient
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install DNSClient on server
server.software_manager.install(DNSClient)
dns_client: DNSClient = server.software_manager.software.get("DNSClient")
dns_client.start()
# configure DatabaseService
dns_client.dns_server = IPv4Address("192.168.0.10")
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: dns_client
type: DNSClient
options:
dns_server: 192.168.0.10
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: DNSClient
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DNSClient``
``dns_server``
""""""""""""""
Optional. Default value is ``None``.
The IP Address of the :ref:`DNSServer`.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.

View File

@@ -0,0 +1,98 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _DNSServer:
DNSServer
#########
Also known as a DNS Resolver, the ``DNSServer`` provides a DNS Server simulation by extending the base Service class.
Key capabilities
================
- Simulates DNS requests and DNSPacket transfer across a network
- Registers domain names and the IP Address linked to the domain name
- Returns the IP address for a given domain name within a DNS Packet that a DNS Client can read
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
Implementation
==============
- DNS request and responses use a ``DNSPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.dns.dns_server import DNSServer
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install DNSServer on server
server.software_manager.install(DNSServer)
dns_server: DNSServer = server.software_manager.software.get("DNSServer")
dns_server.start()
# configure DatabaseService
dns_server.dns_register("arcd.com", IPv4Address("192.168.10.10"))
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: dns_server
type: DNSServer
options:
domain_mapping:
arcd.com: 192.168.0.10
another-example.com: 192.168.10.10
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: DNSServer
.. |SOFTWARE_NAME_BACKTICK| replace:: ``DNSServer``
domain_mapping
""""""""""""""
Domain mapping takes the domain and IP Addresses as a key-value pairs i.e.
If the domain is "arcd.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``arcd.com: 192.168.0.10``
The key must be a string and the IP Address must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.

View File

@@ -0,0 +1,91 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _FTPClient:
FTPClient
#########
The ``FTPClient`` provides a client interface for connecting to the :ref:`FTPServer`.
Key features
============
- Connects to the :ref:`FTPServer` via the ``SoftwareManager``.
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- PORT: specifies the port that server should connect to on the client (currently only uses ``Port.FTP``)
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- QUIT: disconnect from server
- Leverages the Service base class for install/uninstall, status tracking, etc.
- :ref:`FTPClient` and ``FTPServer`` utilise port 21 (FTP) throughout all file transfer / request
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the FTP client service.
- Service runs on FTP (command) port 21 by default
- Execute sending a file to the FTP server with ``send_file``
- Execute retrieving a file from the FTP server with ``request_file``
Implementation
==============
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to transfer files between each other.
- Extends base Service class.
Examples
========
Python
""""""
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.ftp.ftp_client import FTPClient
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.10",
start_up_duration=0,
)
server.power_on()
# Install FTPClient on server
server.software_manager.install(FTPClient)
ftp_client: FTPClient = server.software_manager.software.get("FTPClient")
ftp_client.start()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: ftp_client
type: FTPClient
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: FTPClient
.. |SOFTWARE_NAME_BACKTICK| replace:: ``FTPClient``
**FTPClient has no configuration options**

View File

@@ -0,0 +1,94 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _FTPServer:
FTPServer
#########
Provides a FTP Client-Server simulation by extending the base Service class.
Key capabilities
================
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- Leverages the Service base class for install/uninstall, status tracking, etc.
- :ref:`FTPClient` and ``FTPServer`` utilise port 21 (FTP) throughout all file transfer / request
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the FTP server service.
- Service runs on FTP (command) port 21 by default
Implementation
==============
- FTP request and responses use a ``FTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Examples
========
Python
""""""
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.ftp.ftp_server import FTPServer
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install FTPServer on server
server.software_manager.install(FTPServer)
ftp_server: FTPServer = server.software_manager.software.get("FTPServer")
ftp_server.start()
ftp_server.server_password = "test"
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: ftp_server
type: FTPServer
options:
server_password: test
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: FTPServer
.. |SOFTWARE_NAME_BACKTICK| replace:: ``FTPServer``
``server_password``
"""""""""""""""""""
Optional. Default value is ``None``.
The password that needs to be provided by a connecting :ref:`FTPClient` in order to create a successful connection.

View File

@@ -0,0 +1,95 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _NTPClient:
NTPClient
#########
The NTPClient provides a client interface for connecting to the ``NTPServer``.
Key features
============
- Connects to the ``NTPServer`` via the ``SoftwareManager``.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
==============
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.
Examples
========
Python
""""""
.. code-block:: python
from ipaddress import IPv4Address
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.ntp.ntp_client import NTPClient
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install NTPClient on server
server.software_manager.install(NTPClient)
ntp_client: NTPClient = server.software_manager.software.get("NTPClient")
ntp_client.start()
ntp_client.configure(ntp_server_ip_address=IPv4Address("192.168.0.10"))
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: ntp_client
type: NTPClient
options:
ntp_server_ip: 192.168.0.10
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: NTPClient
.. |SOFTWARE_NAME_BACKTICK| replace:: ``NTPClient``
``ntp_server_ip``
"""""""""""""""""
Optional. Default value is ``None``.
The IP address of an NTP Server which provides a time that the ``NTPClient`` can synchronise to.
This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.

View File

@@ -0,0 +1,86 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _NTPServer:
NTPServer
#########
The ``NTPServer`` provides a NTP Server simulation by extending the base Service class.
NTP Client
==========
The ``NTPClient`` provides a NTP Client simulation by extending the base Service class.
Key capabilities
================
- Simulates NTP requests and NTPPacket transfer across a network
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
==============
- NTP request and responses use a ``NTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Examples
========
Python
""""""
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.ntp.ntp_server import NTPServer
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install NTPServer on server
server.software_manager.install(NTPServer)
ntp_server: NTPServer = server.software_manager.software.get("NTPServer")
ntp_server.start()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: ntp_server
type: NTPServer
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: NTPServer
.. |SOFTWARE_NAME_BACKTICK| replace:: ``NTPServer``
**NTPServer has no configuration options**

View File

@@ -0,0 +1,86 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
.. _WebServer:
WebServer
#########
Provides a Web Server simulation by extending the base Service class.
Key capabilities
================
- Simulates a web server with the capability to also request data from a database
- Allows the emulation of HTTP requests between client (e.g. a web browser) and server
- GET request sends a get all users request to the database server and returns an HTTP 200 status if the database is responsive
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
- A :ref:`DatabaseClient` must be installed and configured on the same node as the ``WebServer`` if it is intended to send a users request i.e.
in the case that the :ref:`WebBrowser` sends a request with users in its request path, the ``WebServer`` will utilise the ``DatabaseClient`` to send a request to the ``DatabaseService``
Implementation
==============
- HTTP request uses a ``HttpRequestPacket`` object
- HTTP response uses a ``HttpResponsePacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Examples
========
Python
""""""
.. code-block:: python
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.system.services.web_server.web_server import WebServer
# Create Server
server = Server(
hostname="server",
ip_address="192.168.2.2",
subnet_mask="255.255.255.0",
default_gateway="192.168.1.1",
start_up_duration=0,
)
server.power_on()
# Install WebServer on server
server.software_manager.install(WebServer)
web_server: WebServer = server.software_manager.software.get("WebServer")
web_server.start()
Via Configuration
"""""""""""""""""
.. code-block:: yaml
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: web_server
type: WebServer
Configuration
=============
.. include:: ../common/common_configuration.rst
.. |SOFTWARE_NAME| replace:: WebServer
.. |SOFTWARE_NAME_BACKTICK| replace:: ``WebServer``
**WebServer has no configuration options**

View File

@@ -16,6 +16,8 @@ ARP, ICMP, or the Web Client. This pathway exemplifies the structured processing
each frame reaches its intended target within the simulated environment.
.. image:: node_session_software_model_example.png
:width: 500
:align: center
Session Manager
---------------

View File

@@ -10,7 +10,7 @@ Software
Base Software
-------------
All software which inherits ``IOSoftware`` installed on a node will not work unless the node has been turned on.
Software which inherits ``IOSoftware`` installed on a node will not work unless the node has been turned on.
See :ref:`Node Start up and Shut down`
@@ -39,15 +39,27 @@ See :ref:`Node Start up and Shut down`
assert node.operating_state is NodeOperatingState.ON
assert web_server.operating_state is ServiceOperatingState.RUNNING # service turned back on when node is powered on
.. _List of Applications:
Services, Processes and Applications:
#####################################
Applications
############
.. toctree::
:maxdepth: 2
These are a list of applications that are currently available in PrimAITE:
database_client_server
data_manipulation_bot
dns_client_server
ftp_client_server
web_browser_and_web_server_service
.. include:: list_of_applications.rst
.. _List of Services:
Services
########
These are a list of services that are currently available in PrimAITE:
.. include:: list_of_services.rst
.. _List of Processes:
Processes
#########
`To be implemented`

View File

@@ -1,110 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Web Browser and Web Server Service
==================================
Web Server Service
------------------
Provides a Web Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates a web server with the capability to also request data from a database
- Allows the emulation of HTTP requests between client (e.g. a web browser) and server
- GET request sends a get all users request to the database server and returns an HTTP 200 status if the database is responsive
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
Implementation
^^^^^^^^^^^^^^
- HTTP request uses a ``HttpRequestPacket`` object
- HTTP response uses a ``HttpResponsePacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Web Browser (Web Client)
------------------------
The ``WebBrowser`` provides a client interface for connecting to the ``WebServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``WebServer`` via the ``SoftwareManager``.
- Simulates HTTP requests and HTTP packet transfer across a network
- Allows the emulation of HTTP requests between client and server:
- Automatically uses ``DNSClient`` to resolve domain names
- GET: performs an HTTP GET request from client to server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the ``WebBrowser``.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
- Execute sending an HTTP GET request with ``get_webpage``
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for making HTTP requests between an HTTP client and server.
- Extends base Service class.
Example Usage
-------------
Dependencies
^^^^^^^^^^^^
.. code-block:: python
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.nodes.computer import Computer
from primaite.simulator.network.hardware.nodes.server import Server
from primaite.simulator.system.applications.web_browser import WebBrowser
from primaite.simulator.system.services.web_server.web_server_service import WebServer
Example peer to peer network
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
net = Network()
pc1 = Computer(hostname="pc1", ip_address="192.168.1.50", subnet_mask="255.255.255.0")
srv = Server(hostname="srv", ip_address="192.168.1.10", subnet_mask="255.255.255.0")
pc1.power_on()
srv.power_on()
net.connect(pc1.network_interface[1], srv.network_interface[1])
Install the Web Server
^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
# web browser is automatically installed in computer nodes
# IRL this is usually included with an OS
client: WebBrowser = pc1.software_manager.software['WebBrowser']
# install web server
srv.software_manager.install(WebServer)
webserv: WebServer = srv.software_manager.software['WebServer']
Open the web page
^^^^^^^^^^^^^^^^^
Using a domain name to connect to a website requires setting up DNS Servers. For this example, it is possible to use the IP address directly
.. code-block:: python
# check that the get request succeeded
print(client.get_webpage("http://192.168.1.10")) # should be True

View File

@@ -12,14 +12,15 @@ and a domain controller for managing software and users.
Each node of the simulation 'tree' has responsibility for creating, deleting, and updating its direct descendants. Also,
when a component's ``describe_state()`` method is called, it will include the state of its descendants. The
``apply_request()`` method can be used to act on a component or one of its descendatnts. The diagram below shows the
``apply_request()`` method can be used to act on a component or one of its descendants. The diagram below shows the
relationship between components.
.. image:: _static/component_relationship.png
.. image:: ../../_static/component_relationship.png
:width: 500
:alt: The top level simulation object owns a NetworkContainer and a DomainController. The DomainController has a
list of accounts. The network container has links and nodes. Nodes can own switchports, NICs, FileSystem,
Application, Service, and Process.
:align: center
:alt: :: The top level simulation object owns a NetworkContainer and a DomainController. The DomainController has a
list of accounts. The network container has links and nodes. Nodes can own switchports, NICs, FileSystem,
Application, Service, and Process.
Actions

View File

@@ -3,7 +3,7 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Simulation State
==============
================
``SimComponent`` objects in the simulation have a method called ``describe_state`` which return a dictionary of the state of the component. This is used to report pertinent data that could impact an agent's actions or rewards. For instance, the name and health status of a node is reported, which can be used by a reward function to punish corrupted or compromised nodes and reward healthy nodes. Each ``SimComponent`` object reports not only its own attributes in the state but also those of its child components. I.e. a computer node will report the state of its ``FileSystem`` and the ``FileSystem`` will report the state of its files and folders. This happens by recursively calling the childrens' own ``describe_state`` methods.

View File

@@ -57,7 +57,7 @@ dev = [
"build==0.10.0",
"flake8==6.0.0",
"flake8-annotations",
"furo==2023.3.27",
"furo==2024.01.29",
"gputil==1.4.0",
"pip-licenses==4.3.0",
"pre-commit==2.20.0",
@@ -67,7 +67,7 @@ dev = [
"pytest-cov==4.0.0",
"pytest-flake8==1.1.1",
"setuptools==66",
"Sphinx==6.1.3",
"Sphinx==7.1.2",
"sphinx-copybutton==0.5.2",
"wheel==0.38.4"
]

View File

@@ -590,8 +590,8 @@ simulation:
nodes:
- ref: router_1
type: router
hostname: router_1
type: router
num_ports: 5
ports:
1:
@@ -626,18 +626,18 @@ simulation:
protocol: ICMP
- ref: switch_1
type: switch
hostname: switch_1
type: switch
num_ports: 8
- ref: switch_2
type: switch
hostname: switch_2
type: switch
num_ports: 8
- ref: domain_controller
type: server
hostname: domain_controller
type: server
ip_address: 192.168.1.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -649,8 +649,8 @@ simulation:
arcd.com: 192.168.1.12 # web server
- ref: web_server
type: server
hostname: web_server
type: server
ip_address: 192.168.1.12
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -666,8 +666,8 @@ simulation:
- ref: database_server
type: server
hostname: database_server
type: server
ip_address: 192.168.1.14
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -681,8 +681,8 @@ simulation:
type: FTPClient
- ref: backup_server
type: server
hostname: backup_server
type: server
ip_address: 192.168.1.16
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -692,8 +692,8 @@ simulation:
type: FTPServer
- ref: security_suite
type: server
hostname: security_suite
type: server
ip_address: 192.168.1.110
subnet_mask: 255.255.255.0
default_gateway: 192.168.1.1
@@ -704,8 +704,8 @@ simulation:
subnet_mask: 255.255.255.0
- ref: client_1
type: computer
hostname: client_1
type: computer
ip_address: 192.168.10.21
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
@@ -727,8 +727,8 @@ simulation:
type: DNSClient
- ref: client_2
type: computer
hostname: client_2
type: computer
ip_address: 192.168.10.22
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1

View File

@@ -572,7 +572,7 @@ class NetworkNICDisableAction(NetworkNICAbstractAction):
class ActionManager:
"""Class which manages the action space for an agent."""
_act_class_identifiers: Dict[str, type] = {
act_class_identifiers: Dict[str, type] = {
"DONOTHING": DoNothingAction,
"NODE_SERVICE_SCAN": NodeServiceScanAction,
"NODE_SERVICE_STOP": NodeServiceStopAction,
@@ -753,7 +753,7 @@ class ActionManager:
# and `options` is an optional dict of options to pass to the init method of the action class
act_type = act_spec.get("type")
act_options = act_spec.get("options", {})
self.actions[act_type] = self._act_class_identifiers[act_type](self, **global_action_args, **act_options)
self.actions[act_type] = self.act_class_identifiers[act_type](self, **global_action_args, **act_options)
self.action_map: Dict[int, Tuple[str, Dict]] = {}
"""

View File

@@ -13,7 +13,7 @@ the structure:
- type: DATABASE_FILE_INTEGRITY
weight: 0.5
options:
node_ref: database_server
node_name: database_server
folder_name: database
file_name: database.db
@@ -21,7 +21,7 @@ the structure:
- type: WEB_SERVER_404_PENALTY
weight: 0.5
options:
node_ref: web_server
node_name: web_server
service_ref: web_server_database_client
```
"""
@@ -184,7 +184,7 @@ class WebServer404Penalty(AbstractReward):
service_name = config.get("service_name")
if not (node_hostname and service_name):
msg = (
f"{cls.__name__} could not be initialised from config because node_ref and service_ref were not "
f"{cls.__name__} could not be initialised from config because node_name and service_ref were not "
"found in reward config."
)
_LOGGER.warning(msg)
@@ -245,12 +245,13 @@ class WebpageUnavailablePenalty(AbstractReward):
class RewardFunction:
"""Manages the reward function for the agent."""
__rew_class_identifiers: Dict[str, Type[AbstractReward]] = {
rew_class_identifiers: Dict[str, Type[AbstractReward]] = {
"DUMMY": DummyReward,
"DATABASE_FILE_INTEGRITY": DatabaseFileIntegrity,
"WEB_SERVER_404_PENALTY": WebServer404Penalty,
"WEBPAGE_UNAVAILABLE_PENALTY": WebpageUnavailablePenalty,
}
"""List of reward class identifiers."""
def __init__(self):
"""Initialise the reward function object."""
@@ -297,7 +298,7 @@ class RewardFunction:
for rew_component_cfg in config["reward_components"]:
rew_type = rew_component_cfg["type"]
weight = rew_component_cfg.get("weight", 1.0)
rew_class = cls.__rew_class_identifiers[rew_type]
rew_class = cls.rew_class_identifiers[rew_type]
rew_instance = rew_class.from_config(config=rew_component_cfg.get("options", {}))
new.register_component(component=rew_instance, weight=weight)
return new

View File

@@ -15,6 +15,7 @@ from primaite.simulator.network.hardware.base import NodeOperatingState
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.network.hardware.nodes.host.host_node import NIC
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.network.hardware.nodes.network.firewall import Firewall
from primaite.simulator.network.hardware.nodes.network.router import Router
from primaite.simulator.network.hardware.nodes.network.switch import Switch
from primaite.simulator.network.nmne import set_nmne_config
@@ -41,6 +42,7 @@ APPLICATION_TYPES_MAPPING = {
"DataManipulationBot": DataManipulationBot,
"DoSBot": DoSBot,
}
"""List of available applications that can be installed on nodes in the PrimAITE Simulation."""
SERVICE_TYPES_MAPPING = {
"DNSClient": DNSClient,
@@ -52,6 +54,7 @@ SERVICE_TYPES_MAPPING = {
"NTPClient": NTPClient,
"NTPServer": NTPServer,
}
"""List of available services that can be installed on nodes in the PrimAITE Simulation."""
class PrimaiteGameOptions(BaseModel):
@@ -228,28 +231,36 @@ class PrimaiteGame:
new_node = Computer(
hostname=node_cfg["hostname"],
ip_address=node_cfg["ip_address"],
subnet_mask=node_cfg["subnet_mask"],
subnet_mask=IPv4Address(node_cfg.get("subnet_mask", "255.255.255.0")),
default_gateway=node_cfg["default_gateway"],
dns_server=node_cfg["dns_server"],
operating_state=NodeOperatingState.ON,
dns_server=node_cfg.get("dns_server", None),
operating_state=NodeOperatingState.ON
if not (p := node_cfg.get("operating_state"))
else NodeOperatingState[p.upper()],
)
elif n_type == "server":
new_node = Server(
hostname=node_cfg["hostname"],
ip_address=node_cfg["ip_address"],
subnet_mask=node_cfg["subnet_mask"],
subnet_mask=IPv4Address(node_cfg.get("subnet_mask", "255.255.255.0")),
default_gateway=node_cfg["default_gateway"],
dns_server=node_cfg.get("dns_server"),
operating_state=NodeOperatingState.ON,
dns_server=node_cfg.get("dns_server", None),
operating_state=NodeOperatingState.ON
if not (p := node_cfg.get("operating_state"))
else NodeOperatingState[p.upper()],
)
elif n_type == "switch":
new_node = Switch(
hostname=node_cfg["hostname"],
num_ports=node_cfg.get("num_ports"),
operating_state=NodeOperatingState.ON,
num_ports=int(node_cfg.get("num_ports", "8")),
operating_state=NodeOperatingState.ON
if not (p := node_cfg.get("operating_state"))
else NodeOperatingState[p.upper()],
)
elif n_type == "router":
new_node = Router.from_config(node_cfg)
elif n_type == "firewall":
new_node = Firewall.from_config(node_cfg)
else:
_LOGGER.warning(f"invalid node type {n_type} in config")
if "services" in node_cfg:
@@ -262,6 +273,9 @@ class PrimaiteGame:
new_node.software_manager.install(SERVICE_TYPES_MAPPING[service_type])
new_service = new_node.software_manager.software[service_type]
game.ref_map_services[service_ref] = new_service.uuid
# start the service
new_service.start()
else:
msg = f"Configuration contains an invalid service type: {service_type}"
_LOGGER.error(msg)
@@ -281,18 +295,16 @@ class PrimaiteGame:
if service_type == "DatabaseService":
if "options" in service_cfg:
opt = service_cfg["options"]
new_service.password = opt.get("backup_server_ip", None)
new_service.configure_backup(backup_server=IPv4Address(opt.get("backup_server_ip")))
new_service.start()
if service_type == "FTPServer":
if "options" in service_cfg:
opt = service_cfg["options"]
new_service.server_password = opt.get("server_password")
new_service.start()
if service_type == "NTPClient":
if "options" in service_cfg:
opt = service_cfg["options"]
new_service.ntp_server = IPv4Address(opt.get("ntp_server_ip"))
new_service.start()
if "applications" in node_cfg:
for application_cfg in node_cfg["applications"]:
new_application = None
@@ -308,13 +320,16 @@ class PrimaiteGame:
_LOGGER.error(msg)
raise ValueError(msg)
# run the application
new_application.run()
if application_type == "DataManipulationBot":
if "options" in application_cfg:
opt = application_cfg["options"]
new_application.configure(
server_ip_address=IPv4Address(opt.get("server_ip")),
server_password=opt.get("server_password"),
payload=opt.get("payload"),
payload=opt.get("payload", "DELETE"),
port_scan_p_of_success=float(opt.get("port_scan_p_of_success", "0.1")),
data_manipulation_p_of_success=float(opt.get("data_manipulation_p_of_success", "0.1")),
)
@@ -329,7 +344,6 @@ class PrimaiteGame:
if "options" in application_cfg:
opt = application_cfg["options"]
new_application.target_url = opt.get("target_url")
elif application_type == "DoSBot":
if "options" in application_cfg:
opt = application_cfg["options"]
@@ -346,10 +360,20 @@ class PrimaiteGame:
for nic_num, nic_cfg in node_cfg["network_interfaces"].items():
new_node.connect_nic(NIC(ip_address=nic_cfg["ip_address"], subnet_mask=nic_cfg["subnet_mask"]))
# temporarily set to 0 so all nodes are initially on
new_node.start_up_duration = 0
new_node.shut_down_duration = 0
net.add_node(new_node)
new_node.power_on()
# run through the power on step if the node is to be turned on at the start
if new_node.operating_state == NodeOperatingState.ON:
new_node.power_on()
game.ref_map_nodes[node_ref] = new_node.uuid
# set start up and shut down duration
new_node.start_up_duration = int(node_cfg.get("start_up_duration", 3))
new_node.shut_down_duration = int(node_cfg.get("shut_down_duration", 3))
# 2. create links between nodes
for link_cfg in links_cfg:
node_a = net.nodes[game.ref_map_nodes[link_cfg["endpoint_a_ref"]]]

View File

@@ -12,8 +12,8 @@ class _SimOutput:
self._path: Path = (
_PRIMAITE_ROOT.parent.parent / "simulation_output" / datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
)
self.save_pcap_logs: bool = True
self.save_sys_logs: bool = True
self.save_pcap_logs: bool = False
self.save_sys_logs: bool = False
@property
def path(self) -> Path:

View File

@@ -1,7 +1,7 @@
# flake8: noqa
"""Core of the PrimAITE Simulator."""
from abc import ABC, abstractmethod
from typing import Callable, ClassVar, Dict, List, Optional, Union
from abc import abstractmethod
from typing import Callable, Dict, List, Optional, Union
from uuid import uuid4
from pydantic import BaseModel, ConfigDict, Field

View File

@@ -1,7 +1,7 @@
from __future__ import annotations
from ipaddress import IPv4Address
from typing import Any, Dict, Optional
from typing import Any, ClassVar, Dict, Optional
from primaite import getLogger
from primaite.simulator.network.hardware.base import IPWiredNetworkInterface, Link, Node
@@ -297,6 +297,16 @@ class HostNode(Node):
* Web Browser: Provides web browsing capabilities.
"""
SYSTEM_SOFTWARE: ClassVar[Dict] = {
"HostARP": HostARP,
"ICMP": ICMP,
"DNSClient": DNSClient,
"FTPClient": FTPClient,
"NTPClient": NTPClient,
"WebBrowser": WebBrowser,
}
"""List of system software that is automatically installed on nodes."""
network_interfaces: Dict[str, NIC] = {}
"The Network Interfaces on the node."
network_interface: Dict[int, NIC] = {}
@@ -313,23 +323,8 @@ class HostNode(Node):
This method equips the host with essential network services and applications, preparing it for various
network-related tasks and operations.
"""
# ARP Service
self.software_manager.install(HostARP)
# ICMP Service
self.software_manager.install(ICMP)
# DNS Client
self.software_manager.install(DNSClient)
# FTP Client
self.software_manager.install(FTPClient)
# NTP Client
self.software_manager.install(NTPClient)
# Web Browser
self.software_manager.install(WebBrowser)
for _, software_class in self.SYSTEM_SOFTWARE.items():
self.software_manager.install(software_class)
super()._install_system_software()

View File

@@ -1,8 +1,10 @@
from ipaddress import IPv4Address
from typing import Dict, Final, Optional, Union
from prettytable import MARKDOWN, PrettyTable
from pydantic import validate_call
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.network.hardware.nodes.network.router import (
AccessControlList,
ACLAction,
@@ -10,6 +12,8 @@ from primaite.simulator.network.hardware.nodes.network.router import (
RouterInterface,
)
from primaite.simulator.network.transmission.data_link_layer import Frame
from primaite.simulator.network.transmission.network_layer import IPProtocol
from primaite.simulator.network.transmission.transport_layer import Port
from primaite.simulator.system.core.sys_log import SysLog
from primaite.utils.validators import IPV4Address
@@ -483,3 +487,123 @@ class Firewall(Router):
"""
self.configure_port(DMZ_PORT_ID, ip_address, subnet_mask)
self.dmz_port.enable()
@classmethod
def from_config(cls, cfg: dict) -> "Firewall":
"""Create a firewall based on a config dict."""
firewall = Firewall(
hostname=cfg["hostname"],
operating_state=NodeOperatingState.ON
if not (p := cfg.get("operating_state"))
else NodeOperatingState[p.upper()],
)
if "ports" in cfg:
internal_port = cfg["ports"]["internal_port"]
external_port = cfg["ports"]["external_port"]
dmz_port = cfg["ports"]["dmz_port"]
# configure internal port
firewall.configure_internal_port(
ip_address=IPV4Address(internal_port.get("ip_address")),
subnet_mask=IPV4Address(internal_port.get("subnet_mask", "255.255.255.0")),
)
# configure external port
firewall.configure_external_port(
ip_address=IPV4Address(external_port.get("ip_address")),
subnet_mask=IPV4Address(external_port.get("subnet_mask", "255.255.255.0")),
)
# configure dmz port
firewall.configure_dmz_port(
ip_address=IPV4Address(dmz_port.get("ip_address")),
subnet_mask=IPV4Address(dmz_port.get("subnet_mask", "255.255.255.0")),
)
if "acl" in cfg:
# acl rules for internal_inbound_acl
if cfg["acl"]["internal_inbound_acl"]:
for r_num, r_cfg in cfg["acl"]["internal_inbound_acl"].items():
firewall.internal_inbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
# acl rules for internal_outbound_acl
if cfg["acl"]["internal_outbound_acl"]:
for r_num, r_cfg in cfg["acl"]["internal_outbound_acl"].items():
firewall.internal_outbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
# acl rules for dmz_inbound_acl
if cfg["acl"]["dmz_inbound_acl"]:
for r_num, r_cfg in cfg["acl"]["dmz_inbound_acl"].items():
firewall.dmz_inbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
# acl rules for dmz_outbound_acl
if cfg["acl"]["dmz_outbound_acl"]:
for r_num, r_cfg in cfg["acl"]["dmz_outbound_acl"].items():
firewall.dmz_outbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
# acl rules for external_inbound_acl
if cfg["acl"]["external_inbound_acl"]:
for r_num, r_cfg in cfg["acl"]["external_inbound_acl"].items():
firewall.external_inbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
# acl rules for external_outbound_acl
if cfg["acl"]["external_outbound_acl"]:
for r_num, r_cfg in cfg["acl"]["external_outbound_acl"].items():
firewall.external_outbound_acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
protocol=None if not (p := r_cfg.get("protocol")) else IPProtocol[p],
src_ip_address=r_cfg.get("src_ip"),
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
if "routes" in cfg:
for route in cfg.get("routes"):
firewall.route_table.add_route(
address=IPv4Address(route.get("address")),
subnet_mask=IPv4Address(route.get("subnet_mask", "255.255.255.0")),
next_hop_ip_address=IPv4Address(route.get("next_hop_ip_address")),
metric=float(route.get("metric", 0)),
)
return firewall

View File

@@ -1398,21 +1398,23 @@ class Router(NetworkNode):
:return: Configured router.
:rtype: Router
"""
new = Router(
router = Router(
hostname=cfg["hostname"],
num_ports=cfg.get("num_ports"),
operating_state=NodeOperatingState.ON,
num_ports=int(cfg.get("num_ports", "5")),
operating_state=NodeOperatingState.ON
if not (p := cfg.get("operating_state"))
else NodeOperatingState[p.upper()],
)
if "ports" in cfg:
for port_num, port_cfg in cfg["ports"].items():
new.configure_port(
router.configure_port(
port=port_num,
ip_address=port_cfg["ip_address"],
subnet_mask=port_cfg["subnet_mask"],
subnet_mask=IPv4Address(port_cfg.get("subnet_mask", "255.255.255.0")),
)
if "acl" in cfg:
for r_num, r_cfg in cfg["acl"].items():
new.acl.add_rule(
router.acl.add_rule(
action=ACLAction[r_cfg["action"]],
src_port=None if not (p := r_cfg.get("src_port")) else Port[p],
dst_port=None if not (p := r_cfg.get("dst_port")) else Port[p],
@@ -1421,4 +1423,12 @@ class Router(NetworkNode):
dst_ip_address=r_cfg.get("dst_ip"),
position=r_num,
)
return new
if "routes" in cfg:
for route in cfg.get("routes"):
router.route_table.add_route(
address=IPv4Address(route.get("address")),
subnet_mask=IPv4Address(route.get("subnet_mask", "255.255.255.0")),
next_hop_ip_address=IPv4Address(route.get("next_hop_ip_address")),
metric=float(route.get("metric", 0)),
)
return router

View File

@@ -9,11 +9,18 @@ _LOGGER = getLogger(__name__)
class IPProtocol(Enum):
"""Enum representing transport layer protocols in IP header."""
"""
Enum representing transport layer protocols in IP header.
.. _List of IPProtocols:
"""
TCP = "tcp"
"""Transmission Control Protocol."""
UDP = "udp"
"""User Datagram Protocol."""
ICMP = "icmp"
"""Internet Control Message Protocol."""
class Precedence(Enum):

View File

@@ -5,7 +5,11 @@ from pydantic import BaseModel
class Port(Enum):
"""Enumeration of common known TCP/UDP ports used by protocols for operation of network applications."""
"""
Enumeration of common known TCP/UDP ports used by protocols for operation of network applications.
.. _List of Ports:
"""
NONE = 0
"Place holder for a non-port."

View File

@@ -23,6 +23,7 @@ class DatabaseService(Service):
"""
password: Optional[str] = None
"""Password that needs to be provided by clients if they want to connect to the DatabaseService."""
backup_server_ip: IPv4Address = None
"""IP address of the backup server."""

View File

@@ -1,3 +1,10 @@
# Basic Switched network
#
# -------------- -------------- --------------
# | client_1 |------| switch_1 |------| client_2 |
# -------------- -------------- --------------
#
training_config:
rl_framework: SB3
rl_algorithm: PPO
@@ -134,6 +141,17 @@ simulation:
default_gateway: 192.168.10.1
dns_server: 192.168.1.10
# pre installed services and applications
- ref: client_3
type: computer
hostname: client_3
ip_address: 192.168.10.23
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
dns_server: 192.168.1.10
start_up_duration: 0
shut_down_duration: 0
operating_state: "OFF"
# pre installed services and applications
links:
- ref: switch_1___client_1

View File

@@ -0,0 +1,300 @@
# Network with DMZ
#
# An example network configuration with an internal network, a DMZ network and a couple of external networks.
#
# ............................................................................
# . .
# . Internal Network .
# . .
# . -------------- -------------- -------------- .
# . | client_1 |------| switch_1 |--------| router_1 | .
# . -------------- -------------- -------------- .
# . (Computer) | .
# ........................................................|...................
# |
# |
# ........................................................|...................
# . | .
# . DMZ Network | .
# . | .
# . ---------------- -------------- -------------- .
# . | dmz_server |------| switch_2 |------| firewall | .
# . ---------------- -------------- -------------- .
# . (Server) | .
# ........................................................|...................
# |
# External Network |
# |
# |
# ----------------------- -------------- ---------------------
# | external_computer |------| switch_3 |------| external_server |
# ----------------------- -------------- ---------------------
#
training_config:
rl_framework: SB3
rl_algorithm: PPO
seed: 333
n_learn_episodes: 1
n_eval_episodes: 5
max_steps_per_episode: 128
deterministic_eval: false
n_agents: 1
agent_references:
- defender
io_settings:
save_checkpoints: true
checkpoint_interval: 5
save_step_metadata: false
save_pcap_logs: true
save_sys_logs: true
game:
max_episode_length: 256
ports:
- ARP
- DNS
- HTTP
- POSTGRES_SERVER
protocols:
- ICMP
- TCP
- UDP
agents:
- ref: client_1_green_user
team: GREEN
type: GreenWebBrowsingAgent
observation_space:
type: UC2GreenObservation
action_space:
action_list:
- type: DONOTHING
- type: NODE_APPLICATION_EXECUTE
options:
nodes:
- node_name: client_1
applications:
- application_name: WebBrowser
max_folders_per_node: 1
max_files_per_folder: 1
max_services_per_node: 1
max_applications_per_node: 1
reward_function:
reward_components:
- type: DUMMY
agent_settings:
start_settings:
start_step: 5
frequency: 4
variance: 3
simulation:
network:
nodes:
- ref: client_1
type: computer
hostname: client_1
ip_address: 192.168.0.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.0.1
dns_server: 192.168.20.11
start_up_duration: 0
shut_down_duration: 0
- ref: switch_1
type: switch
hostname: switch_1
num_ports: 8
start_up_duration: 0
shut_down_duration: 0
- ref: router_1
type: router
hostname: router_1
num_ports: 5
start_up_duration: 0
shut_down_duration: 0
ports:
1:
ip_address: 192.168.0.1
subnet_mask: 255.255.255.0
2:
ip_address: 192.168.1.1
subnet_mask: 255.255.255.0
acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
23:
action: PERMIT
protocol: ICMP
routes:
- address: 192.168.10.10 # route to dmz_server
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.2
metric: 0
- address: 192.168.20.10 # route to external_computer
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.2
metric: 0
- address: 192.168.20.11 # route to external_server
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.2
metric: 0
- ref: dmz_server
type: server
hostname: dmz_server
ip_address: 192.168.10.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.10.1
dns_server: 192.168.20.11
start_up_duration: 0
shut_down_duration: 0
- ref: switch_2
type: switch
hostname: switch_2
num_ports: 8
start_up_duration: 0
shut_down_duration: 0
- ref: firewall
type: firewall
hostname: firewall
start_up_duration: 0
shut_down_duration: 0
ports:
external_port: # port 1
ip_address: 192.168.20.1
subnet_mask: 255.255.255.0
internal_port: # port 2
ip_address: 192.168.1.2
subnet_mask: 255.255.255.0
dmz_port: # port 3
ip_address: 192.168.10.1
subnet_mask: 255.255.255.0
acl:
internal_inbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
23:
action: PERMIT
protocol: ICMP
internal_outbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
23:
action: PERMIT
protocol: ICMP
dmz_inbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
23:
action: PERMIT
protocol: ICMP
dmz_outbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
23:
action: PERMIT
protocol: ICMP
external_inbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
external_outbound_acl:
22:
action: PERMIT
src_port: ARP
dst_port: ARP
routes:
- address: 192.168.0.10 # route to client_1
subnet_mask: 255.255.255.0
next_hop_ip_address: 192.168.1.1
metric: 0
- ref: switch_3
type: switch
hostname: switch_3
num_ports: 8
start_up_duration: 0
shut_down_duration: 0
- ref: external_computer
type: computer
hostname: external_computer
ip_address: 192.168.20.10
subnet_mask: 255.255.255.0
default_gateway: 192.168.20.1
dns_server: 192.168.20.11
start_up_duration: 0
shut_down_duration: 0
- ref: external_server
type: server
hostname: external_server
ip_address: 192.168.20.11
subnet_mask: 255.255.255.0
default_gateway: 192.168.20.1
start_up_duration: 0
shut_down_duration: 0
services:
- ref: domain_controller_dns_server
type: DNSServer
links:
- ref: client_1___switch_1
endpoint_a_ref: client_1
endpoint_a_port: 1
endpoint_b_ref: switch_1
endpoint_b_port: 1
- ref: router_1___switch_1
endpoint_a_ref: router_1
endpoint_a_port: 1
endpoint_b_ref: switch_1
endpoint_b_port: 8
- ref: router_1___firewall
endpoint_a_ref: firewall
endpoint_a_port: 2 # internal firewall port
endpoint_b_ref: router_1
endpoint_b_port: 2
- ref: firewall___switch_2
endpoint_a_ref: firewall
endpoint_a_port: 3 # dmz firewall port
endpoint_b_ref: switch_2
endpoint_b_port: 8
- ref: dmz_server___switch_2
endpoint_a_ref: dmz_server
endpoint_a_port: 1
endpoint_b_ref: switch_2
endpoint_b_port: 1
- ref: firewall___switch_3
endpoint_a_ref: firewall
endpoint_a_port: 1 # external firewall port
endpoint_b_ref: switch_3
endpoint_b_port: 8
- ref: external_computer___switch_3
endpoint_a_ref: external_computer
endpoint_a_port: 1
endpoint_b_ref: switch_3
endpoint_b_port: 1
- ref: external_server___switch_3
endpoint_a_ref: external_server
endpoint_a_port: 1
endpoint_b_ref: switch_3
endpoint_b_port: 2

View File

@@ -1,9 +1,11 @@
# © Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Tuple, Union
import pytest
import yaml
from _pytest.monkeypatch import MonkeyPatch
from primaite import getLogger, PRIMAITE_PATHS
from primaite.game.agent.actions import ActionManager
@@ -12,6 +14,7 @@ from primaite.game.agent.observations import ICSObservation, ObservationManager
from primaite.game.agent.rewards import RewardFunction
from primaite.game.game import PrimaiteGame
from primaite.session.session import PrimaiteSession
from primaite.simulator import SIM_OUTPUT
from primaite.simulator.file_system.file_system import FileSystem
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.nodes.host.computer import Computer
@@ -29,6 +32,7 @@ from primaite.simulator.system.services.dns.dns_client import DNSClient
from primaite.simulator.system.services.dns.dns_server import DNSServer
from primaite.simulator.system.services.service import Service
from primaite.simulator.system.services.web_server.web_server import WebServer
from tests import TEST_ASSETS_ROOT
from tests.mock_and_patch.get_session_path_mock import temp_user_sessions_path
ACTION_SPACE_NODE_VALUES = 1
@@ -37,6 +41,21 @@ ACTION_SPACE_NODE_ACTION_VALUES = 1
_LOGGER = getLogger(__name__)
@pytest.fixture(scope="function", autouse=True)
def set_syslog_output_to_true():
"""Will be run before each test."""
monkeypatch = MonkeyPatch()
monkeypatch.setattr(
SIM_OUTPUT,
"path",
Path(TEST_ASSETS_ROOT.parent.parent / "simulation_output" / datetime.now().strftime("%Y-%m-%d_%H-%M-%S")),
)
monkeypatch.setattr(SIM_OUTPUT, "save_pcap_logs", True)
monkeypatch.setattr(SIM_OUTPUT, "save_sys_logs", True)
yield
class TestService(Service):
"""Test Service class"""

View File

@@ -0,0 +1,19 @@
from pathlib import Path
from typing import Union
import yaml
from primaite.game.game import PrimaiteGame
from tests import TEST_ASSETS_ROOT
BASIC_CONFIG = TEST_ASSETS_ROOT / "configs/basic_switched_network.yaml"
DMZ_NETWORK = TEST_ASSETS_ROOT / "configs/dmz_network.yaml"
def load_config(config_path: Union[str, Path]) -> PrimaiteGame:
"""Returns a PrimaiteGame object which loads the contents of a given yaml path."""
with open(config_path, "r") as f:
cfg = yaml.safe_load(f)
return PrimaiteGame.from_config(cfg)

View File

@@ -0,0 +1,111 @@
import pytest
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.network.hardware.nodes.network.firewall import Firewall
from primaite.simulator.network.hardware.nodes.network.router import ACLAction
from primaite.simulator.network.transmission.network_layer import IPProtocol
from primaite.simulator.network.transmission.transport_layer import Port
from tests.integration_tests.configuration_file_parsing import DMZ_NETWORK, load_config
@pytest.fixture(scope="function")
def dmz_config() -> Network:
game = load_config(DMZ_NETWORK)
return game.simulation.network
def test_firewall_is_in_configuration(dmz_config):
"""Test that the firewall exists in the configuration file."""
network: Network = dmz_config
firewall: Firewall = network.get_node_by_hostname("firewall")
assert firewall
assert firewall.operating_state == NodeOperatingState.ON
def test_firewall_routes_are_correctly_added(dmz_config):
"""Test that the firewall routes have been correctly added to and configured in the network."""
network: Network = dmz_config
firewall: Firewall = network.get_node_by_hostname("firewall")
client_1: Computer = network.get_node_by_hostname("client_1")
dmz_server: Server = network.get_node_by_hostname("dmz_server")
external_computer: Computer = network.get_node_by_hostname("external_computer")
external_server: Server = network.get_node_by_hostname("external_server")
# there should be a route to client_1
assert firewall.route_table.find_best_route(client_1.network_interface[1].ip_address)
assert dmz_server.ping(client_1.network_interface[1].ip_address)
assert external_computer.ping(client_1.network_interface[1].ip_address)
assert external_server.ping(client_1.network_interface[1].ip_address)
# client_1 should be able to ping other nodes
assert client_1.ping(dmz_server.network_interface[1].ip_address)
assert client_1.ping(external_computer.network_interface[1].ip_address)
assert client_1.ping(external_server.network_interface[1].ip_address)
def test_firewall_acl_rules_correctly_added(dmz_config):
"""
Test that makes sure that the firewall ACLs have been configured onto the firewall
node via configuration file.
"""
firewall: Firewall = dmz_config.get_node_by_hostname("firewall")
# ICMP and ARP should be allowed internal_inbound
assert firewall.internal_inbound_acl.num_rules == 2
assert firewall.internal_inbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.internal_inbound_acl.acl[22].src_port == Port.ARP
assert firewall.internal_inbound_acl.acl[22].dst_port == Port.ARP
assert firewall.internal_inbound_acl.acl[23].action == ACLAction.PERMIT
assert firewall.internal_inbound_acl.acl[23].protocol == IPProtocol.ICMP
assert firewall.internal_inbound_acl.implicit_action == ACLAction.DENY
# ICMP and ARP should be allowed internal_outbound
assert firewall.internal_outbound_acl.num_rules == 2
assert firewall.internal_outbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.internal_outbound_acl.acl[22].src_port == Port.ARP
assert firewall.internal_outbound_acl.acl[22].dst_port == Port.ARP
assert firewall.internal_outbound_acl.acl[23].action == ACLAction.PERMIT
assert firewall.internal_outbound_acl.acl[23].protocol == IPProtocol.ICMP
assert firewall.internal_outbound_acl.implicit_action == ACLAction.DENY
# ICMP and ARP should be allowed dmz_inbound
assert firewall.dmz_inbound_acl.num_rules == 2
assert firewall.dmz_inbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.dmz_inbound_acl.acl[22].src_port == Port.ARP
assert firewall.dmz_inbound_acl.acl[22].dst_port == Port.ARP
assert firewall.dmz_inbound_acl.acl[23].action == ACLAction.PERMIT
assert firewall.dmz_inbound_acl.acl[23].protocol == IPProtocol.ICMP
assert firewall.dmz_inbound_acl.implicit_action == ACLAction.DENY
# ICMP and ARP should be allowed dmz_outbound
assert firewall.dmz_outbound_acl.num_rules == 2
assert firewall.dmz_outbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.dmz_outbound_acl.acl[22].src_port == Port.ARP
assert firewall.dmz_outbound_acl.acl[22].dst_port == Port.ARP
assert firewall.dmz_outbound_acl.acl[23].action == ACLAction.PERMIT
assert firewall.dmz_outbound_acl.acl[23].protocol == IPProtocol.ICMP
assert firewall.dmz_outbound_acl.implicit_action == ACLAction.DENY
# ICMP and ARP should be allowed external_inbound
assert firewall.external_inbound_acl.num_rules == 1
assert firewall.external_inbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.external_inbound_acl.acl[22].src_port == Port.ARP
assert firewall.external_inbound_acl.acl[22].dst_port == Port.ARP
# external_inbound should have implicit action PERMIT
# ICMP does not have a provided ACL Rule but implicit action should allow anything
assert firewall.external_inbound_acl.implicit_action == ACLAction.PERMIT
# ICMP and ARP should be allowed external_outbound
assert firewall.external_outbound_acl.num_rules == 1
assert firewall.external_outbound_acl.acl[22].action == ACLAction.PERMIT
assert firewall.external_outbound_acl.acl[22].src_port == Port.ARP
assert firewall.external_outbound_acl.acl[22].dst_port == Port.ARP
# external_outbound should have implicit action PERMIT
# ICMP does not have a provided ACL Rule but implicit action should allow anything
assert firewall.external_outbound_acl.implicit_action == ACLAction.PERMIT

View File

@@ -0,0 +1,69 @@
import pytest
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from primaite.simulator.network.hardware.nodes.host.server import Server
from primaite.simulator.network.hardware.nodes.network.router import ACLAction, Router
from primaite.simulator.network.transmission.network_layer import IPProtocol
from primaite.simulator.network.transmission.transport_layer import Port
from tests.integration_tests.configuration_file_parsing import DMZ_NETWORK, load_config
@pytest.fixture(scope="function")
def dmz_config() -> Network:
game = load_config(DMZ_NETWORK)
return game.simulation.network
def test_router_is_in_configuration(dmz_config):
"""Test that the router exists in the configuration file."""
network: Network = dmz_config
router_1: Router = network.get_node_by_hostname("router_1")
assert router_1
assert router_1.operating_state == NodeOperatingState.ON
def test_router_routes_are_correctly_added(dmz_config):
"""Test that makes sure that router routes have been added from the configuration file."""
network: Network = dmz_config
router_1: Router = network.get_node_by_hostname("router_1")
client_1: Computer = network.get_node_by_hostname("client_1")
dmz_server: Server = network.get_node_by_hostname("dmz_server")
external_computer: Computer = network.get_node_by_hostname("external_computer")
external_server: Server = network.get_node_by_hostname("external_server")
# there should be a route to dmz_server
assert router_1.route_table.find_best_route(dmz_server.network_interface[1].ip_address)
assert client_1.ping(dmz_server.network_interface[1].ip_address)
assert external_computer.ping(dmz_server.network_interface[1].ip_address)
assert external_server.ping(dmz_server.network_interface[1].ip_address)
# there should be a route to external_computer
assert router_1.route_table.find_best_route(external_computer.network_interface[1].ip_address)
assert client_1.ping(external_computer.network_interface[1].ip_address)
assert dmz_server.ping(external_computer.network_interface[1].ip_address)
assert external_server.ping(external_computer.network_interface[1].ip_address)
# there should be a route to external_server
assert router_1.route_table.find_best_route(external_server.network_interface[1].ip_address)
assert client_1.ping(external_server.network_interface[1].ip_address)
assert dmz_server.ping(external_server.network_interface[1].ip_address)
assert external_computer.ping(external_server.network_interface[1].ip_address)
def test_router_acl_rules_correctly_added(dmz_config):
"""Test that makes sure that the router ACLs have been configured onto the router node via configuration file."""
router_1: Router = dmz_config.get_node_by_hostname("router_1")
# ICMP and ARP should be allowed
assert router_1.acl.num_rules == 2
assert router_1.acl.acl[22].action == ACLAction.PERMIT
assert router_1.acl.acl[22].src_port == Port.ARP
assert router_1.acl.acl[22].dst_port == Port.ARP
assert router_1.acl.acl[23].action == ACLAction.PERMIT
assert router_1.acl.acl[23].protocol == IPProtocol.ICMP
assert router_1.acl.implicit_action == ACLAction.DENY

View File

@@ -0,0 +1,44 @@
from primaite.config.load import example_config_path
from primaite.simulator.network.container import Network
from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
from primaite.simulator.network.hardware.nodes.host.computer import Computer
from tests.integration_tests.configuration_file_parsing import BASIC_CONFIG, DMZ_NETWORK, load_config
def test_example_config():
"""Test that the example config can be parsed properly."""
game = load_config(example_config_path())
network: Network = game.simulation.network
assert len(network.nodes) == 10 # 10 nodes in example network
assert len(network.routers) == 1 # 1 router in network
assert len(network.switches) == 2 # 2 switches in network
assert len(network.servers) == 5 # 5 servers in network
def test_dmz_config():
"""Test that the DMZ network config can be parsed properly."""
game = load_config(DMZ_NETWORK)
network: Network = game.simulation.network
assert len(network.nodes) == 9 # 9 nodes in network
assert len(network.routers) == 2 # 2 routers in network
assert len(network.switches) == 3 # 3 switches in network
assert len(network.servers) == 2 # 2 servers in network
def test_basic_config():
"""Test that the basic_switched_network config can be parsed properly."""
game = load_config(BASIC_CONFIG)
network: Network = game.simulation.network
assert len(network.nodes) == 4 # 4 nodes in network
client_1: Computer = network.get_node_by_hostname("client_1")
assert client_1.operating_state == NodeOperatingState.ON
client_2: Computer = network.get_node_by_hostname("client_2")
assert client_2.operating_state == NodeOperatingState.ON
# client 3 should not be online
client_3: Computer = network.get_node_by_hostname("client_3")
assert client_3.operating_state == NodeOperatingState.OFF