oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'feature/#3110-userguide-fixes' into feature/3110-userguide-fixes-Charlie

Browse Source
This commit is contained in:
Charlie Crane
2025-03-13 09:24:02 +00:00
parent 3fa1104d2e 8b91c280cb
commit 2107e2f166
21 changed files with 407 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/_templates/custom-class-template.rst vendored
View File

@@ -12,7 +12,8 @@
.. autoclass:: {{ objname }}
:members:
:show-inheritance:
:inherited-members:
:inherited-members: BaseModel
:exclude-members: model_computed_fields, model_config, model_fields
:special-members: __init__, __call__, __add__, __mul__
{% block methods %}
@@ -22,7 +23,14 @@
.. autosummary::
:nosignatures:
{% for item in methods %}
{%- if not item.startswith('_') %}
{%- if not item.startswith('_') and item not in [
'construct', 'copy', 'dict', 'from_orm', 'json', 'model_construct',
'model_copy', 'model_dump', 'model_dump_json', 'model_json_schema',
'model_parametrized_name', 'model_post_init', 'model_rebuild', '',
'model_validate', 'model_validate_json', 'model_validate_strings',
'parse_file', 'parse_obj', 'parse_raw', 'schema', 'schema_json',
'update_forward_refs', 'validate',
] %}
~{{ name }}.{{ item }}
{%- endif -%}
{%- endfor %}
@@ -35,7 +43,12 @@
.. autosummary::
{% for item in attributes %}
{%- if not item.startswith('_') and item not in [
'model_computed_fields', 'model_config', 'model_extra', 'model_fields',
'model_fields_set',
] %}
~{{ name }}.{{ item }}
{%- endif -%}
{%- endfor %}
{% endif %}
{% endblock %}

5
docs/source/configuration/simulation/nodes/network_examples.rst
View File

@@ -389,7 +389,7 @@ connections, but the ACL that allows the nodes in the LAN to communicate with th
pc_1 = network.get_node_by_hostname("pc_1")
pc_1.ping(pc_1.default_gateway)
pc_1.sys_log.show()
pc_1.sys_log.show()
If SysLog capture is toggled on and the simulation log level is set to INFO, the `pc_1` the result of the ping should be
captured in the `pc_1` SysLog:
@@ -443,7 +443,8 @@ SomeTech. This extended network includes detailed sub-networks with specialised
complex routing capabilities, and robust security protocols implemented through Access Control Lists (ACLs). Designed
to mimic the intricacies of actual network environments, this network provides a detailed look at how various network
components interact and function together to support both internal corporate activities and external communications.
NB: the network described here is not the same as the UC7 network used by notebooks such as ``UC7-Training,ipynb`` or
the network in ``Privilege-Escalation-and-Data-Loss-Example.ipynb``.
.. image:: images/primaite_example_multi_lan_with_internet_network_dark.png
:align: center

2
docs/source/how_to_guides/custom_actions.rst
View File

@@ -18,7 +18,7 @@ An example of a custom action is seen below, with key information about what is
.. code:: Python
class ExampleActionClass(AbstractAction, identifier="ExampleActions"):
class ExampleActionClass(AbstractAction, discriminator="ExampleActions"):
"""Example Action Class"""
config: ExampleAction.ConfigSchema(AbstractAction.ConfigSchema)

152
docs/source/how_to_guides/primaite_yaml_migration_guide.rst
View File

@@ -40,7 +40,7 @@ More information can be found in the detailed in the configuration page: :ref:`i
No reformatting required for ``game`` section.
If users have installed plugins that introduce new ports or protocols then the game must be configured with use them.
If users have installed plugins that introduce new ports or protocols then the game can be configured with use them.
This can be done by adding to the ``ports`` and ``protocols`` list as shown in the yaml snippet below:
@@ -67,31 +67,145 @@ This can be done by adding to the ``ports`` and ``protocols`` list as shown in t
``agents``
==========
PrimAITE 4.0.0 removes the requirement for agents to use indexes in actions.
PrimAITE 4.0.0 changes action parameters to use meaningful names instead of indexes.
To match the new schema, 3.0.0 agent's must adhere to the following:
To match the new schema, agent configs written for PrimAITE 3.X should make the following changes:
- The ``action_list`` sub-section within the ``action_space`` is no longer required and can be removed.
- The ``options`` sub-section can also be removed. (Note that you do not accidentally remove ``options`` sub-section within the ``observation_space``)
- The agent that require an ``action_map`` sub-section require the following alterations:
- Action's must now be converted to kebab-case:
- Action ``options`` that previously required identifiers now instead require names.
``action_space``
----------------
- remove the ``options``, and ``action_list`` sections.
- update the ``action_map`` to use the new naming schema for actions, they use kebab case instead of camel case. A conversion table is provided below.
- update the ``action_map`` to follow the new parameter schemas. ID-based parameters were replaced with name-based parameters. Use your old config's ``action_space.options`` field to find the appropriate mapping for action parameters in your particular scenario.
- ``node_id`` is now ``node_name``
- ``application_id`` is now ``application_name``
- ``service_id`` is now ``service_name``
- ``folder_id`` is now ``folder_name``
- ``nic_id`` is now ``nic_num`` (and is now 1-indexed instead of 0-indexed for consistency with the simulation)
- ``port_id`` is now ``port_num`` (and is now 1-indexed instead of 0-indexed for consistency with the simulation)
- ``source_ip_id`` is now ``src_ip``
- ``source_wildcard_id`` is now ``src_wildcard``
- ``source_port_id`` is now ``src_port``
- ``dest_port_id`` is now ``dst_port``
- ``dest_wildcard_id`` is now ``dst_wildcard``
- ``dest_port_id`` is now ``dst_port``
- ``protocol_id`` is now ``protocol``
**Example on how to map old paramater IDs to new paramter names**
.. code-block:: yaml
# scan webapp service (4.0.0)
1:
action: node-service-scan # kebab-case
options:
node_name: web_server # IDs are no longer used - reference the name directly.
service_name: web-server
game:
max_episode_length: 128
ports:
- FTP
- HTTP
protocols:
- TCP
- UDP
# ...
# scan webapp service (3.0.0)
1:
action: NODE_SERVICE_SCAN
options:
node_id: 1
service_id: 0
nodes:
- node_name: PC-1
applications:
- application_name: DatabaseClient
folders:
- folder_name: downloads
files:
- file_name: chrome.exe
- folder_name: other_folder
files:
- file_name: firefox.exe
- node_name: PC-2
applications:
- application_name: WebBrowser
folders:
- folder_name: folder_1
files:
- file_name: file2.jpg
- folder_name: folder_2
files:
- file_name: file3.jpg
- node_name: PC-3
services:
- service_name: FTPClient
- node_name: PC-4
- node_name: PC-5
max_folders_per_node: 1
max_files_per_folder: 1
max_services_per_node: 2
max_nics_per_node: 8
max_acl_rules: 10
ip_list:
# 0 reserved for padding to align with observations
# 1 reserved for ALL ips
- 192.168.1.11 # 2
- 200.10.1.10 # 3
wildcard_list:
- 0.0.0.1 # 0
- 0.0.0.255 # 1
- 0.0.255.255 # 2
From the above old-style YAML ``action_space.options`` example, the following changes should be made to action map:
- Actions with ``node_id: 0`` should use ``node_name: PC-1``
- Actions with ``node_id: 1`` should use ``node_name: PC-2``
- Actions with ``node_id: 2`` should use ``node_name: PC-3``
- Actions with ``node_id: 3`` should use ``node_name: PC-4``
- Actions with ``node_id: 4`` should use ``node_name: PC-5``
- Actions with ``node_id: 0`` and ``application_id: 0`` should use ``application_name: DatabaseClient`` (The application list is specific to each node)
- Actions with ``node_id: 1`` and ``application_id: 0`` should use ``application_name: WebBrowser`` (The application list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 0`` should use ``folder_name: downloads`` (The folder list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 1`` should use ``folder_name: other_folder`` (The folder list is specific to each node)
- Actions with ``node_id: 1`` and ``folder_id: 0`` should use ``folder_name: folder_1`` (The folder list is specific to each node)
- Actions with ``node_id: 1`` and ``folder_id: 1`` should use ``folder_name: folder_2`` (The folder list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 0`` and ``file_id: 0`` should use ``file_name: chrome.exe`` (The file list is specific to each node and folder)
- Actions with ``node_id: 0`` and ``folder_id: 1`` and ``file_id: 0`` should use ``file_name: firefox.exe`` (The file list is specific to each node and folder)
- Actions with ``node_id: 1`` and ``folder_id: 0`` and ``file_id: 0`` should use ``file_name: file2.jpg`` (The file list is specific to each node and folder)
- Actions with ``node_id: 1`` and ``folder_id: 1`` and ``file_id: 0`` should use ``file_name: file3.jpg`` (The file list is specific to each node and folder)
- Actions with ``nic_id: <N>`` should use ``nic_num: <N+1>``
- Actions with ``port_id: <N>`` should use ``port_num: <N+1>``
- Actions with ``source_ip_id: 0`` should not be present in your original config as this has no effect
- Actions with ``source_ip_id: 1`` should use ``src_ip: ALL``
- Actions with ``source_ip_id: 2`` should use ``src_ip: 192.168.1.11``
- Actions with ``source_ip_id: 3`` should use ``src_ip: 200.10.1.10``
- Actions with ``dest_ip_id: 0`` should not be present in your original config as this has no effect
- Actions with ``dest_ip_id: 1`` should use ``dst_ip: ALL``
- Actions with ``dest_ip_id: 2`` should use ``dst_ip: 192.168.1.11``
- Actions with ``dest_ip_id: 3`` should use ``dst_ip: 200.10.1.10``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.0.1``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.0.255``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.255.255``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.0.1``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.0.255``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.255.255``
- Actions with ``source_port_id: 0`` should not be present in your original config as this has no effect
- Actions with ``source_port_id: 1`` should use ``src_port: ALL``
- Actions with ``source_port_id: 2`` should use ``src_port: FTP``
- Actions with ``source_port_id: 3`` should use ``src_port: HTTP``
- Actions with ``dest_port_id: 0`` should not be present in your original config as this has no effect
- Actions with ``dest_port_id: 1`` should use ``dst_port: ALL``
- Actions with ``dest_port_id: 2`` should use ``dst_port: FTP``
- Actions with ``dest_port_id: 3`` should use ``dst_port: HTTP``
- Actions with ``protocol_id: 0`` should not be present in your original config as this has no effect
- Actions with ``protocol_id: 1`` should use ``protocol: ALL``
- Actions with ``protocol_id: 2`` should use ``protocol: TCP``
- Actions with ``protocol_id: 3`` should use ``protocol: UDP``
``observation_space``
---------------------
- the ``type`` parameter values now use lower kebab case. A conversion table is provided below.
``reward_function``
-------------------
- the ``type`` parameter values now use lower kebab case. A conversion table is provided below.
+-------------------------------------+-------------------------------------+
| *3.0.0 action name* | *4.0.0 action name* |

49
docs/source/request_system.rst
View File

@@ -78,17 +78,26 @@ The ``RequestType`` object stores a reference to a method that executes the requ
The ``RequestManager`` object stores a mapping between strings and request types. It is responsible for processing the request and passing it down the ownership tree. Technically, the ``RequestManager`` is itself a callable that accepts `request, context` tuple, and so it can be chained with other request managers.
A simple example without chaining can be seen in the :py:class:`primaite.simulator.file_system.file_system.File` class.
A simple example without chaining can be seen in the :py:class:`primaite.simulator.file_system.file_systemfile_system_item_abc.FileSystemItemABC` class.
.. code-block:: python
class File(FileSystemItemABC):
class FileSystemItemABC(SimComponent):
...
def _init_request_manager(self):
...
request_manager.add_request("scan", RequestType(func=lambda request, context: RequestResponse.from_bool(self.scan())))
request_manager.add_request("repair", RequestType(func=lambda request, context: RequestResponse.from_bool(self.repair())))
request_manager.add_request("restore", RequestType(func=lambda request, context: RequestResponse.from_bool(self.restore())))
rm.add_request(
name="scan", request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.scan()))
)
rm.add_request(
name="checkhash",
request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.check_hash())),
)
rm.add_request(
name="repair",
request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.repair())),
)
...
*ellipses (``...``) used to omit code impertinent to this explanation*
@@ -103,27 +112,18 @@ An example of how this works is in the :py:class:`primaite.simulator.network.har
.. code-block:: python
class Node(SimComponent):
class Node(SimComponent, ABC):
...
def _init_request_manager(self):
def _init_request_manager(self) -> RequestManager:
...
# a regular action which is processed by the Node itself
request_manager.add_request("turn_on", RequestType(func=lambda request, context: self.turn_on()))
# if the Node receives a request where the first word is 'service', it will use a dummy manager
# called self._service_request_manager to pass on the request to the relevant service. This dummy
# manager is simply here to map the service name that that service's own action manager. This is
# done because the next string after "service" is always the name of that service, so we need an
# RequestManager to pop that string before sending it onto the relevant service's RequestManager.
# since there are potentially many services, create an request manager that can map service name
self._service_request_manager = RequestManager()
request_manager.add_request("service", RequestType(func=self._service_request_manager))
...
rm.add_request("service", RequestType(func=self._service_request_manager, validator=_node_is_on))
self._nic_request_manager = RequestManager()
rm.add_request("network_interface", RequestType(func=self._nic_request_manager, validator=_node_is_on))
rm.add_request("file_system", RequestType(func=self.file_system._request_manager, validator=_node_is_on))
def install_service(self, service):
self.services[service.name] = service
...
# Here, the service name is registered to allow passing actions between the node and the service.
self._service_request_manager.add_request(service.name, RequestType(func=service._request_manager))
This process is repeated until the request word corresponds to a callable function rather than another ``RequestManager`` .
@@ -142,3 +142,8 @@ The :py:class:`primaite.interface.request.RequestResponse<RequestResponse>` carr
For instance, the ``execute`` action on a :py:class:`primaite.simulator.system.applications.web_browser.WebBrowser<WebBrowser>` calls the ``get_webpage()`` method. ``get_webpage()`` returns a True if the webpage was successfully retrieved, and False if unsuccessful for any reason, such as being blocked by an ACL, or if the database server is unresponsive. The boolean returned from ``get_webpage()`` is used to create the request response with ``from_bool()``.
Just as the requests themselves were passed from owner to component, the request response is bubbled back up from component to owner until it arrives at the game layer.
Example notebooks
-----------------
Further examples of the request system and be found in ``Requests-and-Responses.ipynb``
and ``Terminal-Processing.ipynb`` notebooks.

17
docs/source/simulation_components/system/applications/web_browser.rst
View File

@@ -66,7 +66,7 @@ The :ref:`DNSClient` must be configured to use the :ref:`DNSServer`. The :ref:`D
web_browser.run()
# configure the WebBrowser
web_browser.target_url = "arcd.com"
web_browser.target_url = "example.com"
# once DNS server is configured with the correct domain mapping
# this should work
@@ -80,15 +80,13 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_computer
hostname: example_computer
- hostname: example_computer
type: computer
...
applications:
- ref: web_browser
type: web-browser
- type: web-browser
options:
target_url: http://arcd.com/
target_url: http://example.com/
Configuration
=============
@@ -101,11 +99,10 @@ The URL that the ``WebBrowser`` will request when ``get_webpage`` is called with
The URL can be in any format so long as the domain is within it e.g.
The domain ``arcd.com`` can be matched by
The domain ``example.com`` can be matched by
- http://arcd.com/
- http://arcd.com/users/
- arcd.com
- http://example.com/
- example.com
``Common Attributes``

6
docs/source/simulation_components/system/services/database_service.rst
View File

@@ -82,13 +82,11 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
- hostname: example_server
type: server
...
services:
- ref: database_service
type: database-service
- type: database-service
options:
backup_server_ip: 192.168.0.10

14
docs/source/simulation_components/system/services/dns_server.rst
View File

@@ -20,7 +20,7 @@ Key capabilities
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
- Service runs on TCP port 53 by default.
Implementation
==============
@@ -58,7 +58,7 @@ Python
dns_server.start()
# configure DatabaseService
dns_server.dns_register("arcd.com", IPv4Address("192.168.10.10"))
dns_server.dns_register("example.com", IPv4Address("192.168.10.10"))
Via Configuration
@@ -69,16 +69,14 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
- hostname: example_server
type: server
...
services:
- ref: dns_server
type: dns-server
- type: dns-server
options:
domain_mapping:
arcd.com: 192.168.0.10
example.com: 192.168.0.10
another-example.com: 192.168.10.10
Configuration
@@ -90,7 +88,7 @@ Configuration
Domain mapping takes the domain and IP Addresses as a key-value pairs i.e.
If the domain is "arcd.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``arcd.com: 192.168.0.10``
If the domain is "example.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``example.com: 192.168.0.10``
The key must be a string and the IP Address must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.

6
docs/source/simulation_components/system/services/ftp_server.rst
View File

@@ -69,13 +69,11 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
- hostname: example_server
type: server
...
services:
- ref: ftp_server
type: ftp-server
- type: ftp-server
options:
server_password: test

6
docs/source/simulation_components/system/services/terminal.rst
View File

@@ -89,7 +89,7 @@ Agents can execute local commands without needing to perform a separate remote l
...
action: node-send-local-command
options:
node_id: 0
node_name: node_a
username: admin
password: admin
command: # Example command - Creates a file called 'cat.png' in the downloads folder.
@@ -112,7 +112,7 @@ Agents are able to use the terminal to login into remote nodes via ``SSH`` which
...
action: node-session-remote-login
options:
node_id: 0
node_name: node_a
username: admin
password: admin
remote_ip: 192.168.0.10 # Example Ip Address. (The remote host's IP that will be used by ssh)
@@ -129,7 +129,7 @@ After remotely logging into another host, an agent can use the ``node-send-remot
...
action: node-send-remote-command
options:
node_id: 0
node_name: node_a
remote_ip: 192.168.0.10
command:
- "file_system"

2
docs/source/simulation_components/system/services/web_server.rst
View File

@@ -21,7 +21,7 @@ Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
- Service runs on HTTP port 80 by default.
- A :ref:`DatabaseClient` must be installed and configured on the same node as the ``WebServer`` if it is intended to send a users request i.e.
in the case that the :ref:`WebBrowser` sends a request with users in its request path, the ``WebServer`` will utilise the ``DatabaseClient`` to send a request to the ``DatabaseService``

66
docs/source/state_system.rst
View File

@@ -13,19 +13,67 @@ This code snippet demonstrates how the state information is defined within the `
.. code-block:: python
class Node(SimComponent):
class Node(SimComponent, ABC):
"""
A basic Node class that represents a node on the network.
This class manages the state of the node, including the NICs (Network Interface Cards), accounts, applications,
services, processes, file system, and various managers like ARP, ICMP, SessionManager, and SoftwareManager.
:param hostname: The node hostname on the network.
:param operating_state: The node operating state, either ON or OFF.
"""
operating_state: NodeOperatingState = NodeOperatingState.OFF
"The hardware state of the node."
network_interfaces: Dict[str, NetworkInterface] = {}
"The Network Interfaces on the node."
network_interface: Dict[int, NetworkInterface] = {}
"The Network Interfaces on the node by port id."
accounts: Dict[str, Account] = {}
"All accounts on the node."
applications: Dict[str, Application] = {}
"All applications on the node."
services: Dict[str, Service] = {}
"All services on the node."
processes: Dict[str, Process] = {}
"All processes on the node."
file_system: FileSystem
"The nodes file system."
def describe_state(self) -> Dict:
state = super().describe_state()
state["operating_state"] = self.operating_state.value
state["services"] = {uuid: svc.describe_state() for uuid, svc in self.services.items()}
return state
...
class ConfigSchema(BaseModel, ABC):
"""Configuration Schema for Node based classes."""
class Service(SimComponent):
health_state: ServiceHealthState = ServiceHealthState.GOOD
...
revealed_to_red: bool = False
"Informs whether the node has been revealed to a red agent."
...
def describe_state(self) -> Dict:
"""
Produce a dictionary describing the current state of this object.
Please see :py:meth:`primaite.simulator.core.SimComponent.describe_state` for a more detailed explanation.
:return: Current state of this object and child objects.
:rtype: Dict
"""
state = super().describe_state()
state["health_state"] = self.health_state.value
state.update(
{
"hostname": self.config.hostname,
"operating_state": self.operating_state.value,
"NICs": {
eth_num: network_interface.describe_state()
for eth_num, network_interface in self.network_interface.items()
},
"file_system": self.file_system.describe_state(),
"applications": {app.name: app.describe_state() for app in self.applications.values()},
"services": {svc.name: svc.describe_state() for svc in self.services.values()},
"process": {proc.name: proc.describe_state() for proc in self.processes.values()},
"revealed_to_red": self.config.revealed_to_red,
}
)
return state
...

50
src/primaite/notebooks/Requests-and-Responses.ipynb
View File

@@ -8,21 +8,7 @@
"\n",
"© Crown-owned copyright 2025, Defence Science and Technology Laboratory UK\n",
"\n",
"Agents interact with the PrimAITE simulation via the Request system.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sending a request"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's set up a minimal network simulation and send some requests to see how it works."
"This notebook demonstrates how agents interact with the PrimAITE simulation via the Request system.\n"
]
},
{
@@ -45,6 +31,20 @@
"from primaite.simulator.sim_container import Simulation\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sending a request"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before we can send some requests we need to set up a minimal network simulation. The code snippet below creates a PrimAITE simulation with a singular generic host called `client`."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -71,9 +71,16 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"A request is structured in a similar way to a command line interface - a list of strings with positional args. It's also possible to supply an optional `context` dictionary. We will craft a request that stops the pre-installed DNSClient service on the client node.\n",
"Now we can simulation component to interact with, we can start sending some requests."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A request is structured in a similar way to a command line interface - a list of strings with positional args. It's also possible to supply an optional `context` dictionary. We will craft a request that stops the pre-installed `dns-client` service on the client node.\n",
"\n",
"First let's verify that the DNS Client is running on the client.\n"
"First let's verify that the `dns-client` is running on the client."
]
},
{
@@ -82,14 +89,15 @@
"metadata": {},
"outputs": [],
"source": [
"client.software_manager.show()"
"client.software_manager.show()\n",
"client.software_manager.software['dns-client'].operating_state.name"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Send a request to the simulator to stop the DNSClient."
"Send a request to the simulator to stop the `dns-client`."
]
},
{
@@ -110,7 +118,7 @@
"metadata": {},
"source": [
"\n",
"The request returns a `RequestResponse` object which tells us that the request was successfully executed. Let's verify that the DNS client is in a stopped state now."
"The request returns a `RequestResponse` object which tells us that the request was successfully executed. Let's verify that the `dns-client` is in a stopped state now."
]
},
{
@@ -196,7 +204,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, if we try to start the DNSClient back up, we get a failure because we cannot start software on a node that is turned off."
"Now, if we try to start the `dns-client` back up, we get a failure because we cannot start software on a node that is turned off."
]
},
{

132
src/primaite/notebooks/Terminal-Processing.ipynb
View File

@@ -13,23 +13,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simulation Layer Implementation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simulation Layer Implementation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook serves as a guide on the functionality and use of the new Terminal simulation component.\n",
"\n",
"The Terminal service comes pre-installed on most Nodes (The exception being Switches, as these are currently dumb). "
"This notebook serves as a guide on the functionality and use of the `terminal` service from both the simulation and game layers."
]
},
{
@@ -51,8 +35,47 @@
"from primaite.simulator.network.container import Network\n",
"from primaite.simulator.network.hardware.nodes.host.computer import Computer\n",
"from primaite.simulator.system.applications.red_applications.ransomware_script import RansomwareScript\n",
"from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection\n",
"from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simulation Layer Implementation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `terminal` service comes pre-installed on most node types. \n",
"\n",
"_The only exception to this being `switches` network nodes, this is because PrimAITE currently only implements 'dumb' switches. `routers` and `firewalls` however all support the `terminal`._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!primaite setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this notebook, the terminal is demoed on a basic network consisting of two computers, connected together via a link to form a basic LAN network which can be seen by the `basic_network()` method defined below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def basic_network() -> Network:\n",
" \"\"\"Utility function for creating a default network to demonstrate Terminal functionality\"\"\"\n",
" network = Network()\n",
@@ -65,7 +88,6 @@
" # \"startup_duration\": 0,\n",
" }\n",
" )\n",
" print(f\"{node_a=}\")\n",
" node_a.power_on()\n",
" node_b = Computer.from_config(\n",
" config = {\n",
@@ -85,9 +107,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The terminal can be accessed from a `Node` via the `software_manager` as demonstrated below. \n",
"\n",
"In the example, we have a basic network consisting of two computers, connected to form a basic network."
"After setting up the network, the terminal can be accessed from a `Node` via the `software_manager`:"
]
},
{
@@ -107,10 +127,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To be able to send commands from `node_a` to `node_b`, you will need to `login` to `node_b` first, using valid user credentials. In the example below, we are remotely logging in to the 'admin' account on `node_b`, from `node_a`. \n",
"If you are not logged in, any commands sent will be rejected by the remote.\n",
"However, before we're able to send commands from `node_a` to `node_b`, you will need to `login` to `node_b` first, using valid user credentials. \n",
"\n",
"Remote Logins return a RemoteTerminalConnection object, which can be used for sending commands to the remote node. "
"After providing successful credentials, the login method will return type of `TerminalClientConnection` object which can then be used for sending commands to the node. \n",
"\n",
"In the example below, we are remotely logging in to the default ***'admin'*** account on `node_b`, from `node_a` (If you are not logged in, any commands sent will be rejected by the remote).\n"
]
},
{
@@ -143,7 +164,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The new connection object allows us to forward commands to be executed on the target node. The example below demonstrates how you can remotely install an application on the target node."
"As we logged into a remote node, the login method return a `RemoteTerminalConnection` which allows us to forward commands to be executed on the target node. \n",
"\n",
"The example below demonstrates how you can remotely install an application on the target node."
]
},
{
@@ -168,7 +191,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The code block below demonstrates how the Terminal class allows the user of `terminal_a`, on `computer_a`, to send a command to `computer_b` to create a downloads folder. \n"
"As the terminal allows us to leverage the [request system](./Requests-and-Responses.ipynb) we have full access to the request manager on any simulation component. For example, the code snippet below demonstrates how we the `terminal` allows the user of `terminal_a`, on `computer_a`, to send a command (in the form of a request) to `computer_b` to create a downloads folder. \n"
]
},
{
@@ -263,14 +286,16 @@
"\n",
"This notebook section will detail the implementation of how the game layer utilises the terminal to support different agent actions.\n",
"\n",
"The ``Terminal`` is used in a variety of different ways in the game layer. Specifically, the terminal is leveraged to implement the following actions:\n",
"The ``terminal`` is directly leveraged to implement the following agent actions.\n",
"\n",
"\n",
"| Game Layer Action | Simulation Layer |\n",
"|-----------------------------------|--------------------------|\n",
"| ``node-send-local-command`` | Uses the given user credentials, creates a ``LocalTerminalSession`` and executes the given command and returns the ``RequestResponse``.\n",
"| ``node-session-remote-login`` | Uses the given user credentials and remote IP to create a ``RemoteTerminalSession``.\n",
"| ``node-send-remote-command`` | Uses the given remote IP to locate the correct ``RemoteTerminalSession``, executes the given command and returns the ``RequestsResponse``."
"| ``node-send-remote-command`` | Uses the given remote IP to locate the correct ``RemoteTerminalSession``, executes the given command and returns the ``RequestsResponse``.\n",
"\n",
"Additionally, the `terminal` is utilised extensively by the [c2 suite](./Command-and-Control-E2E-Demonstration.ipynb) and it's related actions. "
]
},
{
@@ -302,7 +327,7 @@
"outputs": [],
"source": [
"custom_terminal_agent = \"\"\"\n",
" - ref: CustomC2Agent\n",
" - ref: CustomTerminalAgent\n",
" team: RED\n",
" type: proxy-agent\n",
" action_space:\n",
@@ -373,22 +398,11 @@
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: node-send-local-command\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 1:\n",
" action: node-send-local-command\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" node_name: client_1\n",
" username: admin\n",
" password: admin\n",
" command:\n",
@@ -420,22 +434,11 @@
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: node-session-remote-login\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 2:\n",
" action: node-session-remote-login\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" node_name: client_1\n",
" username: admin\n",
" password: admin\n",
" remote_ip: 192.168.10.22 # client_2's ip address.\n",
@@ -461,24 +464,13 @@
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: node-send-remote-command\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 1:\n",
" 3:\n",
" action: node-send-remote-command\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" remote_ip: 192.168.10.22\n",
" commands:\n",
" node_name: client_1\n",
" remote_ip: 192.168.10.22 # client_2's ip address.\n",
" command:\n",
" - file_system\n",
" - create\n",
" - file\n",
@@ -501,7 +493,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},

31
src/primaite/notebooks/Training-an-RLLIB-MARL-System.ipynb
View File

@@ -8,7 +8,7 @@
"\n",
"© Crown-owned copyright 2025, Defence Science and Technology Laboratory UK\n",
"\n",
"This notebook will demonstrate how to use the `PrimaiteRayMARLEnv` to train a very basic system with two PPO agents."
"This notebook will demonstrate how to use the `PrimaiteRayMARLEnv` to train a very basic system with two PPO agents on the [UC2 scenario](./Data-Manipulation-E2E-Demonstration.ipynb)."
]
},
{
@@ -34,17 +34,20 @@
"outputs": [],
"source": [
"import yaml\n",
"\n",
"from primaite import PRIMAITE_PATHS\n",
"\n",
"import ray\n",
"from primaite import PRIMAITE_PATHS\n",
"from ray.rllib.algorithms.ppo import PPOConfig\n",
"from primaite.session.ray_envs import PrimaiteRayMARLEnv\n",
"from primaite.game.agent.scripted_agents import probabilistic_agent\n",
"\n",
"from primaite.session.ray_envs import PrimaiteRayMARLEnv"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open(PRIMAITE_PATHS.user_config_path / 'example_config/data_manipulation_marl.yaml', 'r') as f:\n",
" cfg = yaml.safe_load(f)\n",
"\n",
"ray.init(local_mode=True)"
]
},
@@ -114,6 +117,18 @@
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,

9
src/primaite/notebooks/Training-an-RLLib-Agent.ipynb
View File

@@ -8,7 +8,7 @@
"\n",
"© Crown-owned copyright 2025, Defence Science and Technology Laboratory UK\n",
"\n",
"This notebook will demonstrate how to use PrimaiteRayEnv to train a basic PPO agent."
"This notebook demonstrates how to use the ``PrimaiteRayEnv`` to train a basic PPO agent on the [UC2 scenario](./Data-Manipulation-E2E-Demonstration.ipynb)."
]
},
{
@@ -27,13 +27,10 @@
"outputs": [],
"source": [
"import yaml\n",
"from primaite.config.load import data_manipulation_config_path\n",
"\n",
"from primaite.session.ray_envs import PrimaiteRayEnv\n",
"import ray\n",
"from primaite.config.load import data_manipulation_config_path\n",
"from primaite.session.ray_envs import PrimaiteRayEnv\n",
"from ray.rllib.algorithms.ppo import PPOConfig\n",
"from primaite.game.agent.scripted_agents import probabilistic_agent\n",
"\n",
"\n",
"# If you get an error saying this config file doesn't exist, you may need to run `primaite setup` in your command line\n",
"# to copy the files to your user data path.\n",

14
src/primaite/notebooks/Training-an-SB3-Agent.ipynb
View File

@@ -8,7 +8,7 @@
"\n",
"© Crown-owned copyright 2025, Defence Science and Technology Laboratory UK\n",
"\n",
"This notebook will demonstrate how to use primaite to create and train a PPO agent, using a pre-defined configuration file."
"This notebook will demonstrate how to use primaite to create and train a PPO agent, using a pre-defined configuration file using the [UC2 scenario](./Data-Manipulation-E2E-Demonstration.ipynb)."
]
},
{
@@ -33,21 +33,11 @@
"metadata": {},
"outputs": [],
"source": [
"from primaite.game.game import PrimaiteGame\n",
"from primaite.session.environment import PrimaiteGymEnv\n",
"from primaite.game.agent.scripted_agents import probabilistic_agent\n",
"from primaite.config.load import data_manipulation_config_path\n",
"import yaml"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from primaite.config.load import data_manipulation_config_path"
]
},
{
"cell_type": "code",
"execution_count": null,

9
src/primaite/notebooks/Using-Episode-Schedules.ipynb
View File

@@ -48,7 +48,6 @@
"from primaite.session.environment import PrimaiteGymEnv\n",
"from primaite import PRIMAITE_PATHS\n",
"from prettytable import PrettyTable\n",
"from primaite.game.agent.scripted_agents import probabilistic_agent, data_manipulation_bot\n",
"scenario_path = PRIMAITE_PATHS.user_config_path / \"example_config/scenario_with_placeholders\""
]
},
@@ -239,7 +238,7 @@
"### Episode 2\n",
"When we reset the environment again, it moves onto episode 2, where it will bring in greens_1 and reds_1 for green and red agent definitions. Let's verify the agent names and that they take actions at the defined frequency.\n",
"\n",
"Most green actions will be `node_application_execute` while red will `DONOTHING` except at steps 10 and 20."
"Most green actions will be `node-application-execute` while red will `do-nothing` except at steps 10 and 20."
]
},
{
@@ -270,7 +269,7 @@
"### Episode 3\n",
"When we reset the environment again, it moves onto episode 3, where it will bring in greens_2 and reds_2 for green and red agent definitions. Let's verify the agent names and that they take actions at the defined frequency.\n",
"\n",
"Now, green will perform `node_application_execute` only 5% of the time, while red will perform `node_application_execute` more frequently than before."
"Now, green will perform `node-application-execute` only 5% of the time, while red will perform `node-application-execute` more frequently than before."
]
},
{
@@ -353,7 +352,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"In the 0th episode, simulation_variant_1.yaml is loaded in and the server gets a `DatabaseService`, while client_1 gets `DatabaseClient`."
"In the 0th episode, `simulation_variant_1.yaml` is loaded in and the server gets a `database-service`, while client_1 gets `database-client`."
]
},
{
@@ -382,7 +381,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"In the 1st episode, `simulation_variant_2.yaml` is loaded in, therefore the server gets a `FTPServer` and client_1 gets a `RansomwareScript`."
"In the 1st episode, `simulation_variant_2.yaml` is loaded in, therefore the server gets a `ftp-server` and client_1 gets a `ransomware-script`."
]
},
{

15
src/primaite/notebooks/multi-processing.ipynb
View File

@@ -8,7 +8,7 @@
"\n",
"© Crown-owned copyright 2025, Defence Science and Technology Laboratory UK\n",
"\n",
"This notebook uses SubprocVecEnv from SB3."
"This notebook uses **SubprocVecEnv** from **SB3** to train an agent on the [UC2 scenario](./Data-Manipulation-E2E-Demonstration.ipynb)."
]
},
{
@@ -37,15 +37,7 @@
"from stable_baselines3 import PPO\n",
"from stable_baselines3.common.utils import set_random_seed\n",
"from stable_baselines3.common.vec_env import SubprocVecEnv\n",
"from primaite.session.environment import PrimaiteGymEnv\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from primaite.session.environment import PrimaiteGymEnv\n",
"from primaite.config.load import data_manipulation_config_path"
]
},
@@ -72,12 +64,11 @@
"metadata": {},
"outputs": [],
"source": [
"\n",
"EPISODE_LEN = 128\n",
"NUM_EPISODES = 10\n",
"NO_STEPS = EPISODE_LEN * NUM_EPISODES\n",
"BATCH_SIZE = 32\n",
"LEARNING_RATE = 3e-4\n"
"LEARNING_RATE = 3e-4"
]
},
{

2
src/primaite/simulator/system/services/terminal/terminal.py
View File

@@ -23,8 +23,6 @@ from primaite.utils.validation.ip_protocol import PROTOCOL_LOOKUP
from primaite.utils.validation.port import PORT_LOOKUP
# TODO 2824: Since remote terminal connections and remote user sessions are the same thing, we could refactor
# the terminal to leverage the user session manager's list. This way we avoid potential bugs and code ducplication
class TerminalClientConnection(BaseModel):
"""
TerminalClientConnection Class.

8
tests/unit_tests/_primaite/_simulator/_network/_hardware/test_node_actions.py
View File

@@ -61,9 +61,6 @@ def test_node_os_scan(node):
"""Test OS Scanning."""
node.operating_state = NodeOperatingState.ON
# add process to node
# TODO implement processes
# add services to node
node.software_manager.install(DummyService)
service = node.software_manager.software.get("dummy-service")
@@ -95,7 +92,6 @@ def test_node_os_scan(node):
node.apply_timestep(timestep=i)
# should update the state of all items
# TODO assert process.health_state_visible == SoftwareHealthState.COMPROMISED
assert service.health_state_visible == SoftwareHealthState.COMPROMISED
assert application.health_state_visible == SoftwareHealthState.COMPROMISED
assert folder.visible_health_status == FileSystemItemHealthStatus.CORRUPT
@@ -107,9 +103,6 @@ def test_node_red_scan(node):
"""Test revealing to red"""
node.operating_state = NodeOperatingState.ON
# add process to node
# TODO implement processes
# add services to node
node.software_manager.install(DummyService)
service = node.software_manager.software.get("dummy-service")
@@ -138,7 +131,6 @@ def test_node_red_scan(node):
node.apply_timestep(timestep=i)
# should update the state of all items
# TODO assert process.revealed_to_red is True
assert service.revealed_to_red is True
assert application.revealed_to_red is True
assert folder.revealed_to_red is True