oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'feature/#3110-userguide-fixes' into feature/3110-userguide-fixes-Charlie

Browse Source
This commit is contained in:
Charlie Crane
2025-03-13 09:24:02 +00:00
parent 3fa1104d2e 8b91c280cb
commit 2107e2f166
21 changed files with 407 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/_templates/custom-class-template.rst vendored
View File

@@ -12,7 +12,8 @@
.. autoclass:: {{ objname }}
:members:
:show-inheritance:
:inherited-members:
:inherited-members: BaseModel
:exclude-members: model_computed_fields, model_config, model_fields
:special-members: __init__, __call__, __add__, __mul__
{% block methods %}
@@ -22,7 +23,14 @@
.. autosummary::
:nosignatures:
{% for item in methods %}
{%- if not item.startswith('_') %}
{%- if not item.startswith('_') and item not in [
'construct', 'copy', 'dict', 'from_orm', 'json', 'model_construct',
'model_copy', 'model_dump', 'model_dump_json', 'model_json_schema',
'model_parametrized_name', 'model_post_init', 'model_rebuild', '',
'model_validate', 'model_validate_json', 'model_validate_strings',
'parse_file', 'parse_obj', 'parse_raw', 'schema', 'schema_json',
'update_forward_refs', 'validate',
] %}
~{{ name }}.{{ item }}
{%- endif -%}
{%- endfor %}
@@ -35,7 +43,12 @@
.. autosummary::
{% for item in attributes %}
{%- if not item.startswith('_') and item not in [
'model_computed_fields', 'model_config', 'model_extra', 'model_fields',
'model_fields_set',
] %}
~{{ name }}.{{ item }}
{%- endif -%}
{%- endfor %}
{% endif %}
{% endblock %}

5
docs/source/configuration/simulation/nodes/network_examples.rst
View File

@@ -389,7 +389,7 @@ connections, but the ACL that allows the nodes in the LAN to communicate with th
pc_1 = network.get_node_by_hostname("pc_1")
pc_1.ping(pc_1.default_gateway)
pc_1.sys_log.show()
pc_1.sys_log.show()
If SysLog capture is toggled on and the simulation log level is set to INFO, the `pc_1` the result of the ping should be
captured in the `pc_1` SysLog:
@@ -443,7 +443,8 @@ SomeTech. This extended network includes detailed sub-networks with specialised
complex routing capabilities, and robust security protocols implemented through Access Control Lists (ACLs). Designed
to mimic the intricacies of actual network environments, this network provides a detailed look at how various network
components interact and function together to support both internal corporate activities and external communications.
NB: the network described here is not the same as the UC7 network used by notebooks such as ``UC7-Training,ipynb`` or
the network in ``Privilege-Escalation-and-Data-Loss-Example.ipynb``.
.. image:: images/primaite_example_multi_lan_with_internet_network_dark.png
:align: center

2
docs/source/how_to_guides/custom_actions.rst
View File

@@ -18,7 +18,7 @@ An example of a custom action is seen below, with key information about what is
.. code:: Python
class ExampleActionClass(AbstractAction, identifier="ExampleActions"):
class ExampleActionClass(AbstractAction, discriminator="ExampleActions"):
"""Example Action Class"""
config: ExampleAction.ConfigSchema(AbstractAction.ConfigSchema)

154
docs/source/how_to_guides/primaite_yaml_migration_guide.rst
View File

@@ -40,7 +40,7 @@ More information can be found in the detailed in the configuration page: :ref:`i
No reformatting required for ``game`` section.
If users have installed plugins that introduce new ports or protocols then the game must be configured with use them.
If users have installed plugins that introduce new ports or protocols then the game can be configured with use them.
This can be done by adding to the ``ports`` and ``protocols`` list as shown in the yaml snippet below:
@@ -67,31 +67,145 @@ This can be done by adding to the ``ports`` and ``protocols`` list as shown in t
``agents``
==========
PrimAITE 4.0.0 removes the requirement for agents to use indexes in actions.
PrimAITE 4.0.0 changes action parameters to use meaningful names instead of indexes.
To match the new schema, 3.0.0 agent's must adhere to the following:
To match the new schema, agent configs written for PrimAITE 3.X should make the following changes:
- The ``action_list`` sub-section within the ``action_space`` is no longer required and can be removed.
- The ``options`` sub-section can also be removed. (Note that you do not accidentally remove ``options`` sub-section within the ``observation_space``)
- The agent that require an ``action_map`` sub-section require the following alterations:
- Action's must now be converted to kebab-case:
- Action ``options`` that previously required identifiers now instead require names.
``action_space``
----------------
- remove the ``options``, and ``action_list`` sections.
- update the ``action_map`` to use the new naming schema for actions, they use kebab case instead of camel case. A conversion table is provided below.
- update the ``action_map`` to follow the new parameter schemas. ID-based parameters were replaced with name-based parameters. Use your old config's ``action_space.options`` field to find the appropriate mapping for action parameters in your particular scenario.
- ``node_id`` is now ``node_name``
- ``application_id`` is now ``application_name``
- ``service_id`` is now ``service_name``
- ``folder_id`` is now ``folder_name``
- ``nic_id`` is now ``nic_num`` (and is now 1-indexed instead of 0-indexed for consistency with the simulation)
- ``port_id`` is now ``port_num`` (and is now 1-indexed instead of 0-indexed for consistency with the simulation)
- ``source_ip_id`` is now ``src_ip``
- ``source_wildcard_id`` is now ``src_wildcard``
- ``source_port_id`` is now ``src_port``
- ``dest_port_id`` is now ``dst_port``
- ``dest_wildcard_id`` is now ``dst_wildcard``
- ``dest_port_id`` is now ``dst_port``
- ``protocol_id`` is now ``protocol``
**Example on how to map old paramater IDs to new paramter names**
.. code-block:: yaml
# scan webapp service (4.0.0)
1:
action: node-service-scan # kebab-case
options:
node_name: web_server # IDs are no longer used - reference the name directly.
service_name: web-server
game:
max_episode_length: 128
ports:
- FTP
- HTTP
protocols:
- TCP
- UDP
# ...
options:
nodes:
- node_name: PC-1
applications:
- application_name: DatabaseClient
folders:
- folder_name: downloads
files:
- file_name: chrome.exe
- folder_name: other_folder
files:
- file_name: firefox.exe
- node_name: PC-2
applications:
- application_name: WebBrowser
folders:
- folder_name: folder_1
files:
- file_name: file2.jpg
- folder_name: folder_2
files:
- file_name: file3.jpg
- node_name: PC-3
services:
- service_name: FTPClient
- node_name: PC-4
- node_name: PC-5
max_folders_per_node: 1
max_files_per_folder: 1
max_services_per_node: 2
max_nics_per_node: 8
max_acl_rules: 10
ip_list:
# 0 reserved for padding to align with observations
# 1 reserved for ALL ips
- 192.168.1.11 # 2
- 200.10.1.10 # 3
wildcard_list:
- 0.0.0.1 # 0
- 0.0.0.255 # 1
- 0.0.255.255 # 2
From the above old-style YAML ``action_space.options`` example, the following changes should be made to action map:
- Actions with ``node_id: 0`` should use ``node_name: PC-1``
- Actions with ``node_id: 1`` should use ``node_name: PC-2``
- Actions with ``node_id: 2`` should use ``node_name: PC-3``
- Actions with ``node_id: 3`` should use ``node_name: PC-4``
- Actions with ``node_id: 4`` should use ``node_name: PC-5``
- Actions with ``node_id: 0`` and ``application_id: 0`` should use ``application_name: DatabaseClient`` (The application list is specific to each node)
- Actions with ``node_id: 1`` and ``application_id: 0`` should use ``application_name: WebBrowser`` (The application list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 0`` should use ``folder_name: downloads`` (The folder list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 1`` should use ``folder_name: other_folder`` (The folder list is specific to each node)
- Actions with ``node_id: 1`` and ``folder_id: 0`` should use ``folder_name: folder_1`` (The folder list is specific to each node)
- Actions with ``node_id: 1`` and ``folder_id: 1`` should use ``folder_name: folder_2`` (The folder list is specific to each node)
- Actions with ``node_id: 0`` and ``folder_id: 0`` and ``file_id: 0`` should use ``file_name: chrome.exe`` (The file list is specific to each node and folder)
- Actions with ``node_id: 0`` and ``folder_id: 1`` and ``file_id: 0`` should use ``file_name: firefox.exe`` (The file list is specific to each node and folder)
- Actions with ``node_id: 1`` and ``folder_id: 0`` and ``file_id: 0`` should use ``file_name: file2.jpg`` (The file list is specific to each node and folder)
- Actions with ``node_id: 1`` and ``folder_id: 1`` and ``file_id: 0`` should use ``file_name: file3.jpg`` (The file list is specific to each node and folder)
- Actions with ``nic_id: <N>`` should use ``nic_num: <N+1>``
- Actions with ``port_id: <N>`` should use ``port_num: <N+1>``
- Actions with ``source_ip_id: 0`` should not be present in your original config as this has no effect
- Actions with ``source_ip_id: 1`` should use ``src_ip: ALL``
- Actions with ``source_ip_id: 2`` should use ``src_ip: 192.168.1.11``
- Actions with ``source_ip_id: 3`` should use ``src_ip: 200.10.1.10``
- Actions with ``dest_ip_id: 0`` should not be present in your original config as this has no effect
- Actions with ``dest_ip_id: 1`` should use ``dst_ip: ALL``
- Actions with ``dest_ip_id: 2`` should use ``dst_ip: 192.168.1.11``
- Actions with ``dest_ip_id: 3`` should use ``dst_ip: 200.10.1.10``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.0.1``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.0.255``
- Actions with ``source_wildcard_id: 0`` should use ``src_wildcard: 0.0.255.255``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.0.1``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.0.255``
- Actions with ``dest_wildcard_id: 0`` should use ``dst_wildcard: 0.0.255.255``
- Actions with ``source_port_id: 0`` should not be present in your original config as this has no effect
- Actions with ``source_port_id: 1`` should use ``src_port: ALL``
- Actions with ``source_port_id: 2`` should use ``src_port: FTP``
- Actions with ``source_port_id: 3`` should use ``src_port: HTTP``
- Actions with ``dest_port_id: 0`` should not be present in your original config as this has no effect
- Actions with ``dest_port_id: 1`` should use ``dst_port: ALL``
- Actions with ``dest_port_id: 2`` should use ``dst_port: FTP``
- Actions with ``dest_port_id: 3`` should use ``dst_port: HTTP``
- Actions with ``protocol_id: 0`` should not be present in your original config as this has no effect
- Actions with ``protocol_id: 1`` should use ``protocol: ALL``
- Actions with ``protocol_id: 2`` should use ``protocol: TCP``
- Actions with ``protocol_id: 3`` should use ``protocol: UDP``
``observation_space``
---------------------
- the ``type`` parameter values now use lower kebab case. A conversion table is provided below.
``reward_function``
-------------------
- the ``type`` parameter values now use lower kebab case. A conversion table is provided below.
# scan webapp service (3.0.0)
1:
action: NODE_SERVICE_SCAN
options:
node_id: 1
service_id: 0
+-------------------------------------+-------------------------------------+
| *3.0.0 action name* | *4.0.0 action name* |

49
docs/source/request_system.rst
View File

@@ -78,17 +78,26 @@ The ``RequestType`` object stores a reference to a method that executes the requ
The ``RequestManager`` object stores a mapping between strings and request types. It is responsible for processing the request and passing it down the ownership tree. Technically, the ``RequestManager`` is itself a callable that accepts `request, context` tuple, and so it can be chained with other request managers.
A simple example without chaining can be seen in the :py:class:`primaite.simulator.file_system.file_system.File` class.
A simple example without chaining can be seen in the :py:class:`primaite.simulator.file_system.file_systemfile_system_item_abc.FileSystemItemABC` class.
.. code-block:: python
class File(FileSystemItemABC):
class FileSystemItemABC(SimComponent):
...
def _init_request_manager(self):
...
request_manager.add_request("scan", RequestType(func=lambda request, context: RequestResponse.from_bool(self.scan())))
request_manager.add_request("repair", RequestType(func=lambda request, context: RequestResponse.from_bool(self.repair())))
request_manager.add_request("restore", RequestType(func=lambda request, context: RequestResponse.from_bool(self.restore())))
rm.add_request(
name="scan", request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.scan()))
)
rm.add_request(
name="checkhash",
request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.check_hash())),
)
rm.add_request(
name="repair",
request_type=RequestType(func=lambda request, context: RequestResponse.from_bool(self.repair())),
)
...
*ellipses (``...``) used to omit code impertinent to this explanation*
@@ -103,27 +112,18 @@ An example of how this works is in the :py:class:`primaite.simulator.network.har
.. code-block:: python
class Node(SimComponent):
class Node(SimComponent, ABC):
...
def _init_request_manager(self):
def _init_request_manager(self) -> RequestManager:
...
# a regular action which is processed by the Node itself
request_manager.add_request("turn_on", RequestType(func=lambda request, context: self.turn_on()))
# if the Node receives a request where the first word is 'service', it will use a dummy manager
# called self._service_request_manager to pass on the request to the relevant service. This dummy
# manager is simply here to map the service name that that service's own action manager. This is
# done because the next string after "service" is always the name of that service, so we need an
# RequestManager to pop that string before sending it onto the relevant service's RequestManager.
# since there are potentially many services, create an request manager that can map service name
self._service_request_manager = RequestManager()
request_manager.add_request("service", RequestType(func=self._service_request_manager))
...
rm.add_request("service", RequestType(func=self._service_request_manager, validator=_node_is_on))
self._nic_request_manager = RequestManager()
rm.add_request("network_interface", RequestType(func=self._nic_request_manager, validator=_node_is_on))
rm.add_request("file_system", RequestType(func=self.file_system._request_manager, validator=_node_is_on))
def install_service(self, service):
self.services[service.name] = service
...
# Here, the service name is registered to allow passing actions between the node and the service.
self._service_request_manager.add_request(service.name, RequestType(func=service._request_manager))
This process is repeated until the request word corresponds to a callable function rather than another ``RequestManager`` .
@@ -142,3 +142,8 @@ The :py:class:`primaite.interface.request.RequestResponse<RequestResponse>` carr
For instance, the ``execute`` action on a :py:class:`primaite.simulator.system.applications.web_browser.WebBrowser<WebBrowser>` calls the ``get_webpage()`` method. ``get_webpage()`` returns a True if the webpage was successfully retrieved, and False if unsuccessful for any reason, such as being blocked by an ACL, or if the database server is unresponsive. The boolean returned from ``get_webpage()`` is used to create the request response with ``from_bool()``.
Just as the requests themselves were passed from owner to component, the request response is bubbled back up from component to owner until it arrives at the game layer.
Example notebooks
-----------------
Further examples of the request system and be found in ``Requests-and-Responses.ipynb``
and ``Terminal-Processing.ipynb`` notebooks.

25
docs/source/simulation_components/system/applications/web_browser.rst
View File

@@ -66,7 +66,7 @@ The :ref:`DNSClient` must be configured to use the :ref:`DNSServer`. The :ref:`D
web_browser.run()
# configure the WebBrowser
web_browser.target_url = "arcd.com"
web_browser.target_url = "example.com"
# once DNS server is configured with the correct domain mapping
# this should work
@@ -80,15 +80,13 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_computer
hostname: example_computer
type: computer
...
applications:
- ref: web_browser
type: web-browser
options:
target_url: http://arcd.com/
- hostname: example_computer
type: computer
...
applications:
- type: web-browser
options:
target_url: http://example.com/
Configuration
=============
@@ -101,11 +99,10 @@ The URL that the ``WebBrowser`` will request when ``get_webpage`` is called with
The URL can be in any format so long as the domain is within it e.g.
The domain ``arcd.com`` can be matched by
The domain ``example.com`` can be matched by
- http://arcd.com/
- http://arcd.com/users/
- arcd.com
- http://example.com/
- example.com
``Common Attributes``

16
docs/source/simulation_components/system/services/database_service.rst
View File

@@ -82,15 +82,13 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: database_service
type: database-service
options:
backup_server_ip: 192.168.0.10
- hostname: example_server
type: server
...
services:
- type: database-service
options:
backup_server_ip: 192.168.0.10
Configuration
=============

26
docs/source/simulation_components/system/services/dns_server.rst
View File

@@ -20,7 +20,7 @@ Key capabilities
Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
- Service runs on TCP port 53 by default.
Implementation
==============
@@ -58,7 +58,7 @@ Python
dns_server.start()
# configure DatabaseService
dns_server.dns_register("arcd.com", IPv4Address("192.168.10.10"))
dns_server.dns_register("example.com", IPv4Address("192.168.10.10"))
Via Configuration
@@ -69,17 +69,15 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: dns_server
type: dns-server
options:
domain_mapping:
arcd.com: 192.168.0.10
another-example.com: 192.168.10.10
- hostname: example_server
type: server
...
services:
- type: dns-server
options:
domain_mapping:
example.com: 192.168.0.10
another-example.com: 192.168.10.10
Configuration
=============
@@ -90,7 +88,7 @@ Configuration
Domain mapping takes the domain and IP Addresses as a key-value pairs i.e.
If the domain is "arcd.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``arcd.com: 192.168.0.10``
If the domain is "example.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``example.com: 192.168.0.10``
The key must be a string and the IP Address must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.

16
docs/source/simulation_components/system/services/ftp_server.rst
View File

@@ -69,15 +69,13 @@ Via Configuration
simulation:
network:
nodes:
- ref: example_server
hostname: example_server
type: server
...
services:
- ref: ftp_server
type: ftp-server
options:
server_password: test
- hostname: example_server
type: server
...
services:
- type: ftp-server
options:
server_password: test
Configuration
=============

6
docs/source/simulation_components/system/services/terminal.rst
View File

@@ -89,7 +89,7 @@ Agents can execute local commands without needing to perform a separate remote l
...
action: node-send-local-command
options:
node_id: 0
node_name: node_a
username: admin
password: admin
command: # Example command - Creates a file called 'cat.png' in the downloads folder.
@@ -112,7 +112,7 @@ Agents are able to use the terminal to login into remote nodes via ``SSH`` which
...
action: node-session-remote-login
options:
node_id: 0
node_name: node_a
username: admin
password: admin
remote_ip: 192.168.0.10 # Example Ip Address. (The remote host's IP that will be used by ssh)
@@ -129,7 +129,7 @@ After remotely logging into another host, an agent can use the ``node-send-remot
...
action: node-send-remote-command
options:
node_id: 0
node_name: node_a
remote_ip: 192.168.0.10
command:
- "file_system"

2
docs/source/simulation_components/system/services/web_server.rst
View File

@@ -21,7 +21,7 @@ Usage
=====
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
- Service runs on HTTP port 80 by default.
- A :ref:`DatabaseClient` must be installed and configured on the same node as the ``WebServer`` if it is intended to send a users request i.e.
in the case that the :ref:`WebBrowser` sends a request with users in its request path, the ``WebServer`` will utilise the ``DatabaseClient`` to send a request to the ``DatabaseService``

72
docs/source/state_system.rst
View File

@@ -13,19 +13,67 @@ This code snippet demonstrates how the state information is defined within the `
.. code-block:: python
class Node(SimComponent):
class Node(SimComponent, ABC):
"""
A basic Node class that represents a node on the network.
This class manages the state of the node, including the NICs (Network Interface Cards), accounts, applications,
services, processes, file system, and various managers like ARP, ICMP, SessionManager, and SoftwareManager.
:param hostname: The node hostname on the network.
:param operating_state: The node operating state, either ON or OFF.
"""
operating_state: NodeOperatingState = NodeOperatingState.OFF
"The hardware state of the node."
network_interfaces: Dict[str, NetworkInterface] = {}
"The Network Interfaces on the node."
network_interface: Dict[int, NetworkInterface] = {}
"The Network Interfaces on the node by port id."
accounts: Dict[str, Account] = {}
"All accounts on the node."
applications: Dict[str, Application] = {}
"All applications on the node."
services: Dict[str, Service] = {}
"All services on the node."
processes: Dict[str, Process] = {}
"All processes on the node."
file_system: FileSystem
"The nodes file system."
def describe_state(self) -> Dict:
state = super().describe_state()
state["operating_state"] = self.operating_state.value
state["services"] = {uuid: svc.describe_state() for uuid, svc in self.services.items()}
return state
...
class ConfigSchema(BaseModel, ABC):
"""Configuration Schema for Node based classes."""
class Service(SimComponent):
health_state: ServiceHealthState = ServiceHealthState.GOOD
def describe_state(self) -> Dict:
state = super().describe_state()
state["health_state"] = self.health_state.value
return state
...
revealed_to_red: bool = False
"Informs whether the node has been revealed to a red agent."
...
def describe_state(self) -> Dict:
"""
Produce a dictionary describing the current state of this object.
Please see :py:meth:`primaite.simulator.core.SimComponent.describe_state` for a more detailed explanation.
:return: Current state of this object and child objects.
:rtype: Dict
"""
state = super().describe_state()
state.update(
{
"hostname": self.config.hostname,
"operating_state": self.operating_state.value,
"NICs": {
eth_num: network_interface.describe_state()
for eth_num, network_interface in self.network_interface.items()
},
"file_system": self.file_system.describe_state(),
"applications": {app.name: app.describe_state() for app in self.applications.values()},
"services": {svc.name: svc.describe_state() for svc in self.services.values()},
"process": {proc.name: proc.describe_state() for proc in self.processes.values()},
"revealed_to_red": self.config.revealed_to_red,
}
)
return state
...