#2257: rearrange software pages + creating a list of applications and services which is hopefully a single point that should be referred to

This commit is contained in:
Czar Echavez
2024-02-23 08:55:32 +00:00
parent 98fb28cbbc
commit 5836ea68e3
42 changed files with 329 additions and 351 deletions

View File

@@ -27,6 +27,11 @@ with open("../src/primaite/VERSION", "r") as file:
# The full version, including alpha/beta/rc tags
release = version
# set global variables
rst_prolog = f"""
.. |VERSION| replace:: {release}
"""
html_title = f"{project} v{release} docs"
# -- General configuration ---------------------------------------------------

View File

@@ -2,8 +2,8 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Primaite v3 config
******************
PrimAITE |VERSION| Configuration
********************************
PrimAITE uses a single configuration file to define everything needed to train and evaluate an RL policy in a custom cybersecurity scenario. This includes the configuration of the network, the scripted or trained agents that interact with the network, as well as settings that define how to perform training in Stable Baselines 3 or Ray RLLib.
The entire config is used by the ``PrimaiteSession`` object for users who wish to let PrimAITE handle the agent definition and training. If you wish to define custom agents and control the training loop yourself, you can use the config with the ``PrimaiteGame``, and ``PrimaiteGymEnv`` objects instead. That way, only the network configuration and agent setup parts of the config are used, and the training section is ignored.

View File

@@ -49,6 +49,8 @@ In order to recreate the network below, we will need to create 2 links:
- a link from computer_2 to the switch
.. image:: ../../_static/switched_p2p_network.png
:width: 500
:align: center
this results in:

View File

@@ -106,9 +106,13 @@ There are 6 ACLs that can be defined for a firewall
- ``external_outbound_acl`` for traffic going towards the external network
.. image:: ../../../../_static/firewall_acl.png
:width: 500
:align: center
By default, ``external_inbound_acl`` and ``external_outbound_acl`` will permit any traffic through.
``internal_inbound_acl``, ``internal_outbound_acl``, ``dmz_inbound_acl`` and ``dmz_outbound_acl`` will deny any traffic by default, so must be configured to allow defined ``src_port`` and ``dst_port`` or ``protocol``.
See :py:mod:`primaite.simulator.network.hardware.nodes.network.router.AccessControlList`
See :ref:`List of Ports <List of Ports>` for a list of ports.

View File

@@ -5,27 +5,21 @@
``applications``
----------------
List of available applications that can be installed on a |NODE|:
List of available applications that can be installed on a |NODE| can be found in :ref:`List of Applications <List of Applications>`
.. toctree::
:maxdepth: 1
application in configuration
""""""""""""""""""""""""""""
../software/applications/data_manipulation_bot.rst
../software/applications/database_client.rst
../software/applications/dos_bot.rst
../software/applications/web_browser.rst
Applications takes a list of applications as shown in the example below.
More info :py:mod:`primaite.game.game.APPLICATION_TYPES_MAPPING`
.. code-block:: yaml
.. include:: ../software/common/system_software.rst
.. toctree::
:maxdepth: 1
../software/applications/web_browser.rst
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.SYSTEM_SOFTWARE`
.. |SOFTWARE_TYPE| replace:: application
.. |SOFTWARE_TYPES| replace:: applications
- ref: client_1
hostname: client_1
type: computer
...
applications:
- ref: example_application
type: example_application_type
options:
# this section is different for each application

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DataManipulationBot``
-----------------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DatabaseClient``
------------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``WebBrowser``
--------------
test

View File

@@ -1,12 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``system software``
"""""""""""""""""""
Some |SOFTWARE_TYPES| are pre installed on nodes - this is similar to how some |SOFTWARE_TYPES| are included with the Operating System.
The |SOFTWARE_TYPE| may not be configured as needed, in which case, follow the steps above to configure them.
The list of |SOFTWARE_TYPES| that are considered system software are:

View File

@@ -5,33 +5,21 @@
``services``
------------
List of available services that can be installed on a |NODE|:
List of available services that can be installed on a |NODE| can be found in :ref:`List of Services <List of Services>`
.. toctree::
:maxdepth: 1
services in configuration
"""""""""""""""""""""""""
../software/services/database_service.rst
../software/services/dns_client.rst
../software/services/dns_server.rst
../software/services/ftp_client.rst
../software/services/ftp_server.rst
../software/services/ntp_client.rst
../software/services/ntp_server.rst
../software/services/web_server.rst
Services takes a list of services as shown in the example below.
More info :py:mod:`primaite.game.game.SERVICE_TYPES_MAPPING`
.. code-block:: yaml
.. include:: ../software/common/system_software.rst
.. toctree::
:maxdepth: 1
../software/services/dns_client.rst
../software/services/ftp_client.rst
../software/services/ntp_client.rst
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.SYSTEM_SOFTWARE`
.. |SOFTWARE_TYPE| replace:: service
.. |SOFTWARE_TYPES| replace:: services
- ref: client_1
hostname: client_1
type: computer
...
applications:
- ref: example_service
type: example_service_type
options:
# this section is different for each service

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DatabaseService``
-------------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DNSClient``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DNSServer``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``FTPClient``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``FTPServer``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``NTPClient``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``NTPServer``
-------------
test

View File

@@ -1,8 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``WebServer``
-------------
test

View File

@@ -18,7 +18,7 @@ Game layer
The game layer is responsible for managing agents and getting them to interface with the simulator correctly. It consists of several components:
PrimAITE Session
^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^
``PrimaiteSession`` is the main entry point into Primaite and it allows the simultaneous coordination of a simulation and agents that interact with it. ``PrimaiteSession`` keeps track of multiple agents of different types.

View File

@@ -13,6 +13,8 @@ facilitates modular development, enhances maintainability, and supports scalabil
allowing for focused enhancements within each layer.
.. image:: primaite_network_interface_model.png
:width: 500
:align: center
Layer Descriptions
==================

View File

@@ -229,7 +229,7 @@ To limit database server access to selected external IP addresses:
position=7
)
**Permitting DMZ Web Server Access while Blocking Specific Threats*
**Permitting DMZ Web Server Access while Blocking Specific Threats**
To authorize HTTP/HTTPS access to a DMZ-hosted web server, excluding known malicious IPs:

View File

@@ -27,7 +27,7 @@ in the transmission and routing of data within the simulated environment.
**Key Features:**
- **Frame Processing:** Central to the class is the ability to receive and process network frames, facilitating the
simulation of data flow through network devices.
simulation of data flow through network devices.
- **Abstract Methods:** Includes abstract methods such as ``receive_frame``, which subclasses must implement to specify
how devices handle incoming traffic.

View File

@@ -16,6 +16,7 @@ The bot is intended to simulate a malicious actor carrying out attacks like:
- Dropping tables
- Deleting records
- Modifying data
on a database server by abusing an application's trusted database connectivity.
The bot performs attacks in the following stages to simulate the real pattern of an attack:

View File

@@ -0,0 +1,38 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
DatabaseClient
===============
The DatabaseClient provides a client interface for connecting to the ``DatabaseService``.
Key features
^^^^^^^^^^^^
- Connects to the ``DatabaseService`` via the ``SoftwareManager``.
- Handles connecting and disconnecting.
- Executes SQL queries and retrieves result sets.
Usage
^^^^^
- Initialise with server IP address and optional password.
- Connect to the ``DatabaseService`` with ``connect``.
- Retrieve results in a dictionary.
- Disconnect when finished.
To create database backups:
- Configure the backup server on the ``DatabaseService`` by providing the Backup server ``IPv4Address`` with ``configure_backup``
- Create a backup using ``backup_database``. This fails if the backup server is not configured.
- Restore a backup using ``restore_backup``. By default, this uses the database created via ``backup_database``.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Connect and disconnect methods manage sessions.
- Payloads serialised as dictionaries for transmission.
- Extends base Application class.

View File

@@ -2,7 +2,7 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``DoSBot``
----------
DoSBot
------
test

View File

@@ -2,35 +2,9 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Web Browser and Web Server Service
==================================
Web Server Service
------------------
Provides a Web Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates a web server with the capability to also request data from a database
- Allows the emulation of HTTP requests between client (e.g. a web browser) and server
- GET request sends a get all users request to the database server and returns an HTTP 200 status if the database is responsive
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
Implementation
^^^^^^^^^^^^^^
- HTTP request uses a ``HttpRequestPacket`` object
- HTTP response uses a ``HttpResponsePacket`` object
- Extends Service class for integration with ``SoftwareManager``.
Web Browser (Web Client)
------------------------
WebBrowser
==========
The ``WebBrowser`` provides a client interface for connecting to the ``WebServer``.

View File

@@ -1,71 +0,0 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
Database Client Server
======================
Database Service
----------------
The ``DatabaseService`` provides a SQL database server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Creates a database file in the ``Node`` 's ``FileSystem`` upon creation.
- Handles connecting clients by maintaining a dictionary of connections mapped to session IDs.
- Authenticates connections using a configurable password.
- Simulates ``SELECT``, ``DELETE`` and ``INSERT`` SQL queries.
- Returns query results and status codes back to clients.
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Clients connect, execute queries, and disconnect.
- Service runs on TCP port 5432 by default.
Implementation
^^^^^^^^^^^^^^
- Creates the database file within the node's file system.
- Manages client connections in a dictionary by session ID.
- Processes SQL queries.
- Returns results and status codes in a standard dictionary format.
- Extends Service class for integration with ``SoftwareManager``.
Database Client
---------------
The DatabaseClient provides a client interface for connecting to the ``DatabaseService``.
Key features
^^^^^^^^^^^^
- Connects to the ``DatabaseService`` via the ``SoftwareManager``.
- Handles connecting and disconnecting.
- Executes SQL queries and retrieves result sets.
Usage
^^^^^
- Initialise with server IP address and optional password.
- Connect to the ``DatabaseService`` with ``connect``.
- Retrieve results in a dictionary.
- Disconnect when finished.
To create database backups:
- Configure the backup server on the ``DatabaseService`` by providing the Backup server ``IPv4Address`` with ``configure_backup``
- Create a backup using ``backup_database``. This fails if the backup server is not configured.
- Restore a backup using ``restore_backup``. By default, this uses the database created via ``backup_database``.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Connect and disconnect methods manage sessions.
- Payloads serialised as dictionaries for transmission.
- Extends base Application class.

View File

@@ -0,0 +1,11 @@
.. toctree::
:maxdepth: 1
applications/data_manipulation_bot.rst
applications/database_client.rst
applications/dos_bot.rst
applications/web_browser.rst
More info :py:mod:`primaite.game.game.APPLICATION_TYPES_MAPPING`
.. include:: list_of_system_applications.rst

View File

@@ -0,0 +1,15 @@
.. toctree::
:maxdepth: 1
services/database_service.rst
services/dns_client.rst
services/dns_server.rst
services/ftp_client.rst
services/ftp_server.rst
services/ntp_client.rst
services/ntp_server.rst
services/web_server.rst
More info :py:mod:`primaite.game.game.SERVICE_TYPES_MAPPING`
.. include:: list_of_system_services.rst

View File

@@ -0,0 +1,19 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``system applications``
"""""""""""""""""""""""
Some applications are pre installed on nodes - this is similar to how some applications are included with the Operating System.
The application may not be configured as needed, in which case, see the relevant application page.
The list of applications that are considered system software are:
.. toctree::
:maxdepth: 1
applications/web_browser.rst
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.SYSTEM_SOFTWARE`

View File

@@ -0,0 +1,21 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
``system services``
"""""""""""""""""""
Some services are pre installed on nodes - this is similar to how some services are included with the Operating System.
The service may not be configured as needed, in which case, see the relevant service page.
The list of services that are considered system software are:
.. toctree::
:maxdepth: 1
services/dns_client.rst
services/ftp_client.rst
services/ntp_client.rst
More info :py:mod:`primaite.simulator.network.hardware.nodes.host.host_node.SYSTEM_SOFTWARE`

View File

@@ -0,0 +1,33 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
DatabaseService
===============
The ``DatabaseService`` provides a SQL database server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Creates a database file in the ``Node`` 's ``FileSystem`` upon creation.
- Handles connecting clients by maintaining a dictionary of connections mapped to session IDs.
- Authenticates connections using a configurable password.
- Simulates ``SELECT``, ``DELETE`` and ``INSERT`` SQL queries.
- Returns query results and status codes back to clients.
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Clients connect, execute queries, and disconnect.
- Service runs on TCP port 5432 by default.
Implementation
^^^^^^^^^^^^^^
- Creates the database file within the node's file system.
- Manages client connections in a dictionary by session ID.
- Processes SQL queries.
- Returns results and status codes in a standard dictionary format.
- Extends Service class for integration with ``SoftwareManager``.

View File

@@ -2,34 +2,8 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
DNS Client Server
=================
DNS Server
----------
Also known as a DNS Resolver, the ``DNSServer`` provides a DNS Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates DNS requests and DNSPacket transfer across a network
- Registers domain names and the IP Address linked to the domain name
- Returns the IP address for a given domain name within a DNS Packet that a DNS Client can read
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
Implementation
^^^^^^^^^^^^^^
- DNS request and responses use a ``DNSPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
DNS Client
----------
DNSClient
=========
The DNSClient provides a client interface for connecting to the ``DNSServer``.

View File

@@ -0,0 +1,26 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
DNSServer
=========
Also known as a DNS Resolver, the ``DNSServer`` provides a DNS Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates DNS requests and DNSPacket transfer across a network
- Registers domain names and the IP Address linked to the domain name
- Returns the IP address for a given domain name within a DNS Packet that a DNS Client can read
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on TCP port 53 by default. (TODO: TCP for now, should be UDP in future)
Implementation
^^^^^^^^^^^^^^
- DNS request and responses use a ``DNSPacket`` object
- Extends Service class for integration with ``SoftwareManager``.

View File

@@ -2,35 +2,9 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
FTP Client Server
=================
FTP Server
----------
Provides a FTP Client-Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the FTP server service.
- Service runs on FTP (command) port 21 by default. (TODO: look at in depth implementation of FTP PORT command)
Implementation
^^^^^^^^^^^^^^
- FTP request and responses use a ``FTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
FTP Client
----------
FTPClient
=========
The ``FTPClient`` provides a client interface for connecting to the ``FTPServer``.

View File

@@ -0,0 +1,27 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
FTPServer
=========
Provides a FTP Client-Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates FTP requests and FTPPacket transfer across a network
- Allows the emulation of FTP commands between an FTP client and server:
- STOR: stores a file from client to server
- RETR: retrieves a file from the FTP server
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the FTP server service.
- Service runs on FTP (command) port 21 by default. (TODO: look at in depth implementation of FTP PORT command)
Implementation
^^^^^^^^^^^^^^
- FTP request and responses use a ``FTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.

View File

@@ -0,0 +1,26 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
NTPClient
=========
The NTPClient provides a client interface for connecting to the ``NTPServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``NTPServer`` via the ``SoftwareManager``.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.

View File

@@ -2,11 +2,8 @@
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
NTP Client Server
=================
NTP Server
----------
NTPServer
=========
The ``NTPServer`` provides a NTP Server simulation by extending the base Service class.
NTP Client
@@ -29,26 +26,3 @@ Implementation
- NTP request and responses use a ``NTPPacket`` object
- Extends Service class for integration with ``SoftwareManager``.
NTP Client
----------
The NTPClient provides a client interface for connecting to the ``NTPServer``.
Key features
^^^^^^^^^^^^
- Connects to the ``NTPServer`` via the ``SoftwareManager``.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the database service.
- Service runs on UDP port 123 by default.
Implementation
^^^^^^^^^^^^^^
- Leverages ``SoftwareManager`` for sending payloads over the network.
- Provides easy interface for Nodes to find IP addresses via domain names.
- Extends base Service class.

View File

@@ -0,0 +1,27 @@
.. only:: comment
© Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
WebServer
=========
Provides a Web Server simulation by extending the base Service class.
Key capabilities
^^^^^^^^^^^^^^^^
- Simulates a web server with the capability to also request data from a database
- Allows the emulation of HTTP requests between client (e.g. a web browser) and server
- GET request sends a get all users request to the database server and returns an HTTP 200 status if the database is responsive
- Leverages the Service base class for install/uninstall, status tracking, etc.
Usage
^^^^^
- Install on a Node via the ``SoftwareManager`` to start the `WebServer`.
- Service runs on HTTP port 80 by default. (TODO: HTTPS)
Implementation
^^^^^^^^^^^^^^
- HTTP request uses a ``HttpRequestPacket`` object
- HTTP response uses a ``HttpResponsePacket`` object
- Extends Service class for integration with ``SoftwareManager``.

View File

@@ -16,6 +16,8 @@ ARP, ICMP, or the Web Client. This pathway exemplifies the structured processing
each frame reaches its intended target within the simulated environment.
.. image:: node_session_software_model_example.png
:width: 500
:align: center
Session Manager
---------------

View File

@@ -39,16 +39,27 @@ See :ref:`Node Start up and Shut down`
assert node.operating_state is NodeOperatingState.ON
assert web_server.operating_state is ServiceOperatingState.RUNNING # service turned back on when node is powered on
.. _List of Applications:
Services, Processes and Applications:
#####################################
Applications
############
.. toctree::
:maxdepth: 2
These are a list of applications that are currently available in PrimAITE:
database_client_server
data_manipulation_bot
dns_client_server
ftp_client_server
ntp_client_server
web_browser_and_web_server_service
.. include:: list_of_applications.rst
.. _List of Services:
Services
########
These are a list of services that are currently available in PrimAITE:
.. include:: list_of_services.rst
.. _List of Processes:
Processes
#########
`To be implemented`

View File

@@ -12,14 +12,15 @@ and a domain controller for managing software and users.
Each node of the simulation 'tree' has responsibility for creating, deleting, and updating its direct descendants. Also,
when a component's ``describe_state()`` method is called, it will include the state of its descendants. The
``apply_request()`` method can be used to act on a component or one of its descendatnts. The diagram below shows the
``apply_request()`` method can be used to act on a component or one of its descendants. The diagram below shows the
relationship between components.
.. image:: _static/component_relationship.png
.. image:: ../../_static/component_relationship.png
:width: 500
:alt: The top level simulation object owns a NetworkContainer and a DomainController. The DomainController has a
list of accounts. The network container has links and nodes. Nodes can own switchports, NICs, FileSystem,
Application, Service, and Process.
:align: center
:alt: :: The top level simulation object owns a NetworkContainer and a DomainController. The DomainController has a
list of accounts. The network container has links and nodes. Nodes can own switchports, NICs, FileSystem,
Application, Service, and Process.
Actions