diff --git a/docs/index.rst b/docs/index.rst index 29680d08..a59533c7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -39,105 +39,3 @@ PrimAITE incorporates the following features: - Agent action logs provide a description of every action taken by each agent during the episode. This includes timestep, action, parameters, request and response, for all Blue agent activity, which is aligned with the Track 2 Common Action / Observation Space (CAOS) format. Action logs also details of all scripted / stochastic red / green agent actions; - Environment ground truth is provided at every timestep, providing a full description of the environment’s true state; - Alignment with CAOS provides the ability to transfer agents between CAOS compliant environments. - -Architecture -^^^^^^^^^^^^ - -PrimAITE is a Python application and will operate on multiple Operating Systems (Windows, Linux and Mac); -a comprehensive installation and user guide is provided with each release to support its usage. - -Configuration of PrimAITE is achieved via included YAML files which support full control over the network / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count. -A Simulation Controller layer manages the overall running of the simulation, keeping track of all low-level objects. - -It is agnostic to the number of agents, their action / observation spaces, and the RL library being used. - -It presents a public API providing a method for describing the current state of the simulation, a method that accepts action requests and provides responses, and a method that triggers a timestep advancement. -The Game Layer converts the simulation into a playable game for the agent(s). - -It translates between simulation state and Gymnasium.Spaces to pass action / observation data between the agent(s) and the simulation. It is responsible for calculating rewards, managing Multi-Agent RL (MARL) action turns, and via a single agent interface can interact with Blue, Red and Green agents. - -Agents can either generate their own scripted behaviour or accept input behaviour from an RL agent. - -Finally, a Gymnasium / Ray RLlib Environment Layer forwards requests to the Game Layer as the agent sends them. This layer also manages most of the I/O, such as reading in the configuration files and saving agent logs. - -.. image:: ../../_static/primAITE_architecture.png - :width: 500 - :align: center - - -Training & Evaluation Capability -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its Gymnasium / Ray RLlib compliant interface. - -Scenarios can be constructed to reflect network / system laydowns consisting of any configuration of nodes (e.g., PCs, servers etc.) and the networking equipment and links between them. - -All nodes can be configured to contain applications, services, folders and files (and their status). - -Traffic flows between services and applications as directed by an ‘execution definition,’ with the traffic flow on the network governed by the network equipment (switches, routers and firewalls) and the ACL rules and routing tables they employ. - -Highlights of PrimAITE’s training and evaluation capability are: - -- The scenario is not bound to a representation of any platform, system, or technology; -- Fully configurable (network / system laydown, green pattern-of-life, red personas, reward function, ACL rules for each device, number of episodes / steps, action / observation space) and repeatable to suit the requirements of AI agents; -- Can integrate with any Gymnasium / Ray RLlib compliant AI agent . - - -PrimAITE provides a number of use cases (network and red/green action configurations) by default which the user is able to extend and modify as required. - -What is PrimAITE built with ---------------------------- - -* `Gymnasium `_ is used as the basis for AI blue agent interaction with the PrimAITE environment -* `Networkx `_ is used as the underlying data structure used for the PrimAITE environment -* `Stable Baselines 3 `_ is used as a default source of RL algorithms (although PrimAITE is not limited to SB3 agents) -* `Ray RLlib `_ is used as an additional source of RL algorithms -* `Typer `_ is used for building CLIs (Command Line Interface applications) -* `Jupyterlab `_ is used as an extensible environment for interactive and reproducible computing, based on the Jupyter Notebook Architecture -* `Platformdirs `_ is used for finding the right location to store user data and configuration but varies per platform -* `Plotly `_ is used for building high level charts - - -Getting Started with PrimAITE ------------------------------ - -Head over to the :ref:`getting-started` page to install and setup PrimAITE! - -.. toctree:: - :maxdepth: 8 - :caption: About PrimAITE: - :hidden: - - source/about - source/dependencies - source/glossary - -.. toctree:: - :caption: Usage: - :hidden: - - source/getting_started - source/game_layer - source/simulation - source/config - source/customising_scenarios - source/varying_config_files - source/environment - source/action_masking - -.. toctree:: - :caption: Notebooks: - :hidden: - - source/example_notebooks - source/notebooks/executed_notebooks - -.. toctree:: - :caption: Developer information: - :hidden: - - source/developer_tools - source/state_system - source/request_system - PrimAITE API - PrimAITE Tests diff --git a/docs/source/about.rst b/docs/source/about.rst index 488154a0..06eff34e 100644 --- a/docs/source/about.rst +++ b/docs/source/about.rst @@ -7,39 +7,10 @@ About PrimAITE ============== -The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effective simulation capability for training and evaluating AI in a cyber-defensive role. It incorporates the functionality required of a primary-level ARCD environment: - -- The ability to model a relevant system context; -- Modelling an adversarial agent that the defensive agent can be trained and evaluated against; -- The ability to model key characteristics of a system by representing hosts, servers, network devices, IP addresses, ports, operating systems, folders / files, applications, services and links; -- Modelling background (green) pattern-of-life; -- Operates at machine-speed to enable fast training cycles via Reinforcement Learning (RL). - -Features -^^^^^^^^ - -PrimAITE incorporates the following features: - -- Architected with a separate Simulation layer and Game layer. This separation of concerns defines a clear path towards transfer learning with environments of differing fidelity; -- Ability to reconfigure an RL reward function based on (a) the ability to counter the modelled adversarial cyber-attack, and (b) the ability to ensure success for green agents; -- Access Control List (ACL) functions for network devices (routers and firewalls), following standard ACL rule format (e.g., DENY / ALLOW, source / destination IP addresses, protocol and port); -- Application of traffic to the links of the system laydown adheres to the ACL rulesets and routing tables contained within each network device; -- Provides RL environments adherent to the Farama Foundation Gymnasium (Previously OpenAI Gym) API, allowing integration with any compliant RL Agent frameworks; -- Provides RL environments adherent to Ray RLlib environment specifications for single-agent and multi-agent scenarios; -- Assessed for compatibility with Stable-Baselines3 (SB3), Ray RLlib, and bespoke agents; -- Persona-based adversarial (Red) agent behaviour; several out-the-box personas are provided, and more can be developed to suit the needs of the task. Stochastic variations in Red agent behaviour are also included as required; -- A robust system logging tool, automatically enabled at the node level and featuring various log levels and terminal output options, enables PrimAITE users to conduct in-depth network simulations; -- A PCAP service is seamlessly integrated within the simulation, automatically capturing and logging frames for both - inbound and outbound traffic at the network interface level. This automatic functionality, combined with the ability - to separate traffic directions, significantly enhances network analysis and troubleshooting capabilities; -- Agent action logs provide a description of every action taken by each agent during the episode. This includes timestep, action, parameters, request and response, for all Blue agent activity, which is aligned with the Track 2 Common Action / Observation Space (CAOS) format. Action logs also details of all scripted / stochastic red / green agent actions; -- Environment ground truth is provided at every timestep, providing a full description of the environment’s true state; -- Alignment with CAOS provides the ability to transfer agents between CAOS compliant environments. - Architecture ^^^^^^^^^^^^ -PrimAITE is a Python application and will operate on multiple Operating Systems (Windows, Linux and macOS); +PrimAITE is a Python application and will operate on multiple Operating Systems (Windows, Linux and Mac); a comprehensive installation and user guide is provided with each release to support its usage. Configuration of PrimAITE is achieved via included YAML files which support full control over the network / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count. @@ -66,17 +37,17 @@ Training & Evaluation Capability PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its Gymnasium / Ray RLlib compliant interface. -Scenarios can be constructed to reflect network / system laydowns consisting of any configuration of nodes (e.g. PCs, servers etc.) and the networking equipment and links between them. +Scenarios can be constructed to reflect network / system laydowns consisting of any configuration of nodes (e.g., PCs, servers etc.) and the networking equipment and links between them. All nodes can be configured to contain applications, services, folders and files (and their status). -Traffic flows between services and applications as directed by an ‘execution definition’ with the traffic flow on the network governed by the network equipment (switches, routers and firewalls) and the ACL rules and routing tables they employ. +Traffic flows between services and applications as directed by an ‘execution definition,’ with the traffic flow on the network governed by the network equipment (switches, routers and firewalls) and the ACL rules and routing tables they employ. Highlights of PrimAITE’s training and evaluation capability are: - The scenario is not bound to a representation of any platform, system, or technology; - Fully configurable (network / system laydown, green pattern-of-life, red personas, reward function, ACL rules for each device, number of episodes / steps, action / observation space) and repeatable to suit the requirements of AI agents; -- Can integrate with any Gymnasium / Ray RLlib compliant AI agent. +- Can integrate with any Gymnasium / Ray RLlib compliant AI agent . PrimAITE provides a number of use cases (network and red/green action configurations) by default which the user is able to extend and modify as required. @@ -99,6 +70,46 @@ Getting Started with PrimAITE Head over to the :ref:`getting-started` page to install and setup PrimAITE! +.. toctree:: + :maxdepth: 8 + :caption: About PrimAITE: + :hidden: + + source/about + source/dependencies + source/glossary + +.. toctree:: + :caption: Usage: + :hidden: + + source/getting_started + source/game_layer + source/simulation + source/config + source/customising_scenarios + source/varying_config_files + source/environment + source/action_masking + +.. toctree:: + :caption: Notebooks: + :hidden: + + source/example_notebooks + source/notebooks/executed_notebooks + +.. toctree:: + :caption: Developer information: + :hidden: + + source/developer_tools + source/state_system + source/request_system + PrimAITE API + PrimAITE Tests + + .. Architecture - Nodes and Links ****************************** diff --git a/docs/source/example_notebooks.rst b/docs/source/example_notebooks.rst index 26c1bb1f..920175c9 100644 --- a/docs/source/example_notebooks.rst +++ b/docs/source/example_notebooks.rst @@ -85,6 +85,6 @@ The following extensions should now be installed :width: 300 :align: center -VSCode will then ask for a Python environment version to use. PrimAITE is compatible with Python versions 3.9 - 3.11 +VSCode will then ask for a Python environment version to use. PrimAITE is compatible with Python versions 3.8 - 3.11 You should now be able to interact with the notebook. diff --git a/docs/source/primaite-dependencies.rst b/docs/source/primaite-dependencies.rst index e7333b53..04987054 100644 --- a/docs/source/primaite-dependencies.rst +++ b/docs/source/primaite-dependencies.rst @@ -2,40 +2,44 @@ © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| Name | Version | License | Description | URL | -+===================+=========+====================================+=======================================================================================================+==============================================+ -| gymnasium | 0.28.1 | MIT License | A standard API for reinforcement learning and a diverse set of reference environments (formerly Gym). | https://farama.org | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| ipywidgets | 8.1.3 | BSD License | Jupyter interactive widgets | http://jupyter.org | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| jupyterlab | 3.6.1 | BSD License | JupyterLab computational environment | https://jupyter.org | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| kaleido | 0.2.1 | MIT | Static image export for web-based visualization libraries with zero dependencies | https://github.com/plotly/Kaleido | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| matplotlib | 3.7.1 | Python Software Foundation License | Python plotting package | https://matplotlib.org | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| networkx | 3.1 | BSD License | Python package for creating and manipulating graphs and networks | https://networkx.org/ | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| numpy | 1.23.5 | BSD License | NumPy is the fundamental package for array computing with Python. | https://www.numpy.org | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| platformdirs | 3.5.1 | MIT License | A small Python package for determining appropriate platform-specific dirs, e.g. a "user data dir". | https://github.com/platformdirs/platformdirs | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| plotly | 5.15.0 | MIT License | An open-source, interactive data visualization library for Python | https://plotly.com/python/ | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| polars | 0.18.4 | MIT License | Blazingly fast DataFrame library | https://www.pola.rs/ | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| prettytable | 3.8.0 | BSD License (BSD (3 clause)) | A simple Python library for easily displaying tabular data in a visually appealing ASCII table format | https://github.com/jazzband/prettytable | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| pydantic | 2.7.0 | MIT License | Data validation using Python type hints | https://github.com/pydantic/pydantic | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| PyYAML | 6.0 | MIT License | YAML parser and emitter for Python | https://pyyaml.org/ | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| ray | 2.23.0 | Apache 2.0 | Ray provides a simple, universal API for building distributed applications. | https://github.com/ray-project/ray | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| stable-baselines3 | 2.1.0 | MIT | Pytorch version of Stable Baselines, implementations of reinforcement learning algorithms. | https://github.com/DLR-RM/stable-baselines3 | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| tensorflow | 2.12.0 | Apache Software License | TensorFlow is an open source machine learning framework for everyone. | https://www.tensorflow.org/ | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ -| typer | 0.9.0 | MIT License | Typer, build great CLIs. Easy to code. Based on Python type hints. | https://github.com/tiangolo/typer | -+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+ ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| Name | Version | License | Description | URL | ++===================+=========+====================================+=======================================================================================================+====================================================================+ +| gymnasium | 0.28.1 | MIT License | A standard API for reinforcement learning and a diverse set of reference environments (formerly Gym). | https://farama.org | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| ipywidgets | 8.1.3 | BSD License | Jupyter interactive widgets | http://jupyter.org | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| jupyterlab | 3.6.1 | BSD License | JupyterLab computational environment | https://jupyter.org | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| kaleido | 0.2.1 | MIT | Static image export for web-based visualization libraries with zero dependencies | https://github.com/plotly/Kaleido | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| matplotlib | 3.7.1 | Python Software Foundation License | Python plotting package | https://matplotlib.org | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| networkx | 3.1 | BSD License | Python package for creating and manipulating graphs and networks | https://networkx.org/ | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| numpy | 1.23.5 | BSD License | NumPy is the fundamental package for array computing with Python. | https://www.numpy.org | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| platformdirs | 3.5.1 | MIT License | A small Python package for determining appropriate platform-specific dirs, e.g. a "user data dir". | https://github.com/platformdirs/platformdirs | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| plotly | 5.15.0 | MIT License | An open-source, interactive data visualization library for Python | https://plotly.com/python/ | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| polars | 0.18.4 | MIT License | Blazingly fast DataFrame library | https://www.pola.rs/ | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| prettytable | 3.8.0 | BSD License (BSD (3 clause)) | A simple Python library for easily displaying tabular data in a visually appealing ASCII table format | https://github.com/jazzband/prettytable | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| pydantic | 2.7.0 | MIT License | Data validation using Python type hints | https://github.com/pydantic/pydantic | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| PyYAML | 6.0 | MIT License | YAML parser and emitter for Python | https://pyyaml.org/ | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| ray | 2.23.0 | Apache 2.0 | Ray provides a simple, universal API for building distributed applications. | https://github.com/ray-project/ray | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| stable-baselines3 | 2.1.0 | MIT | Pytorch version of Stable Baselines, implementations of reinforcement learning algorithms. | https://github.com/DLR-RM/stable-baselines3 | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| tensorflow | 2.12.0 | Apache Software License | TensorFlow is an open source machine learning framework for everyone. | https://www.tensorflow.org/ | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| typer | 0.9.0 | MIT License | Typer, build great CLIs. Easy to code. Based on Python type hints. | https://github.com/tiangolo/typer | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| Deepdiff | 7.0.1 | MIT License | Deep difference of dictionaries, iterables, strings, and any other object objects. | https://github.com/seperman/deepdiff | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ +| sb3_contrib | 2.3.0 | MIT License | Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code (Action Masking)| https://github.com/Stable-Baselines-Team/stable-baselines3-contrib | ++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+ diff --git a/docs/source/simulation_components/system/applications/nmap.rst b/docs/source/simulation_components/system/applications/nmap.rst index 9ea0c60e..abfa439d 100644 --- a/docs/source/simulation_components/system/applications/nmap.rst +++ b/docs/source/simulation_components/system/applications/nmap.rst @@ -345,3 +345,11 @@ Perform a full box scan on all ports, over both TCP and UDP, on a whole subnet: | 192.168.1.13 | 123 | NTP | UDP | | 192.168.1.13 | 219 | ARP | UDP | +--------------+------+-----------------+----------+ + +Configuration +============= + +.. include:: ../common/common_configuration.rst + +.. |SOFTWARE_NAME| replace:: NMAP +.. |SOFTWARE_NAME_BACKTICK| replace:: ``NMAP`` diff --git a/docs/source/simulation_components/system/services/ftp_client.rst b/docs/source/simulation_components/system/services/ftp_client.rst index 889a2c36..fdf9cfcf 100644 --- a/docs/source/simulation_components/system/services/ftp_client.rst +++ b/docs/source/simulation_components/system/services/ftp_client.rst @@ -87,5 +87,3 @@ Configuration .. |SOFTWARE_NAME| replace:: FTPClient .. |SOFTWARE_NAME_BACKTICK| replace:: ``FTPClient`` - -**FTPClient has no configuration options** diff --git a/docs/source/simulation_components/system/services/ntp_server.rst b/docs/source/simulation_components/system/services/ntp_server.rst index 0369cd15..3ddb51ea 100644 --- a/docs/source/simulation_components/system/services/ntp_server.rst +++ b/docs/source/simulation_components/system/services/ntp_server.rst @@ -82,5 +82,3 @@ Configuration .. |SOFTWARE_NAME| replace:: NTPServer .. |SOFTWARE_NAME_BACKTICK| replace:: ``NTPServer`` - -**NTPServer has no configuration options** diff --git a/docs/source/simulation_components/system/services/web_server.rst b/docs/source/simulation_components/system/services/web_server.rst index 5b205f5e..f0294223 100644 --- a/docs/source/simulation_components/system/services/web_server.rst +++ b/docs/source/simulation_components/system/services/web_server.rst @@ -82,5 +82,3 @@ Configuration .. |SOFTWARE_NAME| replace:: WebServer .. |SOFTWARE_NAME_BACKTICK| replace:: ``WebServer`` - -**WebServer has no configuration options**