Merge branch 'github_dev' into downstream_github_sync

# Conflicts: # docs/index.rst # docs/source/primaite_session.rst
2023-08-03 16:07:15 +01:00
parent 3660e27c15 fed65db7fc
commit e7e39a8c1d
6 changed files with 162 additions and 111 deletions
--- a/README.md
+++ b/README.md
@@ -6,7 +6,7 @@ The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effect

 - The ability to model a relevant platform / system context;

- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, services and processes;
+- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems and services;

 - Operates at machine-speed to enable fast training cycles.

@@ -24,7 +24,7 @@ PrimAITE presents the following features:

 - Application of IERs to the platform / system laydown adheres to the ACL ruleset;

- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any OpenAI gym compliant defensive agents;
+- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any compliant defensive agents;

 - Full capture of discrete logs relating to agent training (full system state, agent actions taken, instantaneous and average reward for every step of every episode);

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -43,7 +43,6 @@ extensions = [
    "sphinx.ext.viewcode",  # Add a link to the Python source code for classes, functions etc.
    "sphinx.ext.todo",
    "sphinx_copybutton",  # Adds a copy button to code blocks
-    "sphinx_code_tabs",  # Enables tabbed code blocks
 ]


--- a/docs/index.rst
+++ b/docs/index.rst
@@ -8,11 +8,68 @@ Welcome to PrimAITE's documentation
 What is PrimAITE?
 -----------------

-PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. It incorporates the functionality required of a Primary-level environment, as specified in the Dstl ARCD Training Environment Matrix document:
+Overview
+^^^^^^^^
+
+The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effective simulation capability for the purposes of training and evaluating AI in a cyber-defensive role. It incorporates the functionality required of a primary-level ARCD environment, which includes:
+
+- The ability to model a relevant platform / system context;
+- Modelling an adversarial agent that the defensive agent can be trained and evaluated against;
+- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, operating systems, services and traffic loading on links;
+- Modelling background pattern-of-life;
+- Operates at machine-speed to enable fast training cycles.
+
+Features
+^^^^^^^^
+
+PrimAITE incorporates the following features:
+
+- Highly configurable (via YAML files) to provide the means to model a variety of platform / system laydowns and adversarial attack scenarios;
+- A Reinforcement Learning (RL) reward function based on (a) the ability to counter the modelled adversarial cyber-attack, and (b) the ability to ensure success;
+- Provision of logging to support AI performance / effectiveness assessment;
+- Uses the concept of Information Exchange Requirements (IERs) to model background pattern of life and adversarial behaviour;
+- An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP address, destination IP address, protocol and port);
+- Application of traffic to the links of the platform / system laydown adheres to the ACL ruleset;
+- Presents both an OpenAI gym and Ray RLLib interface to the environment, allowing integration with any compliant defensive agents;
+- Allows for the saving and loading of trained defensive agents;
+- Stochastic adversarial agent behaviour;
+- Full capture of discrete logs relating to agent training or evaluation (system state, agent actions taken, instantaneous and average reward for every step of every episode);
+- Distinct control over running a training and / or evaluation session;
+- NetworkX provides laydown visualisation capability.
+
+Architecture
+^^^^^^^^^^^^
+
+PrimAITE is a Python application and is therefore Operating System agnostic. The OpenAI gym and Ray RLLib frameworks are employed to provide an interface and source for AI agents. Configuration of PrimAITE is achieved via included YAML files which support full control over the platform / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count. NetworkX based nodes and links host Python classes to present attributes and methods, and hence a more representative platform / system can be modelled within the simulation.
+
+
+
+Training & Evaluation Capability
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its OpenAI Gym and RLLib compliant interface. Scenarios can be constructed to reflect platform / system laydowns consisting of any configuration of nodes (e.g. PCs, servers, switches etc.) and network links between them. All nodes can be configured to model services (and their status) and the traffic loading between them over the network links. Traffic loading is broken down into a per service granularity, relating directly to a protocol (e.g. Service A would be configured as a TCP service, and TCP traffic then flows between instances of Service A under the direction of a tailored IER). Highlights of PrimAITE’s training and evaluation capability are:
+
+- The scenario is not bound to a representation of any platform, system, or technology;
+- Fully configurable (network / system laydown, IERs, node pattern-of-life, ACL, number of episodes, steps per episode) and repeatable to suit the requirements of AI agents;
+- Can integrate with any OpenAI Gym or RLLib compliant AI agent.
+
+Use of PrimAITE default scenarios within ARCD is supported by a “Use Case Profile” tailored to the scenario.
+
+AI Assessment Capability
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+PrimAITE includes the capability to support in-depth assessment of cyber defence AI by outputting logs of the environment state and AI behaviour throughout both training and evaluation sessions. These logs include the following data:
+
+- Timestamp;
+- Episode and step number;
+- Agent identifier;
+- Observation space;
+- Action taken (by defensive AI);
+- Reward value.
+
+Logs are available in CSV format and provide coverage of the above data for every step of every episode.
+

-* The ability to model a relevant platform / system context;
-* The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes;
-* Operates at machine-speed to enable fast training cycles.


 What is PrimAITE built with
@@ -36,13 +93,13 @@ Head over to the :ref:`getting-started` page to install and setup PrimAITE!
 .. toctree::
   :maxdepth: 8
   :caption: Contents:
+   :hidden:

   source/getting_started
   source/about
   source/config
   source/primaite_session
   source/custom_agent
-   source/simulation
   PrimAITE API <source/_autosummary/primaite>
   PrimAITE Tests <source/_autosummary/tests>
   source/dependencies
--- a/docs/source/getting_started.rst
+++ b/docs/source/getting_started.rst
@@ -14,20 +14,19 @@ Pre-Requisites
 In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.10 installed. If you don't already have it, this is how to install it:


-.. tabs:: lang
+.. code-block:: bash
+    :caption: Unix

-    .. code-tab:: bash
-        :caption: Unix
+    sudo add-apt-repository ppa:deadsnakes/ppa
+    sudo apt install python3.10
+    sudo apt-get install python3-pip
+    sudo apt-get install python3-venv

-        sudo add-apt-repository ppa:deadsnakes/ppa
-        sudo apt install python3.10
-        sudo apt-get install python3-pip
-        sudo apt-get install python3-venv

-    .. code-tab:: text
-        :caption: Windows (Powershell)
+.. code-block:: text
+    :caption: Windows (Powershell)

-        - Manual install from: https://www.python.org/downloads/release/python-31011/
+    - Manual install from: https://www.python.org/downloads/release/python-31011/

 **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS.

@@ -36,30 +35,30 @@ Install PrimAITE

 1. Create a primaite directory in your home directory:

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix

-        mkdir ~/primaite/2.0.0
+.. code-block:: bash
+    :caption: Unix

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+    mkdir ~/primaite/2.0.0

-        mkdir ~\primaite\2.0.0
+.. code-block:: powershell
+    :caption: Windows (Powershell)
+
+    mkdir ~\primaite\2.0.0

 2. Navigate to the primaite directory and create a new python virtual environment (venv)

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix

-        cd ~/primaite/2.0.0
-        python3 -m venv .venv
+.. code-block:: bash
+    :caption: Unix

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+    cd ~/primaite/2.0.0
+    python3 -m venv .venv
+
+.. code-block:: powershell
+    :caption: Windows (Powershell)

        cd ~\primaite\2.0.0
        python3 -m venv .venv
@@ -67,44 +66,41 @@ Install PrimAITE

 3. Activate the venv

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix
+.. code-block:: bash
+    :caption: Unix

-        source .venv/bin/activate
+    source .venv/bin/activate

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+.. code-block:: powershell
+    :caption: Windows (Powershell)

-        .\.venv\Scripts\activate
+    .\.venv\Scripts\activate


 4. Install PrimAITE using pip from PyPi

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix
+.. code-block:: bash
+    :caption: Unix

-        pip install primaite
+    pip install primaite

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+.. code-block:: powershell
+    :caption: Windows (Powershell)

-        pip install primaite
+    pip install primaite

 5. Perform the PrimAITE setup

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix
+.. code-block:: bash
+    :caption: Unix

-        primaite setup
+    primaite setup

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+.. code-block:: powershell
+    :caption: Windows (Powershell)

        primaite setup

@@ -123,33 +119,31 @@ of your choice:

 Create and activate your Python virtual environment (venv)

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix
+.. code-block:: bash
+    :caption: Unix

-        python3 -m venv venv
-        source venv/bin/activate
+    python3 -m venv venv
+    source venv/bin/activate

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+.. code-block:: powershell
+    :caption: Windows (Powershell)

-        python3 -m venv venv
-        .\venv\Scripts\activate
+    python3 -m venv venv
+    .\venv\Scripts\activate

 Install PrimAITE with the dev extra

-.. tabs:: lang

-    .. code-tab:: bash
-        :caption: Unix
+.. code-block:: bash
+    :caption: Unix

-        pip install -e .[dev]
+    pip install -e .[dev]

-    .. code-tab:: powershell
-        :caption: Windows (Powershell)
+.. code-block:: powershell
+    :caption: Windows (Powershell)

-        pip install -e .[dev]
+    pip install -e .[dev]


 To view the complete list of packages installed during PrimAITE installation, go to the dependencies page (:ref:`Dependencies`).
--- a/docs/source/primaite_session.rst
+++ b/docs/source/primaite_session.rst
@@ -15,31 +15,31 @@ A PrimAITE session can be ran either with the ``primaite session`` command from
 Both the ``primaite session`` and :func:`primaite.main.run` take a training config and a lay down config as parameters.


-.. tabs::
-
-    .. code-tab:: bash
-        :caption: Unix CLI
-
-        cd ~/primaite/2.0.0
-        source ./.venv/bin/activate
-        primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml
-
-    .. code-tab:: powershell
-        :caption: Powershell CLI
-
-        cd ~\primaite\2.0.0
-        .\.venv\Scripts\activate
-        primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml


-    .. code-tab:: python
-        :caption: Python
+.. code-block:: bash
+    :caption: Unix CLI

-        from primaite.main import run
+    cd ~/primaite/2.0.0
+    source ./.venv/bin/activate
+    primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml

-        training_config = <path to training config yaml file>
-        lay_down_config = <path to lay down config yaml file>
-        run(training_config, lay_down_config)
+.. code-block:: powershell
+    :caption: Powershell CLI
+
+    cd ~\primaite\2.0.0
+    .\.venv\Scripts\activate
+    primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml
+
+
+.. code-block:: python
+    :caption: Python
+
+    from primaite.main import run
+
+    training_config = <path to training config yaml file>
+    lay_down_config = <path to lay down config yaml file>
+    run(training_config, lay_down_config)

 When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/2.0.0/sessions``).
 The sub-directory is formatted as such: ``~/primaite/2.0.0/sessions/<yyyy-mm-dd>/<yyyy-mm-dd>_<hh-mm-dd>/``
@@ -51,31 +51,33 @@ For example, when running a session at 17:30:00 on 31st January 2023, the sessio

 To run a PrimAITE session using legacy training or laydown config files, add the ``--legacy-tc`` and/or ``legacy-ldc`` options.

-.. tabs::
-
-    .. code-tab:: bash
-        :caption: Unix CLI
-
-        cd ~/primaite/2.0.0
-        source ./.venv/bin/activate
-        primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc
-
-    .. code-tab:: powershell
-        :caption: Powershell CLI
-
-        cd ~\primaite\2.0.0
-        .\.venv\Scripts\activate
-        primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc


-    .. code-tab:: python
-        :caption: Python
+.. code-block:: bash
+    :caption: Unix CLI
+
+    cd ~/primaite/2.0.0
+    source ./.venv/bin/activate
+    primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc
+
+.. code-block:: powershell
+    :caption: Powershell CLI
+
+    cd ~\primaite\2.0.0
+    .\.venv\Scripts\activate
+    primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc
+
+
+.. code-block:: python
+    :caption: Python
+
+    from primaite.main import run
+
+    training_config = <path to legacy training config yaml file>
+    lay_down_config = <path to legacy lay down config yaml file>
+    run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True)

-        from primaite.main import run

-        training_config = <path to legacy training config yaml file>
-        lay_down_config = <path to legacy lay down config yaml file>
-        run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True)


 Outputs
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -66,7 +66,6 @@ dev = [
    "pytest-flake8==1.1.1",
    "setuptools==66",
    "Sphinx==6.1.3",
-    "sphinx-code-tabs==0.5.3",
    "sphinx-copybutton==0.5.2",
    "wheel==0.38.4"
 ]