diff --git a/.github/workflows/build-sphinx.yml b/.github/workflows/build-sphinx.yml new file mode 100644 index 00000000..82da1c6b --- /dev/null +++ b/.github/workflows/build-sphinx.yml @@ -0,0 +1,60 @@ +name: build-sphinx-to-github-pages + +env: + GITHUB_ACTOR: Autonomous-Resilient-Cyber-Defence + GITHUB_REPOSITORY: Autonomous-Resilient-Cyber-Defence/PrimAITE + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN}} + +on: + push: + branches: [main] + + +jobs: + build: + + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10"] + + steps: + - uses: actions/checkout@v3 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python-version }} + + - name: Install python dev + run: | + set -x + sudo apt-get update + sudo add-apt-repository ppa:deadsnakes/ppa -y + sudo apt install python${{ matrix.python-version}}-dev -y + + - name: Install Git + run: | + set -x + sudo apt-get install -y git + shell: bash + + - name: Set pip, wheel, setuptools versions + run: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build + + - name: Install PrimAITE for docs autosummary + run: | + set -x + python -m pip install -e .[dev] + + - name: Run build script for Sphinx pages + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + run: | + set -x + bash $PWD/docs/build-sphinx-docs-to-github-pages.sh diff --git a/LICENSE b/LICENSE index a47c5477..93d6f98b 100644 --- a/LICENSE +++ b/LICENSE @@ -1,28 +1,21 @@ -GFX License +MIT License -GFX Conditions +Copyright (c) 2023 - 2025 Defence Science and Technology Laboratory UK (https://dstl.gov.uk) -These GFX conditions confirm the provision of the following artefacts as GFX by Defence Science and Technology -Laboratory UK (DSTL) to QinetiQ Training and Simulation Ltd (QTSL) (and subcontractors engaged in activity on task by -request to the QQ mailbox): +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: -- PrimAITE Overview -- Access to PrimAITE & user instructions -- Track 2 tech support +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. -Suppliers will be required to sign up to the QTSL Collaborative Working Environment (CWE) SyOPs and fill out a User -Access Request Form. Provided they have a minimum of Cyber Essentials, and the user has the required clearance, they -will then be provided with credentials to access the site by QQ. - -DSTL mandate that any changes made to the PrimAITE source code be passed back to QTSL (during or on termination of the -task) so that QQ can capture any potential enhancements to PrimAITE. - -This contains OFFICIAL information to be used to inform work on ARCD tasks (under SERAPIS). - -The material is supplied in confidence to QQ and their subcontractors under SERAPIS, and is issued to inform only those -that need to know its contents in the course of their official duties whilst engaged in activities under the contract. -The material consists of proprietary information which is the property of the Crown. The information contained within -the material may constitute valuable technical information and be commercially sensitive in relation to third parties; -therefore it may not be used or copied for any non-Governmental or commercial purpose without the prior written consent -of DSTL. The material must be stored and protected appropriately.All material must be destroyed at the end of the task. -Please note the contractual obligations relating to provision of these materials. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 2a6a1f7e..b7c7145c 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effect - The ability to model a relevant platform / system context; -- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, services and processes; +- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems and services; - Operates at machine-speed to enable fast training cycles. @@ -24,7 +24,7 @@ PrimAITE presents the following features: - Application of IERs to the platform / system laydown adheres to the ACL ruleset; -- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any OpenAI gym compliant defensive agents; +- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any compliant defensive agents; - Full capture of discrete logs relating to agent training (full system state, agent actions taken, instantaneous and average reward for every step of every episode); diff --git a/docs/build-sphinx-docs-to-github-pages.sh b/docs/build-sphinx-docs-to-github-pages.sh new file mode 100644 index 00000000..f1d40647 --- /dev/null +++ b/docs/build-sphinx-docs-to-github-pages.sh @@ -0,0 +1,67 @@ +#!/bin/bash +set -x + +apt-get update +apt-get -y install git rsync python3-sphinx + +pwd ls -lah +export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) + +############## +# BUILD DOCS # +############## + +cd docs +# Python Sphinx, configured with source/conf.py +# See https://www.sphinx-doc.org/ +make clean +make html + +cd .. +####################### +# Update GitHub Pages # +####################### + +git config --global user.name "${GITHUB_ACTOR}" +git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com" + +docroot=`mktemp -d` + +rsync -av $PWD/docs/_build/html/ "${docroot}/" + +pushd "${docroot}" + +git init +git remote add deploy "https://token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" +git checkout -b sphinx-docs-github-pages + +# Adds .nojekyll file to the root to signal to GitHub that +# directories that start with an underscore (_) can remain +touch .nojekyll + +# Add README +cat > README.md <`_), and help bridge the Sim-to-Real gap into Secondary level environments. What is PrimAITE built with --------------------------------------- +--------------------------- * `OpenAI's Gym `_ is used as the basis for AI blue agent interaction with the PrimAITE environment * `Networkx `_ is used as the underlying data structure used for the PrimAITE environment @@ -29,8 +85,8 @@ What is PrimAITE built with * `Plotly `_ is used for building high level charts -Where next? ------------- +Getting Started with PrimAITE +----------------------------- Head over to the :ref:`getting-started` page to install and setup PrimAITE! diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst index f07f1d27..1dbf9dec 100644 --- a/docs/source/getting_started.rst +++ b/docs/source/getting_started.rst @@ -14,20 +14,19 @@ Pre-Requisites In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.10 installed. If you don't already have it, this is how to install it: -.. tabs:: lang +.. code-block:: bash + :caption: Unix - .. code-tab:: bash - :caption: Unix + sudo add-apt-repository ppa:deadsnakes/ppa + sudo apt install python3.10 + sudo apt-get install python3-pip + sudo apt-get install python3-venv - sudo add-apt-repository ppa:deadsnakes/ppa - sudo apt install python3.10 - sudo apt-get install python3-pip - sudo apt-get install python3-venv - .. code-tab:: text - :caption: Windows (Powershell) +.. code-block:: text + :caption: Windows (Powershell) - - Manual install from: https://www.python.org/downloads/release/python-31011/ + - Manual install from: https://www.python.org/downloads/release/python-31011/ **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS. @@ -36,30 +35,30 @@ Install PrimAITE 1. Create a primaite directory in your home directory: -.. tabs:: lang - .. code-tab:: bash - :caption: Unix - mkdir ~/primaite/2.0.0 +.. code-block:: bash + :caption: Unix - .. code-tab:: powershell - :caption: Windows (Powershell) + mkdir ~/primaite/2.0.0 - mkdir ~\primaite\2.0.0 +.. code-block:: powershell + :caption: Windows (Powershell) + + mkdir ~\primaite\2.0.0 2. Navigate to the primaite directory and create a new python virtual environment (venv) -.. tabs:: lang - .. code-tab:: bash - :caption: Unix - cd ~/primaite/2.0.0 - python3 -m venv .venv +.. code-block:: bash + :caption: Unix - .. code-tab:: powershell - :caption: Windows (Powershell) + cd ~/primaite/2.0.0 + python3 -m venv .venv + +.. code-block:: powershell + :caption: Windows (Powershell) cd ~\primaite\2.0.0 python3 -m venv .venv @@ -67,44 +66,41 @@ Install PrimAITE 3. Activate the venv -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - source .venv/bin/activate + source .venv/bin/activate - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - .\.venv\Scripts\activate + .\.venv\Scripts\activate 4. Install PrimAITE using pip from PyPi -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - pip install primaite + pip install primaite - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - pip install primaite + pip install primaite 5. Perform the PrimAITE setup -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - primaite setup + primaite setup - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) primaite setup @@ -123,33 +119,31 @@ of your choice: Create and activate your Python virtual environment (venv) -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - python3 -m venv venv - source venv/bin/activate + python3 -m venv venv + source venv/bin/activate - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - python3 -m venv venv - .\venv\Scripts\activate + python3 -m venv venv + .\venv\Scripts\activate Install PrimAITE with the dev extra -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - pip install -e .[dev] + pip install -e .[dev] - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - pip install -e .[dev] + pip install -e .[dev] To view the complete list of packages installed during PrimAITE installation, go to the dependencies page (:ref:`Dependencies`). diff --git a/docs/source/primaite_session.rst b/docs/source/primaite_session.rst index ed023499..8ccc9070 100644 --- a/docs/source/primaite_session.rst +++ b/docs/source/primaite_session.rst @@ -15,31 +15,31 @@ A PrimAITE session can be ran either with the ``primaite session`` command from Both the ``primaite session`` and :func:`primaite.main.run` take a training config and a lay down config as parameters. -.. tabs:: - - .. code-tab:: bash - :caption: Unix CLI - - cd ~/primaite/2.0.0 - source ./.venv/bin/activate - primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml - - .. code-tab:: powershell - :caption: Powershell CLI - - cd ~\primaite\2.0.0 - .\.venv\Scripts\activate - primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml - .. code-tab:: python - :caption: Python +.. code-block:: bash + :caption: Unix CLI - from primaite.main import run + cd ~/primaite/2.0.0 + source ./.venv/bin/activate + primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml - training_config = - lay_down_config = - run(training_config, lay_down_config) +.. code-block:: powershell + :caption: Powershell CLI + + cd ~\primaite\2.0.0 + .\.venv\Scripts\activate + primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml + + +.. code-block:: python + :caption: Python + + from primaite.main import run + + training_config = + lay_down_config = + run(training_config, lay_down_config) When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/2.0.0/sessions``). The sub-directory is formatted as such: ``~/primaite/2.0.0/sessions//_/`` @@ -49,6 +49,36 @@ For example, when running a session at 17:30:00 on 31st January 2023, the sessio ``primaite session`` can be ran in the terminal/command prompt without arguments. It will use the default configs in the directory ``primaite/config/example_config``. +To run a PrimAITE session using legacy training or laydown config files, add the ``--legacy-tc`` and/or ``legacy-ldc`` options. + + + +.. code-block:: bash + :caption: Unix CLI + + cd ~/primaite/2.0.0 + source ./.venv/bin/activate + primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc + +.. code-block:: powershell + :caption: Powershell CLI + + cd ~\primaite\2.0.0 + .\.venv\Scripts\activate + primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc + + +.. code-block:: python + :caption: Python + + from primaite.main import run + + training_config = + lay_down_config = + run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True) + + + Outputs ------- diff --git a/pyproject.toml b/pyproject.toml index a33b2183..229d29d5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ license = {file = "LICENSE"} requires-python = ">=3.8, <3.11" dynamic = ["version", "readme"] classifiers = [ - "License :: GFX", + "License :: OSI Approved :: MIT License", "Development Status :: 5 - Production/Stable", "Operating System :: Microsoft :: Windows", "Operating System :: MacOS", @@ -65,7 +65,6 @@ dev = [ "pytest-flake8==1.1.1", "setuptools==66", "Sphinx==6.1.3", - "sphinx-code-tabs==0.5.3", "sphinx-copybutton==0.5.2", "wheel==0.38.4" ]