From 3fae05d9718517b46a458e04fa5cdd9d765e08d3 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Fri, 28 Jul 2023 14:41:39 +0100 Subject: [PATCH 1/8] #1711 - Last minute docs changes --- docs/index.rst | 9 ++++----- docs/source/primaite_session.rst | 28 ++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 3b1a13ec..2c7d4690 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,7 +6,7 @@ Welcome to PrimAITE's documentation ==================================== What is PrimAITE? ------------------------- +----------------- PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. It incorporates the functionality required of a Primary-level environment, as specified in the Dstl ARCD Training Environment Matrix document: @@ -14,10 +14,9 @@ PrimAITE (Primary-level AI Training Environment) is a simulation environment for * The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes; * Operates at machine-speed to enable fast training cycles. -PrimAITE aims to evolve into an ARCD environment that could be used as the follow-on from Reception level approaches (e.g. `Yawning-Titan `_), and help bridge the Sim-to-Real gap into Secondary level environments. What is PrimAITE built with --------------------------------------- +--------------------------- * `OpenAI's Gym `_ is used as the basis for AI blue agent interaction with the PrimAITE environment * `Networkx `_ is used as the underlying data structure used for the PrimAITE environment @@ -29,8 +28,8 @@ What is PrimAITE built with * `Plotly `_ is used for building high level charts -Where next? ------------- +Getting Started with PrimAITE +----------------------------- Head over to the :ref:`getting-started` page to install and setup PrimAITE! diff --git a/docs/source/primaite_session.rst b/docs/source/primaite_session.rst index ed023499..15ba9f4c 100644 --- a/docs/source/primaite_session.rst +++ b/docs/source/primaite_session.rst @@ -49,6 +49,34 @@ For example, when running a session at 17:30:00 on 31st January 2023, the sessio ``primaite session`` can be ran in the terminal/command prompt without arguments. It will use the default configs in the directory ``primaite/config/example_config``. +To run a PrimAITE session using legacy training or laydown config files, add the ``--legacy-tc`` and/or ``legacy-ldc`` options. + +.. tabs:: + + .. code-tab:: bash + :caption: Unix CLI + + cd ~/primaite/2.0.0 + source ./.venv/bin/activate + primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc + + .. code-tab:: powershell + :caption: Powershell CLI + + cd ~\primaite\2.0.0 + .\.venv\Scripts\activate + primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc + + + .. code-tab:: python + :caption: Python + + from primaite.main import run + + training_config = + lay_down_config = + run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True) + Outputs ------- From da8f91c78cca05d95ad69b8b7e6270a81b60cbff Mon Sep 17 00:00:00 2001 From: jamesshort1 <107395948+jamesshort1@users.noreply.github.com> Date: Mon, 31 Jul 2023 09:16:24 +0100 Subject: [PATCH 2/8] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2a6a1f7e..a2adfde8 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effect - The ability to model a relevant platform / system context; -- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, services and processes; +- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems and services; - Operates at machine-speed to enable fast training cycles. From efb230a5fc77ec5559e3926e1ff7b49cf801d1d4 Mon Sep 17 00:00:00 2001 From: jamesshort1 <107395948+jamesshort1@users.noreply.github.com> Date: Mon, 31 Jul 2023 09:17:48 +0100 Subject: [PATCH 3/8] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a2adfde8..b7c7145c 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ PrimAITE presents the following features: - Application of IERs to the platform / system laydown adheres to the ACL ruleset; -- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any OpenAI gym compliant defensive agents; +- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any compliant defensive agents; - Full capture of discrete logs relating to agent training (full system state, agent actions taken, instantaneous and average reward for every step of every episode); From 4ed31a52e5ed165a3d93f5c54e5360ffb84a3c27 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Thu, 3 Aug 2023 16:04:23 +0100 Subject: [PATCH 4/8] Updated the What is PrimAITE? section in index.rst. Dropped the use of sphinx-code-tabs in the docs as building the docs in pdf (make latexpdf) is suddenly complaining about the tab buttons. --- docs/conf.py | 1 - docs/index.rst | 65 ++++++++++++++++-- docs/source/getting_started.rst | 114 +++++++++++++++---------------- docs/source/primaite_session.rst | 86 +++++++++++------------ pyproject.toml | 1 - 5 files changed, 159 insertions(+), 108 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 4a805916..efd60b49 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -43,7 +43,6 @@ extensions = [ "sphinx.ext.viewcode", # Add a link to the Python source code for classes, functions etc. "sphinx.ext.todo", "sphinx_copybutton", # Adds a copy button to code blocks - "sphinx_code_tabs", # Enables tabbed code blocks ] diff --git a/docs/index.rst b/docs/index.rst index 2c7d4690..918e1350 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -8,11 +8,68 @@ Welcome to PrimAITE's documentation What is PrimAITE? ----------------- -PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. It incorporates the functionality required of a Primary-level environment, as specified in the Dstl ARCD Training Environment Matrix document: +Overview +^^^^^^^^ + +The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effective simulation capability for the purposes of training and evaluating AI in a cyber-defensive role. It incorporates the functionality required of a primary-level ARCD environment, which includes: + +- The ability to model a relevant platform / system context; +- Modelling an adversarial agent that the defensive agent can be trained and evaluated against; +- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, operating systems, services and traffic loading on links; +- Modelling background pattern-of-life; +- Operates at machine-speed to enable fast training cycles. + +Features +^^^^^^^^ + +PrimAITE incorporates the following features: + +- Highly configurable (via YAML files) to provide the means to model a variety of platform / system laydowns, mission profiles and adversarial attack scenarios; +- A Reinforcement Learning (RL) reward function based on (a) the ability to counter the modelled adversarial cyber-attack, and (b) the ability to ensure mission success; +- Provision of logging to support AI performance / effectiveness assessment; +- Uses the concept of Information Exchange Requirements (IERs) to model background pattern of life, adversarial behaviour and mission data (on a sliding scale of criticality); +- An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP address, destination IP address, protocol and port); +- Application of traffic to the links of the platform / system laydown adheres to the ACL ruleset; +- Presents both an OpenAI gym and Ray RLLib interface to the environment, allowing integration with any compliant defensive agents; +- Allows for the saving and loading of trained defensive agents; +- Stochastic adversarial agent behaviour; +- Full capture of discrete logs relating to agent training or evaluation (system state, agent actions taken, instantaneous and average reward for every step of every episode); +- Distinct control over running a training and / or evaluation session; +- NetworkX provides laydown visualisation capability. + +Architecture +^^^^^^^^^^^^ + +PrimAITE is a Python application and is therefore Operating System agnostic. The OpenAI gym and Ray RLLib frameworks are employed to provide an interface and source for AI agents. Configuration of PrimAITE is achieved via included YAML files which support full control over the platform / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count. NetworkX based nodes and links host Python classes to present attributes and methods, and hence a more representative platform / system can be modelled within the simulation. + + + +Training & Evaluation Capability +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its OpenAI Gym and RLLib compliant interface. Scenarios can be constructed to reflect platform / system laydowns consisting of any configuration of nodes (e.g. PCs, servers, switches etc.) and network links between them. All nodes can be configured to model services (and their status) and the traffic loading between them over the network links. Traffic loading is broken down into a per service granularity, relating directly to a protocol (e.g. Service A would be configured as a TCP service, and TCP traffic then flows between instances of Service A under the direction of a tailored IER). Highlights of PrimAITE’s training and evaluation capability are: + +- The scenario is not bound to a representation of any platform, system, technology or mission; +- Fully configurable (network / system laydown, IERs, node pattern-of-life, ACL, number of episodes, steps per episode) and repeatable to suit the requirements of AI agents; +- Can integrate with any OpenAI Gym or RLLib compliant AI agent. + +Use of PrimAITE default scenarios within ARCD is supported by a “Use Case Profile” tailored to the scenario. + +AI Assessment Capability +^^^^^^^^^^^^^^^^^^^^^^^^ + +PrimAITE includes the capability to support in-depth assessment of cyber defence AI by outputting logs of the environment state and AI behaviour throughout both training and evaluation sessions. These logs include the following data: + +- Timestamp; +- Episode and step number; +- Agent identifier; +- Observation space; +- Action taken (by defensive AI); +- Reward value. + +Logs are available in CSV format and provide coverage of the above data for every step of every episode. + -* The ability to model a relevant platform / system context; -* The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes; -* Operates at machine-speed to enable fast training cycles. What is PrimAITE built with diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst index f07f1d27..1dbf9dec 100644 --- a/docs/source/getting_started.rst +++ b/docs/source/getting_started.rst @@ -14,20 +14,19 @@ Pre-Requisites In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.10 installed. If you don't already have it, this is how to install it: -.. tabs:: lang +.. code-block:: bash + :caption: Unix - .. code-tab:: bash - :caption: Unix + sudo add-apt-repository ppa:deadsnakes/ppa + sudo apt install python3.10 + sudo apt-get install python3-pip + sudo apt-get install python3-venv - sudo add-apt-repository ppa:deadsnakes/ppa - sudo apt install python3.10 - sudo apt-get install python3-pip - sudo apt-get install python3-venv - .. code-tab:: text - :caption: Windows (Powershell) +.. code-block:: text + :caption: Windows (Powershell) - - Manual install from: https://www.python.org/downloads/release/python-31011/ + - Manual install from: https://www.python.org/downloads/release/python-31011/ **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS. @@ -36,30 +35,30 @@ Install PrimAITE 1. Create a primaite directory in your home directory: -.. tabs:: lang - .. code-tab:: bash - :caption: Unix - mkdir ~/primaite/2.0.0 +.. code-block:: bash + :caption: Unix - .. code-tab:: powershell - :caption: Windows (Powershell) + mkdir ~/primaite/2.0.0 - mkdir ~\primaite\2.0.0 +.. code-block:: powershell + :caption: Windows (Powershell) + + mkdir ~\primaite\2.0.0 2. Navigate to the primaite directory and create a new python virtual environment (venv) -.. tabs:: lang - .. code-tab:: bash - :caption: Unix - cd ~/primaite/2.0.0 - python3 -m venv .venv +.. code-block:: bash + :caption: Unix - .. code-tab:: powershell - :caption: Windows (Powershell) + cd ~/primaite/2.0.0 + python3 -m venv .venv + +.. code-block:: powershell + :caption: Windows (Powershell) cd ~\primaite\2.0.0 python3 -m venv .venv @@ -67,44 +66,41 @@ Install PrimAITE 3. Activate the venv -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - source .venv/bin/activate + source .venv/bin/activate - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - .\.venv\Scripts\activate + .\.venv\Scripts\activate 4. Install PrimAITE using pip from PyPi -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - pip install primaite + pip install primaite - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - pip install primaite + pip install primaite 5. Perform the PrimAITE setup -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - primaite setup + primaite setup - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) primaite setup @@ -123,33 +119,31 @@ of your choice: Create and activate your Python virtual environment (venv) -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - python3 -m venv venv - source venv/bin/activate + python3 -m venv venv + source venv/bin/activate - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - python3 -m venv venv - .\venv\Scripts\activate + python3 -m venv venv + .\venv\Scripts\activate Install PrimAITE with the dev extra -.. tabs:: lang - .. code-tab:: bash - :caption: Unix +.. code-block:: bash + :caption: Unix - pip install -e .[dev] + pip install -e .[dev] - .. code-tab:: powershell - :caption: Windows (Powershell) +.. code-block:: powershell + :caption: Windows (Powershell) - pip install -e .[dev] + pip install -e .[dev] To view the complete list of packages installed during PrimAITE installation, go to the dependencies page (:ref:`Dependencies`). diff --git a/docs/source/primaite_session.rst b/docs/source/primaite_session.rst index 15ba9f4c..8ccc9070 100644 --- a/docs/source/primaite_session.rst +++ b/docs/source/primaite_session.rst @@ -15,31 +15,31 @@ A PrimAITE session can be ran either with the ``primaite session`` command from Both the ``primaite session`` and :func:`primaite.main.run` take a training config and a lay down config as parameters. -.. tabs:: - - .. code-tab:: bash - :caption: Unix CLI - - cd ~/primaite/2.0.0 - source ./.venv/bin/activate - primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml - - .. code-tab:: powershell - :caption: Powershell CLI - - cd ~\primaite\2.0.0 - .\.venv\Scripts\activate - primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml - .. code-tab:: python - :caption: Python +.. code-block:: bash + :caption: Unix CLI - from primaite.main import run + cd ~/primaite/2.0.0 + source ./.venv/bin/activate + primaite session --tc ./config/my_training_config.yaml --ldc ./config/my_lay_down_config.yaml - training_config = - lay_down_config = - run(training_config, lay_down_config) +.. code-block:: powershell + :caption: Powershell CLI + + cd ~\primaite\2.0.0 + .\.venv\Scripts\activate + primaite session --tc .\config\my_training_config.yaml --ldc .\config\my_lay_down_config.yaml + + +.. code-block:: python + :caption: Python + + from primaite.main import run + + training_config = + lay_down_config = + run(training_config, lay_down_config) When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/2.0.0/sessions``). The sub-directory is formatted as such: ``~/primaite/2.0.0/sessions//_/`` @@ -51,31 +51,33 @@ For example, when running a session at 17:30:00 on 31st January 2023, the sessio To run a PrimAITE session using legacy training or laydown config files, add the ``--legacy-tc`` and/or ``legacy-ldc`` options. -.. tabs:: - - .. code-tab:: bash - :caption: Unix CLI - - cd ~/primaite/2.0.0 - source ./.venv/bin/activate - primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc - - .. code-tab:: powershell - :caption: Powershell CLI - - cd ~\primaite\2.0.0 - .\.venv\Scripts\activate - primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc - .. code-tab:: python - :caption: Python +.. code-block:: bash + :caption: Unix CLI + + cd ~/primaite/2.0.0 + source ./.venv/bin/activate + primaite session --tc ./config/my_legacy_training_config.yaml --legacy-tc --ldc ./config/my_legacy_lay_down_config.yaml --legacy-ldc + +.. code-block:: powershell + :caption: Powershell CLI + + cd ~\primaite\2.0.0 + .\.venv\Scripts\activate + primaite session --tc .\config\my_legacy_training_config.yaml --legacy-tc --ldc .\config\my_legacy_lay_down_config.yaml --legacy-ldc + + +.. code-block:: python + :caption: Python + + from primaite.main import run + + training_config = + lay_down_config = + run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True) - from primaite.main import run - training_config = - lay_down_config = - run(training_config, lay_down_config, legacy_training_config=True, legacy_lay_down_config=True) Outputs diff --git a/pyproject.toml b/pyproject.toml index a33b2183..2f17a827 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -65,7 +65,6 @@ dev = [ "pytest-flake8==1.1.1", "setuptools==66", "Sphinx==6.1.3", - "sphinx-code-tabs==0.5.3", "sphinx-copybutton==0.5.2", "wheel==0.38.4" ] From 6bf348e5c382b4202b5377bfcd99e5a4e9b268cb Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Tue, 15 Aug 2023 11:14:23 +0100 Subject: [PATCH 5/8] Added the DSTL MIT license and updated the license in pyproject.toml --- LICENSE | 41 +++++++++++++++++------------------------ pyproject.toml | 2 +- 2 files changed, 18 insertions(+), 25 deletions(-) diff --git a/LICENSE b/LICENSE index a47c5477..93d6f98b 100644 --- a/LICENSE +++ b/LICENSE @@ -1,28 +1,21 @@ -GFX License +MIT License -GFX Conditions +Copyright (c) 2023 - 2025 Defence Science and Technology Laboratory UK (https://dstl.gov.uk) -These GFX conditions confirm the provision of the following artefacts as GFX by Defence Science and Technology -Laboratory UK (DSTL) to QinetiQ Training and Simulation Ltd (QTSL) (and subcontractors engaged in activity on task by -request to the QQ mailbox): +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: -- PrimAITE Overview -- Access to PrimAITE & user instructions -- Track 2 tech support +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. -Suppliers will be required to sign up to the QTSL Collaborative Working Environment (CWE) SyOPs and fill out a User -Access Request Form. Provided they have a minimum of Cyber Essentials, and the user has the required clearance, they -will then be provided with credentials to access the site by QQ. - -DSTL mandate that any changes made to the PrimAITE source code be passed back to QTSL (during or on termination of the -task) so that QQ can capture any potential enhancements to PrimAITE. - -This contains OFFICIAL information to be used to inform work on ARCD tasks (under SERAPIS). - -The material is supplied in confidence to QQ and their subcontractors under SERAPIS, and is issued to inform only those -that need to know its contents in the course of their official duties whilst engaged in activities under the contract. -The material consists of proprietary information which is the property of the Crown. The information contained within -the material may constitute valuable technical information and be commercially sensitive in relation to third parties; -therefore it may not be used or copied for any non-Governmental or commercial purpose without the prior written consent -of DSTL. The material must be stored and protected appropriately.All material must be destroyed at the end of the task. -Please note the contractual obligations relating to provision of these materials. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/pyproject.toml b/pyproject.toml index 2f17a827..229d29d5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ license = {file = "LICENSE"} requires-python = ">=3.8, <3.11" dynamic = ["version", "readme"] classifiers = [ - "License :: GFX", + "License :: OSI Approved :: MIT License", "Development Status :: 5 - Production/Stable", "Operating System :: Microsoft :: Windows", "Operating System :: MacOS", From aaf1b16912cb0b7a40234dd9185f3b728bf5dda5 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Tue, 15 Aug 2023 11:26:15 +0100 Subject: [PATCH 6/8] Added sphinx docs build pipeline for GitHub pages on release --- .github/workflows/build-sphinx.yml | 60 ++++++++++++++++++++ docs/build-sphinx-docs-to-github-pages.sh | 67 +++++++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 .github/workflows/build-sphinx.yml create mode 100644 docs/build-sphinx-docs-to-github-pages.sh diff --git a/.github/workflows/build-sphinx.yml b/.github/workflows/build-sphinx.yml new file mode 100644 index 00000000..82da1c6b --- /dev/null +++ b/.github/workflows/build-sphinx.yml @@ -0,0 +1,60 @@ +name: build-sphinx-to-github-pages + +env: + GITHUB_ACTOR: Autonomous-Resilient-Cyber-Defence + GITHUB_REPOSITORY: Autonomous-Resilient-Cyber-Defence/PrimAITE + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN}} + +on: + push: + branches: [main] + + +jobs: + build: + + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10"] + + steps: + - uses: actions/checkout@v3 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python-version }} + + - name: Install python dev + run: | + set -x + sudo apt-get update + sudo add-apt-repository ppa:deadsnakes/ppa -y + sudo apt install python${{ matrix.python-version}}-dev -y + + - name: Install Git + run: | + set -x + sudo apt-get install -y git + shell: bash + + - name: Set pip, wheel, setuptools versions + run: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build + + - name: Install PrimAITE for docs autosummary + run: | + set -x + python -m pip install -e .[dev] + + - name: Run build script for Sphinx pages + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + run: | + set -x + bash $PWD/docs/build-sphinx-docs-to-github-pages.sh diff --git a/docs/build-sphinx-docs-to-github-pages.sh b/docs/build-sphinx-docs-to-github-pages.sh new file mode 100644 index 00000000..f1d40647 --- /dev/null +++ b/docs/build-sphinx-docs-to-github-pages.sh @@ -0,0 +1,67 @@ +#!/bin/bash +set -x + +apt-get update +apt-get -y install git rsync python3-sphinx + +pwd ls -lah +export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) + +############## +# BUILD DOCS # +############## + +cd docs +# Python Sphinx, configured with source/conf.py +# See https://www.sphinx-doc.org/ +make clean +make html + +cd .. +####################### +# Update GitHub Pages # +####################### + +git config --global user.name "${GITHUB_ACTOR}" +git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com" + +docroot=`mktemp -d` + +rsync -av $PWD/docs/_build/html/ "${docroot}/" + +pushd "${docroot}" + +git init +git remote add deploy "https://token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" +git checkout -b sphinx-docs-github-pages + +# Adds .nojekyll file to the root to signal to GitHub that +# directories that start with an underscore (_) can remain +touch .nojekyll + +# Add README +cat > README.md < Date: Tue, 15 Aug 2023 13:23:25 +0100 Subject: [PATCH 7/8] Create spinx-pipeline --- .github/workflows/spinx-pipeline | 36 ++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 .github/workflows/spinx-pipeline diff --git a/.github/workflows/spinx-pipeline b/.github/workflows/spinx-pipeline new file mode 100644 index 00000000..22f666b0 --- /dev/null +++ b/.github/workflows/spinx-pipeline @@ -0,0 +1,36 @@ +# This is a basic workflow to help you get started with Actions + +name: CI + +# Controls when the workflow will run +on: + # Triggers the workflow on push or pull request events but only for the "dev" branch + push: + branches: [ "dev" ] + pull_request: + branches: [ "dev" ] + + # Allows you to run this workflow manually from the Actions tab + workflow_dispatch: + +# A workflow run is made up of one or more jobs that can run sequentially or in parallel +jobs: + # This workflow contains a single job called "build" + build: + # The type of runner that the job will run on + runs-on: ubuntu-latest + + # Steps represent a sequence of tasks that will be executed as part of the job + steps: + # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it + - uses: actions/checkout@v3 + + # Runs a single command using the runners shell + - name: Run a one-line script + run: echo Hello, world! + + # Runs a set of commands using the runners shell + - name: Run a multi-line script + run: | + echo Add other actions to build, + echo test, and deploy your project. From 506b0836eaafeb3abc648b3d6c4fe7202e69657c Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Tue, 15 Aug 2023 13:28:02 +0100 Subject: [PATCH 8/8] Dropped sphinx-pipeline --- .github/workflows/spinx-pipeline | 36 -------------------------------- 1 file changed, 36 deletions(-) delete mode 100644 .github/workflows/spinx-pipeline diff --git a/.github/workflows/spinx-pipeline b/.github/workflows/spinx-pipeline deleted file mode 100644 index 22f666b0..00000000 --- a/.github/workflows/spinx-pipeline +++ /dev/null @@ -1,36 +0,0 @@ -# This is a basic workflow to help you get started with Actions - -name: CI - -# Controls when the workflow will run -on: - # Triggers the workflow on push or pull request events but only for the "dev" branch - push: - branches: [ "dev" ] - pull_request: - branches: [ "dev" ] - - # Allows you to run this workflow manually from the Actions tab - workflow_dispatch: - -# A workflow run is made up of one or more jobs that can run sequentially or in parallel -jobs: - # This workflow contains a single job called "build" - build: - # The type of runner that the job will run on - runs-on: ubuntu-latest - - # Steps represent a sequence of tasks that will be executed as part of the job - steps: - # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - - uses: actions/checkout@v3 - - # Runs a single command using the runners shell - - name: Run a one-line script - run: echo Hello, world! - - # Runs a set of commands using the runners shell - - name: Run a multi-line script - run: | - echo Add other actions to build, - echo test, and deploy your project.