From 7f8666b3d1f8029b9978837f469c610b78adce17 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Thu, 20 Jul 2023 10:55:59 +0100 Subject: [PATCH 1/9] Create README.md --- README.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 00000000..78f36fba --- /dev/null +++ b/README.md @@ -0,0 +1 @@ +# PrimAITE From c6cbb5ae8d22cffc1e91e8aec878ee731afed96f Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 19:38:28 +0100 Subject: [PATCH 2/9] Create python-package.yml CI pipeline --- .github/workflows/python-package.yml | 66 ++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 .github/workflows/python-package.yml diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml new file mode 100644 index 00000000..055882d2 --- /dev/null +++ b/.github/workflows/python-package.yml @@ -0,0 +1,66 @@ +name: Python package + +on: + push: + branches: + - main + - dev + - dev-gui + - 'release/**' + pull_request: + branches: + - main + - dev + - dev-gui + - 'release/**' +jobs: + build: + + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.8", "3.9", "3.10"] + + steps: + - uses: actions/checkout@v3 + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python-version }} + + - name: Install python dev + run: | + sudo apt update + sudo add-apt-repository ppa:deadsnakes/ppa -y + sudo apt install python${{ matrix.python-version}}-dev -y + + - name: Install Build Dependencies + run: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build + + - name: Build PrimAITE + run: | + python -m build + + - name: Install Yawning-Titan + run: | + PRIMAITE_WHEEL=$(ls ./dist/primaite*.whl) + python -m pip install $PRIMAITE_WHEEL[dev] + + - name: Perform PrimAITE Setup + run: | + primaite setup + + - name: Lint with flake8 + run: | + # stop the build if there are Python syntax errors or undefined names + flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics + # exit-zero treats all errors as warnings. + flake8 . --count --exit-zero --max-complexity=10 --max-line-length=120 --statistics + + - name: Run unmarked tests + run: | + pytest tests/ From 5c5528bb94eb40ffaf6b97bd72367f11235db850 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 19:49:24 +0100 Subject: [PATCH 3/9] Updated the README.md with developer install specific instructions --- README.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index f7c6efd7..066a8af3 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ ## Getting Started with PrimAITE + ### Pre-Requisites In order to get **PrimAITE** installed, you will need to have the following installed: @@ -12,38 +13,37 @@ In order to get **PrimAITE** installed, you will need to have the following inst **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS. -### Installation from source -#### 1. Navigate to the PrimAITE folder and create a new python virtual environment (venv) +### Installation from source (Developer Install) +#### 1. Create a new python virtual environment (venv) ```unix -python3 -m venv +python3 -m venv venv ``` #### 2. Activate the venv ##### Unix ```bash -source /bin/activate +source venv/bin/activate ``` -##### Windows +##### Windows (Powershell) ```powershell -.\\Scripts\activate +.\venv\Scripts\activate ``` -#### 3. Install `primaite` into the venv along with all of it's dependencies - -```bash -python3 -m pip install -e . -``` - -### Development Installation -To install the development dependencies, postfix the command in step 3 above with the `[dev]` extra. Example: +#### 3. Install `primaite` with the dev extra into the venv along with all of it's dependencies ```bash python3 -m pip install -e .[dev] ``` +#### 4. Perform the PrimAITE setup: + +```bash +primaite setup +``` + ## Building documentation The PrimAITE documentation can be built with the following commands: @@ -53,7 +53,7 @@ cd docs make html ``` -##### Windows +##### Windows (Powershell) ```powershell cd docs .\make.bat html From 1c4695d3919422de169af08a2b00308651e9c011 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 20:05:44 +0100 Subject: [PATCH 4/9] Dropped the ADF build files and updated the package name install step in python-package.yml. Added bug_report.md and feature_request.md files for GitHub --- .azure/.pypirc | 6 -- .azure/artifact-release-pipeline.yaml | 38 --------- .azure/azure-build-deploy-docs-pipeline.yml | 49 ----------- .azure/azure-ci-build-pipeline.yaml | 90 --------------------- .azuredevops/pull_request_template.md | 12 --- .github/ISSUE_TEMPLATE/bug_report.md | 41 ++++++++++ .github/ISSUE_TEMPLATE/feature_request.md | 24 ++++++ .github/workflows/python-package.yml | 6 +- README.md | 4 - 9 files changed, 68 insertions(+), 202 deletions(-) delete mode 100644 .azure/.pypirc delete mode 100644 .azure/artifact-release-pipeline.yaml delete mode 100644 .azure/azure-build-deploy-docs-pipeline.yml delete mode 100644 .azure/azure-ci-build-pipeline.yaml delete mode 100644 .azuredevops/pull_request_template.md create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md diff --git a/.azure/.pypirc b/.azure/.pypirc deleted file mode 100644 index 9f89d0ea..00000000 --- a/.azure/.pypirc +++ /dev/null @@ -1,6 +0,0 @@ -[distutils] -Index-servers = - PrimAITE - -[PrimAITE] -Repository = https://pkgs.dev.azure.com/ma-dev-uk/PrimAITE/_packaging/PrimAITE/pypi/upload/ diff --git a/.azure/artifact-release-pipeline.yaml b/.azure/artifact-release-pipeline.yaml deleted file mode 100644 index 47e9aacc..00000000 --- a/.azure/artifact-release-pipeline.yaml +++ /dev/null @@ -1,38 +0,0 @@ -trigger: -- main - -pool: - vmImage: ubuntu-latest -strategy: - matrix: - Python310: - python.version: '3.10' - -steps: -- task: UsePythonVersion@0 - inputs: - versionSpec: '$(python.version)' - displayName: 'Use Python $(python.version)' - -- script: | - python -m pip install --upgrade pip==23.0.1 - pip install wheel==0.38.4 --upgrade - pip install setuptools==66 --upgrade - pip install build==0.10.0 - pip install twine - pip install keyring - pip install artifacts-keyring - displayName: 'Install build dependencies' - -- script: | - python -m build - displayName: 'Build PrimAITE sdist and wheel' - -- task: TwineAuthenticate@1 - displayName: 'Twine Authenticate' - inputs: - artifactFeed: PrimAITE/PrimAITE - -- script: | - python -m twine upload --verbose -r PrimAITE --config-file $(PYPIRC_PATH) dist/*.whl - displayName: 'Artifact Upload' diff --git a/.azure/azure-build-deploy-docs-pipeline.yml b/.azure/azure-build-deploy-docs-pipeline.yml deleted file mode 100644 index 0f44b0c8..00000000 --- a/.azure/azure-build-deploy-docs-pipeline.yml +++ /dev/null @@ -1,49 +0,0 @@ -name: Azure Static Web Apps CI/CD - -pr: none -trigger: - branches: - include: - - dev - -jobs: -- job: build_and_deploy_job - displayName: Build and Deploy Job - condition: or(eq(variables['Build.Reason'], 'Manual'),or(eq(variables['Build.Reason'], 'PullRequest'),eq(variables['Build.Reason'], 'IndividualCI'))) - pool: - vmImage: ubuntu-latest - variables: - - group: Azure-Static-Web-Apps-nice-bay-0ad032c03-variable-group - steps: - - checkout: self - submodules: true - - - script: | - python -m pip install --upgrade pip==23.0.1 - pip install wheel==0.38.4 --upgrade - pip install setuptools==66 --upgrade - pip install build==0.10.0 - displayName: 'Install build dependencies' - - - script: | - pip install -e .[dev] - displayName: 'Install Yawning-Titan for docs autosummary' - - - script: | - primaite setup - displayName: 'Perform PrimAITE Setup' - - - script: | - cd docs - make html - cd .. - cd .. - displayName: 'Build Docs' - - - task: AzureStaticWebApp@0 - inputs: - azure_static_web_apps_api_token: $(AZURE_STATIC_WEB_APPS_API_TOKEN_NICE_BAY_0AD032C03) - app_location: "/docs/_build/html" - api_location: "" - output_location: "/" - displayName: 'Deploy Docs to nice-bay-0ad032c03' diff --git a/.azure/azure-ci-build-pipeline.yaml b/.azure/azure-ci-build-pipeline.yaml deleted file mode 100644 index 0bb03594..00000000 --- a/.azure/azure-ci-build-pipeline.yaml +++ /dev/null @@ -1,90 +0,0 @@ -trigger: -- main -- dev -- feature/* -- hotfix/* -- bugfix/* -- release/* - -parameters: - # https://stackoverflow.com/a/70046417 - - name: matrix - type: object - default: - - job_name: 'UbuntuPython38' - py: '3.8' - img: 'ubuntu-latest' - every_time: false - - job_name: 'UbuntuPython310' - py: '3.10' - img: 'ubuntu-latest' - every_time: true - - job_name: 'WindowsPython38' - py: '3.8' - img: 'windows-latest' - every_time: false - - job_name: 'WindowsPython310' - py: '3.10' - img: 'windows-latest' - every_time: false - - job_name: 'MacOSPython38' - py: '3.8' - img: 'macOS-latest' - every_time: false - - job_name: 'MacOSPython310' - py: '3.10' - img: 'macOS-latest' - every_time: false - -stages: - - stage: Test - jobs: - - ${{ each item in parameters.matrix }}: - - job: ${{ item.job_name }} - pool: - vmImage: ${{ item.img }} - - condition: or( eq(variables['Build.Reason'], 'PullRequest'), ${{ item.every_time }} ) - - steps: - - task: UsePythonVersion@0 - inputs: - versionSpec: ${{ item.py }} - displayName: 'Use Python ${{ item.py }}' - - - script: | - python -m pip install pre-commit - pre-commit install - pre-commit run --all-files - displayName: 'Run pre-commits' - - - script: | - python -m pip install --upgrade pip==23.0.1 - pip install wheel==0.38.4 --upgrade - pip install setuptools==66 --upgrade - pip install build==0.10.0 - pip install pytest-azurepipelines - displayName: 'Install build dependencies' - - - script: | - python -m build - displayName: 'Build PrimAITE' - - - script: | - PRIMAITE_WHEEL=$(ls ./dist/primaite*.whl) - python -m pip install $PRIMAITE_WHEEL[dev] - displayName: 'Install PrimAITE' - condition: or(eq( variables['Agent.OS'], 'Linux' ), eq( variables['Agent.OS'], 'Darwin' )) - - - script: | - forfiles /p dist\ /m *.whl /c "cmd /c python -m pip install @file[dev]" - displayName: 'Install PrimAITE' - condition: eq( variables['Agent.OS'], 'Windows_NT' ) - - - script: | - primaite setup - displayName: 'Perform PrimAITE Setup' - - - script: | - pytest -n 4 - displayName: 'Run tests' diff --git a/.azuredevops/pull_request_template.md b/.azuredevops/pull_request_template.md deleted file mode 100644 index 538baf5c..00000000 --- a/.azuredevops/pull_request_template.md +++ /dev/null @@ -1,12 +0,0 @@ -## Summary -*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?* - -## Test process -*How have you tested this (if applicable)?* - -## Checklist -- [ ] This PR is linked to a **work item** -- [ ] I have performed **self-review** of the code -- [ ] I have written **tests** for any new functionality added with this PR -- [ ] I have updated the **documentation** if this PR changes or adds functionality -- [ ] I have run **pre-commit** checks for code style diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000..1ee05bfa --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,41 @@ +--- +name: Bug report +about: Create a report to help us improve +title: "[BUG] - " +labels: bug +assignees: '' + +--- + +### Describe the bug: + +A clear and concise description of what the bug is. + +### To Reproduce: + +Steps to reproduce the behaviour: + +1. Import '...' +2. Instantiate '....' +3. Pass to '....' +4. Run '....' +5. See error + +### Expected behaviour + +A clear and concise description of what you expected to happen. + +### Screenshots/Outputs + +If applicable, add screenshots to help explain your problem. + +### Environment (please complete the following information) + + - **OS:** [e.g. Ubuntu 22.04] + - **Python:** [e.g. 3.10.11] + - **PrimAITE Version:** [e.g. v2.0.0] + - **Software:** [e.g. cli, Jupyter, PyCharm, VSCode etc.] + +### Additional context + +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000..c0a20642 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,24 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: "[REQUEST] - " +labels: feature_request +assignees: '' + +--- + +### Is your feature request related to a problem? + +If so, please give a concise description of what the problem is. Ex. I'm always frustrated when [...] + +### Describe the solution you'd like: + +A clear and concise description of what you want to happen. + +### Describe alternatives you've considered: + +A clear and concise description of any alternative solutions or features you've considered. + +### Additional context: + +Add any other context or screenshots about the feature request here. diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml index 055882d2..ed94ad97 100644 --- a/.github/workflows/python-package.yml +++ b/.github/workflows/python-package.yml @@ -45,11 +45,11 @@ jobs: run: | python -m build - - name: Install Yawning-Titan + - name: Install PrimAITE run: | PRIMAITE_WHEEL=$(ls ./dist/primaite*.whl) python -m pip install $PRIMAITE_WHEEL[dev] - + - name: Perform PrimAITE Setup run: | primaite setup @@ -61,6 +61,6 @@ jobs: # exit-zero treats all errors as warnings. flake8 . --count --exit-zero --max-complexity=10 --max-line-length=120 --statistics - - name: Run unmarked tests + - name: Run tests run: | pytest tests/ diff --git a/README.md b/README.md index 066a8af3..d078829e 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,3 @@ make html cd docs .\make.bat html ``` - -This will build the documentation as a collection of HTML files which uses the Read The Docs sphinx theme. Other build -options are available but may require additional dependencies such as LaTeX and PDF. Please refer to the Sphinx documentation -for your specific output requirements. From efbb6ef8df4f21afdc2e50d4d5b181ad9adf85c1 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 20:17:29 +0100 Subject: [PATCH 5/9] Added a CONTRIBUTING.md and added a URL to the Yawning-Titan reference in index.rst --- CONTRIBUTING.md | 39 +++++++++++++++++++++++++++++++++++++++ docs/index.rst | 4 +--- 2 files changed, 40 insertions(+), 3 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..4d70ebb3 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,39 @@ +# How to contribute to PrimAITE? + + +### **Did you find a bug?** + + +* **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues). +* If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=bug&projects=&template=bug_report.md&title=%5BBUG%5D+-+%3Cbug+title+goes+here%3E). Be sure to follow our bug report template with the headers **Describe the bug**, **To Reproduce**, **Expected behaviour**, **Screenshots/Outputs**, **Environment**, and **Additional context** + + +### **Do you have a solution to fix the bug?** + +* [Fork the repository](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/fork). +* Install the pre-commit hook with `pre-commit install`. +* Implement the bug fix. +* Update documentation where applicable. +* Update the **UNRELEASED** section of the [CHANGELOG.md](CHANGELOG.md) file +* Write a suitable test/tests. +* Commit the bug fix to the dev branch on your fork. If the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the commit message (e.g. #1 references issue 1). +* Submit a pull request from your dev branch to the Autonomous-Resilient-Cyber-Defence/PrimAITE dev branch. Again, if the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the pull request description. + +### **Did you fix whitespace, format code, or make a purely cosmetic patch?** + +Changes that are cosmetic in nature and do not add anything substantial to the stability, functionality, or testability of PrimAITE will generally not be accepted. + +### **Do you intend to add a new feature or change an existing one?** + +* Submit a [feature request issue](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=feature_request&projects=&template=feature_request.md&title=%5BREQUEST%5D+-+%3Crequest+title+goes+here%3E). +* Know how to implement the new feature or change? Follow the same steps in the bug fix section above to fork, build, document, test, commit, and submit a pull request. + +### **Do you have questions about the source code?** + +Ask any question about how to use PrimAITE in our discussions section. + +### **Do you want to contribute to the PrimAITE documentation?** + +Please follow the "Do you intend to add a new feature or change an existing one?" section above and tag your feature request issue and pull request with the documentation tag. + +Thank you from the PrimAITE dev team! 🙌 diff --git a/docs/index.rst b/docs/index.rst index 208d5abc..3b1a13ec 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -14,9 +14,7 @@ PrimAITE (Primary-level AI Training Environment) is a simulation environment for * The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes; * Operates at machine-speed to enable fast training cycles. -PrimAITE aims to evolve into an ARCD environment that could be used as the follow-on from Reception level approaches (e.g. YAWNING TITAN), and help bridge the Sim-to-Real gap into Secondary level environments (e.g. IMAGINARY YAK). - -This is similar to the approach taken by FVEY international partners (e.g. AUS CyBORG, US NSA FARLAND and CAN CyGil). These environments are referenced by the Dstl ARCD Agent Training Environments Knowledge Transfer document (TR141342). +PrimAITE aims to evolve into an ARCD environment that could be used as the follow-on from Reception level approaches (e.g. `Yawning-Titan `_), and help bridge the Sim-to-Real gap into Secondary level environments. What is PrimAITE built with -------------------------------------- From 499219eb40d633076bb58b24b9743b5504435552 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 21:11:15 +0100 Subject: [PATCH 6/9] Added project urls to pyproject.toml and a setup.cfg file for PyPi to pickup author and url --- pyproject.toml | 6 ++++++ setup.cfg | 4 ++++ 2 files changed, 10 insertions(+) create mode 100644 setup.cfg diff --git a/pyproject.toml b/pyproject.toml index 9691f65c..c2c8076b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -81,3 +81,9 @@ order_by_type = "False" [tool.black] line-length = 120 + +[project.urls] +Homepage = "https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE" +Documentation = "https://Autonomous-Resilient-Cyber-Defence.github.io/PrimAITE/" +Repository = "https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE" +Changelog = "https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/blob/dev/CHANGELOG.md" diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 00000000..e46aafd6 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,4 @@ +[metadata] +url = https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE +author = Defence Science and Technology Laboratory UK +author_email = oss@dstl.gov.uk From fce4e7893356fb708437f30f3b710b17a20ca864 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 21:40:30 +0100 Subject: [PATCH 7/9] Synced with the github repo release --- .azure/.pypirc | 6 ++ .azure/artifact-release-pipeline.yaml | 38 +++++++++ .azure/azure-build-deploy-docs-pipeline.yml | 49 +++++++++++ .azure/azure-ci-build-pipeline.yaml | 90 +++++++++++++++++++++ .azuredevops/pull_request_template.md | 12 +++ 5 files changed, 195 insertions(+) create mode 100644 .azure/.pypirc create mode 100644 .azure/artifact-release-pipeline.yaml create mode 100644 .azure/azure-build-deploy-docs-pipeline.yml create mode 100644 .azure/azure-ci-build-pipeline.yaml create mode 100644 .azuredevops/pull_request_template.md diff --git a/.azure/.pypirc b/.azure/.pypirc new file mode 100644 index 00000000..9f89d0ea --- /dev/null +++ b/.azure/.pypirc @@ -0,0 +1,6 @@ +[distutils] +Index-servers = + PrimAITE + +[PrimAITE] +Repository = https://pkgs.dev.azure.com/ma-dev-uk/PrimAITE/_packaging/PrimAITE/pypi/upload/ diff --git a/.azure/artifact-release-pipeline.yaml b/.azure/artifact-release-pipeline.yaml new file mode 100644 index 00000000..47e9aacc --- /dev/null +++ b/.azure/artifact-release-pipeline.yaml @@ -0,0 +1,38 @@ +trigger: +- main + +pool: + vmImage: ubuntu-latest +strategy: + matrix: + Python310: + python.version: '3.10' + +steps: +- task: UsePythonVersion@0 + inputs: + versionSpec: '$(python.version)' + displayName: 'Use Python $(python.version)' + +- script: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build==0.10.0 + pip install twine + pip install keyring + pip install artifacts-keyring + displayName: 'Install build dependencies' + +- script: | + python -m build + displayName: 'Build PrimAITE sdist and wheel' + +- task: TwineAuthenticate@1 + displayName: 'Twine Authenticate' + inputs: + artifactFeed: PrimAITE/PrimAITE + +- script: | + python -m twine upload --verbose -r PrimAITE --config-file $(PYPIRC_PATH) dist/*.whl + displayName: 'Artifact Upload' diff --git a/.azure/azure-build-deploy-docs-pipeline.yml b/.azure/azure-build-deploy-docs-pipeline.yml new file mode 100644 index 00000000..0f44b0c8 --- /dev/null +++ b/.azure/azure-build-deploy-docs-pipeline.yml @@ -0,0 +1,49 @@ +name: Azure Static Web Apps CI/CD + +pr: none +trigger: + branches: + include: + - dev + +jobs: +- job: build_and_deploy_job + displayName: Build and Deploy Job + condition: or(eq(variables['Build.Reason'], 'Manual'),or(eq(variables['Build.Reason'], 'PullRequest'),eq(variables['Build.Reason'], 'IndividualCI'))) + pool: + vmImage: ubuntu-latest + variables: + - group: Azure-Static-Web-Apps-nice-bay-0ad032c03-variable-group + steps: + - checkout: self + submodules: true + + - script: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build==0.10.0 + displayName: 'Install build dependencies' + + - script: | + pip install -e .[dev] + displayName: 'Install Yawning-Titan for docs autosummary' + + - script: | + primaite setup + displayName: 'Perform PrimAITE Setup' + + - script: | + cd docs + make html + cd .. + cd .. + displayName: 'Build Docs' + + - task: AzureStaticWebApp@0 + inputs: + azure_static_web_apps_api_token: $(AZURE_STATIC_WEB_APPS_API_TOKEN_NICE_BAY_0AD032C03) + app_location: "/docs/_build/html" + api_location: "" + output_location: "/" + displayName: 'Deploy Docs to nice-bay-0ad032c03' diff --git a/.azure/azure-ci-build-pipeline.yaml b/.azure/azure-ci-build-pipeline.yaml new file mode 100644 index 00000000..0bb03594 --- /dev/null +++ b/.azure/azure-ci-build-pipeline.yaml @@ -0,0 +1,90 @@ +trigger: +- main +- dev +- feature/* +- hotfix/* +- bugfix/* +- release/* + +parameters: + # https://stackoverflow.com/a/70046417 + - name: matrix + type: object + default: + - job_name: 'UbuntuPython38' + py: '3.8' + img: 'ubuntu-latest' + every_time: false + - job_name: 'UbuntuPython310' + py: '3.10' + img: 'ubuntu-latest' + every_time: true + - job_name: 'WindowsPython38' + py: '3.8' + img: 'windows-latest' + every_time: false + - job_name: 'WindowsPython310' + py: '3.10' + img: 'windows-latest' + every_time: false + - job_name: 'MacOSPython38' + py: '3.8' + img: 'macOS-latest' + every_time: false + - job_name: 'MacOSPython310' + py: '3.10' + img: 'macOS-latest' + every_time: false + +stages: + - stage: Test + jobs: + - ${{ each item in parameters.matrix }}: + - job: ${{ item.job_name }} + pool: + vmImage: ${{ item.img }} + + condition: or( eq(variables['Build.Reason'], 'PullRequest'), ${{ item.every_time }} ) + + steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: ${{ item.py }} + displayName: 'Use Python ${{ item.py }}' + + - script: | + python -m pip install pre-commit + pre-commit install + pre-commit run --all-files + displayName: 'Run pre-commits' + + - script: | + python -m pip install --upgrade pip==23.0.1 + pip install wheel==0.38.4 --upgrade + pip install setuptools==66 --upgrade + pip install build==0.10.0 + pip install pytest-azurepipelines + displayName: 'Install build dependencies' + + - script: | + python -m build + displayName: 'Build PrimAITE' + + - script: | + PRIMAITE_WHEEL=$(ls ./dist/primaite*.whl) + python -m pip install $PRIMAITE_WHEEL[dev] + displayName: 'Install PrimAITE' + condition: or(eq( variables['Agent.OS'], 'Linux' ), eq( variables['Agent.OS'], 'Darwin' )) + + - script: | + forfiles /p dist\ /m *.whl /c "cmd /c python -m pip install @file[dev]" + displayName: 'Install PrimAITE' + condition: eq( variables['Agent.OS'], 'Windows_NT' ) + + - script: | + primaite setup + displayName: 'Perform PrimAITE Setup' + + - script: | + pytest -n 4 + displayName: 'Run tests' diff --git a/.azuredevops/pull_request_template.md b/.azuredevops/pull_request_template.md new file mode 100644 index 00000000..5ff03e18 --- /dev/null +++ b/.azuredevops/pull_request_template.md @@ -0,0 +1,12 @@ +## Summary +*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?* + +## Test process +*How have you tested this (if applicable)?* + +## Checklist +- [ ] This PR is linked to a **work item** +- [ ] I have performed **self-review** of the code +- [ ] I have written **tests** for any new functionality added with this PR +- [ ] I have updated the **documentation** if this PR changes or adds functionality +- [ ] I have run **pre-commit** checks for code style From 472e85cffb17762342ce49d4f6608adabc2831a2 Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 21:49:36 +0100 Subject: [PATCH 8/9] Added additional install instructions to the README.md --- README.md | 76 ++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 61 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index d078829e..53d58509 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,72 @@ # PrimAITE +PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. + ## Getting Started with PrimAITE - -### Pre-Requisites - -In order to get **PrimAITE** installed, you will need to have the following installed: - -- `python3.8+` -- `python3-pip` -- `virtualenv` - +### 💫 Install & Run **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS. +Currently, the PRimAITE wheel can only be installed from GitHub. This may change in the future with release to PyPi. -### Installation from source (Developer Install) -#### 1. Create a new python virtual environment (venv) +#### Windows (PowerShell) + +**Prerequisites:** +* Manual install of Python >= 3.8 < 3.11 + +**Install:** + +``` powershell +mkdir ~\primaite +cd ~\primaite +python3 -m venv .venv +attrib +h .venv /s /d # Hides the .venv directory +.\.venv\Scripts\activate +pip install https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/releases/download/v2.0.0/primaite-2.0.0-py3-none-any.whl +primaite setup +``` + +#### Unix + +**Prerequisites:** +* Manual install of Python >= 3.8 < 3.11 + +``` bash +sudo add-apt-repository ppa:deadsnakes/ppa +sudo apt install python3.10 +sudo apt-get install python3-pip +sudo apt-get install python3-venv +``` +**Install:** + +``` bash +mkdir ~/primaite +cd ~/primaite +python3 -m venv .venv +source .venv/bin/activate +pip install https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/releases/download/v2.0.0/primaite-2.0.0-py3-none-any.whl +primaite setup +``` + + +### Developer Install from Source +To make your own changes to PrimAITE, perform the install from source (developer install) + +#### 1. Clone the PrimAITE repository +``` unix +git clone git@github.com:Autonomous-Resilient-Cyber-Defence/PrimAITE.git +``` + +#### 2. CD into the repo directory +``` unix +cd PrimAITE +``` +#### 3. Create a new python virtual environment (venv) ```unix python3 -m venv venv ``` -#### 2. Activate the venv +#### 4. Activate the venv ##### Unix ```bash @@ -32,19 +78,19 @@ source venv/bin/activate .\venv\Scripts\activate ``` -#### 3. Install `primaite` with the dev extra into the venv along with all of it's dependencies +#### 5. Install `primaite` with the dev extra into the venv along with all of it's dependencies ```bash python3 -m pip install -e .[dev] ``` -#### 4. Perform the PrimAITE setup: +#### 6. Perform the PrimAITE setup: ```bash primaite setup ``` -## Building documentation +## 📚 Building documentation The PrimAITE documentation can be built with the following commands: ##### Unix From c5a23fa035f7a04c551ab78178c837e26402aabc Mon Sep 17 00:00:00 2001 From: Chris McCarthy Date: Wed, 26 Jul 2023 22:10:59 +0100 Subject: [PATCH 9/9] Added run section with primaite session command in the README.md --- README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.md b/README.md index 53d58509..326cc27e 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,12 @@ pip install https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/relea primaite setup ``` +**Run:** + +``` bash +primaite session +``` + #### Unix **Prerequisites:** @@ -47,6 +53,12 @@ pip install https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/relea primaite setup ``` +**Run:** + +``` bash +primaite session +``` + ### Developer Install from Source To make your own changes to PrimAITE, perform the install from source (developer install)