Merged PR 440: #2620: add nbmake and pytest xdist to run the notebooks as part of pieline

![image.png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/440/attachments/image.png)
![image (2).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/440/attachments/image%20%282%29.png)

## Summary
Added nbmake so that the pytest can be used to run the notebooks in primaite as part of the pipeline

## Test process
n/a

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2620: add nbmake and pytest xdist to run the notebooks as part of pieline

Related work items: #2620

.azure/azure-ci-build-pipeline.yaml

@@ -107,11 +107,39 @@ stages:
               coverage html -d htmlcov -i
             displayName: 'Run tests and code coverage'
 
+          # Run the notebooks
+          - script: |
+              pytest --nbmake -n=auto src/primaite/notebooks --junit-xml=./notebook-tests/notebooks.xml
+              notebooks_exit_code=$?
+              pytest --nbmake -n=auto src/primaite/simulator/_package_data --junit-xml=./notebook-tests/package-notebooks.xml
+              package_notebooks_exit_code=$?
+              # Fail step if either of these do not have exit code 0
+              if [ $notebooks_exit_code -ne 0 ] || [ $package_notebooks_exit_code -ne 0 ]; then
+                exit 1
+              fi
+            displayName: 'Run notebooks on Linux and macOS'
+            condition: or(eq(variables['Agent.OS'], 'Linux'), eq(variables['Agent.OS'], 'Darwin'))
+
+          # Run notebooks
+          - script: |
+              pytest --nbmake -n=auto src/primaite/notebooks --junit-xml=./notebook-tests/notebooks.xml
+              set notebooks_exit_code=%ERRORLEVEL%
+              pytest --nbmake -n=auto src/primaite/simulator/_package_data --junit-xml=./notebook-tests/package-notebooks.xml
+              set package_notebooks_exit_code=%ERRORLEVEL%
+              rem Fail step if either of these do not have exit code 0
+              if %notebooks_exit_code% NEQ 0 exit /b 1
+              if %package_notebooks_exit_code% NEQ 0 exit /b 1
+            displayName: 'Run notebooks on Windows'
+            condition: eq(variables['Agent.OS'], 'Windows_NT')
+
           - task: PublishTestResults@2
             condition: succeededOrFailed()
+            displayName: 'Publish Test Results'
             inputs:
               testRunner: JUnit
-              testResultsFiles: 'junit/**.xml'
+              testResultsFiles: |
+                'junit/**.xml'
+                'notebook-tests/**.xml'
               testRunTitle: 'Publish test results'
               failTaskOnFailedTests: true
 

.gitignore

@@ -54,6 +54,7 @@ cover/
 tests/assets/**/*.png
 tests/assets/**/tensorboard_logs/
 tests/assets/**/checkpoints/
+notebook-tests/*.xml
 
 # Translations
 *.mo

pyproject.toml

@@ -72,7 +72,9 @@ dev = [
     "Sphinx==7.1.2",
     "sphinx-copybutton==0.5.2",
     "wheel==0.38.4",
-    "nbsphinx==0.9.4"
+    "nbsphinx==0.9.4",
+    "nbmake==1.5.4",
+    "pytest-xdist==3.3.1"
 ]
 
 [project.scripts]

src/primaite/notebooks/multi-processing.ipynb

@@ -143,7 +143,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.10.11"
   }
  },
  "nbformat": 4,