Merged PR 452: Automate the benchmark process
## Summary This pull request integrates full automation of the PrimAITE benchmarking process into our Azure DevOps pipeline, enhancing consistency and minimizing manual effort. #### Key Changes: 1. **Pipeline Automation**: Configured to trigger on commits to `dev` and `release` branches, with scheduled runs during off-peak hours. 2. **Error Handling Improvements**: The pipeline is set to fail on any Python execution errors, ensuring immediate identification of issues. 3. **Artifact Publishing**: Benchmark results are now automatically compressed and published as artifacts, facilitating easy access and analysis. ## Test process Tested the pipeline runs from end-to-end and produces a working benchmark report: https://dev.azure.com/ma-dev-uk/PrimAITE/_build/results?buildId=5280&view=results ## Checklist - [ ] PR is linked to a **work item** - [ ] **acceptance criteria** of linked ticket are met - [ ] performed **self-review** of the code - [ ] written **tests** for any new functionality added with this PR - [ ] updated the **documentation** if this PR changes or adds functionality - [ ] written/updated **design docs** if this PR implements new functionality - [ ] updated the **change log** - [ ] ran **pre-commit** checks for code style - [ ] attended to any **TO-DOs** left in the code Related work items: #2739
This commit is contained in:
@@ -11,14 +11,17 @@ schedules:
|
||||
branches:
|
||||
include:
|
||||
- 'refs/heads/dev'
|
||||
|
||||
pool:
|
||||
vmImage: ubuntu-latest
|
||||
|
||||
variables:
|
||||
VERSION: ''
|
||||
MAJOR_VERSION: ''
|
||||
|
||||
jobs:
|
||||
- job: PrimAITE_Benchmark
|
||||
timeoutInMinutes: 360 # 6-hour maximum
|
||||
pool:
|
||||
vmImage: ubuntu-latest
|
||||
workspace:
|
||||
clean: all
|
||||
steps:
|
||||
- checkout: self
|
||||
persistCredentials: true
|
||||
@@ -50,6 +53,7 @@ steps:
|
||||
displayName: 'Install Dependencies'
|
||||
|
||||
- script: |
|
||||
set -e
|
||||
cd benchmark
|
||||
python3 primaite_benchmark.py
|
||||
cd ..
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
"av_s_per_session": 3205.6340542,
|
||||
"av_s_per_step": 0.10017606419375,
|
||||
"av_s_per_100_steps_10_nodes": 10.017606419375,
|
||||
"combined_av_reward_per_episode": {
|
||||
"combined_total_reward_per_episode": {
|
||||
"1": -53.42999999999999,
|
||||
"2": -25.18000000000001,
|
||||
"3": -42.00000000000002,
|
||||
|
||||
Reference in New Issue
Block a user