Commit Graph

1810 Commits

Author SHA1 Message Date
Czar Echavez
12a6aa5e7f #2620: make the windows script work 2024-07-02 01:31:59 +01:00
Czar Echavez
88ab3c3ca1 #2620: remove shell 2024-07-02 00:45:24 +01:00
Czar Echavez
1faacc8d7c #2620: fix indentation 2024-07-02 00:28:59 +01:00
Czar Echavez
918eba2217 #2620: fix indentation 2024-07-02 00:25:30 +01:00
Czar Echavez
8097884ae2 #2620: modify script 2024-07-02 00:00:20 +01:00
Czar Echavez
ab3e84b8b9 #2620: remove irrelevant change 2024-07-01 17:03:14 +01:00
Czar Echavez
afcb844501 #2620: remove cell that fails on purpose 2024-07-01 16:45:28 +01:00
Czar Echavez
a47a14b86e #2620: Going around azure dev ops to fail the script 2024-07-01 15:57:15 +01:00
Czar Echavez
4dd50be11a #2620: run notebooks after test so that the results can be published 2024-07-01 14:31:58 +01:00
Czar Echavez
cb61756e43 #2620: attempting to fail pipeline if notebook fails 2024-07-01 12:21:05 +01:00
Czar Echavez
ee4e152f13 #2620: publish result of test and checking if pipeline fails 2024-07-01 12:00:17 +01:00
Czar Echavez
ce58f3960c #2620: downgrade pytest-xdist version 2024-06-28 16:43:10 +01:00
Czar Echavez
a4424608dd #2620: add nbmake and pytest xdist to run the notebooks as part of pieline 2024-06-28 16:33:24 +01:00
Christopher McCarthy
e7fc01781e Merged PR 437: Automate Benchmarking Process with Scheduled and Release Triggers
## Summary

This pull request introduces an Azure DevOps pipeline that automates the benchmarking process for our project. The pipeline is designed to run under specific conditions and ensures that the benchmarking results are committed and published as artifacts. Additionally, the pipeline generates detailed benchmark reports in markdown format.

**Key Features:**

1.  **Triggering Conditions:**

    -   Triggered on creation of branches matching `refs/heads/release/*`.
    -   Runs automatically at 2 AM every weekday for the `dev` branch.
2.  **Version Handling:**

    -   Updates the version number with a `+dev.YYYYMMDD` suffix for the `dev` branch.
3.  **Dependency Installation:**

    -   Sets up Python 3.11 and installs project dependencies, including development and reinforcement learning extras.
4.  **Benchmark Execution:**

    -   Runs the benchmarking script and stores results in a version-based directory structure.
    -   Generates detailed markdown reports for benchmark results.
5.  **Commit and Push Results:**

    -   Commits and pushes benchmark results for release branches.
6.  **Artifact Preparation and Publishing:**

    -   Packages benchmark results into a tarball and publishes it as a pipeline artifact.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2648
2024-06-27 12:58:33 +00:00
Chris McCarthy
7f0a0c562f #2648 - changed av_reward_per_episode to total_reward_per_episode in primaite_benchmark.py and report.py 2024-06-26 13:26:18 +01:00
Chris McCarthy
e7f979b78e #2648 - reverted temp changes to benchmark durations and branch names for testing purposes 2024-06-26 12:52:29 +01:00
Chris McCarthy
795b5a80fb #2648 - testing the new benchmark artifact name 2024-06-26 12:00:56 +01:00
Chris McCarthy
20c9719f1e #2648 - added full branch name reference for push 2024-06-26 11:32:17 +01:00
Chris McCarthy
3b118fa0ad #2648 - now testing the release benchmark auto commit and push 2024-06-26 10:52:23 +01:00
Chris McCarthy
08ca4d8889 #2648 - now testing the release benchmark auto commit and push 2024-06-26 10:50:05 +01:00
Chris McCarthy
112f116a89 #2648 - fixed error whereby VERSION variable isn't set before setting the dev build version in the VERSION file 2024-06-26 09:46:05 +01:00
Chris McCarthy
2192516c9a #2648 - reordered steps so that dev version is set first before version variables are set 2024-06-26 09:40:16 +01:00
Chris McCarthy
ad2b132a10 #2648 - added dev and rl extras to the pip install step 2024-06-26 09:13:55 +01:00
Chris McCarthy
0ee243a242 #2648 - trying to fix the artifacts publish stage. currently creating tar.gz and publishing that 2024-06-26 09:09:16 +01:00
Chris McCarthy
4249314672 #2648 - reduced the benchmark sessions and episodes for fail-fast speed while testing 2024-06-25 17:28:05 +01:00
Chris McCarthy
a7731c94e3 Merge remote-tracking branch 'origin/feature/2648_Automate-the-benchmarking-process' into feature/2648_Automate-the-benchmarking-process 2024-06-25 17:25:23 +01:00
Chris McCarthy
55d69d6568 #2648 - updated benchmark run command 2024-06-25 17:23:04 +01:00
Christopher McCarthy
1033e696cd Set up CI with Azure Pipelines
[skip ci]
2024-06-25 16:01:18 +00:00
Chris McCarthy
824729276e #2648 - updated benchmark process to output markdown file instead of LaTeX. Added pipeline that runs benchmarking at 2am on a weekday and automatically upon creation of release branch 2024-06-25 16:58:39 +01:00
Marek Wolan
bf5f443604 Merged PR 433: Bump version to 3.1 2024-06-25 12:05:13 +00:00
Marek Wolan
e915d5e090 Updated VERSION 2024-06-25 10:44:15 +00:00
Marek Wolan
9b195ea0f4 Merged PR 431: #2682 Backport changes to core PrimAITE
Related work items: #2682
2024-06-25 10:32:10 +00:00
Marek Wolan
a861ccb1ee Fix typo in log message 2024-06-25 10:31:28 +00:00
Marek Wolan
28dabad66b #2682 Backport changes to core PrimAITE 2024-06-25 11:04:52 +01:00
Marek Wolan
4a81dc3b2c Merged PR 424: fix infini-ARP bug when pinging broadcast/network address
Related work items: #2661
2024-06-19 11:09:52 +00:00
Chris McCarthy
636ec0cdd4 #26661 - Added tests for testing arp doesn't work but also doesn't crash the network if attempting to resolve a broadcast or network address 2024-06-19 11:31:13 +01:00
Marek Wolan
7375bb383a #2661 Improve style on ARP check code to be more concise 2024-06-18 10:09:06 +01:00
Marek Wolan
c90c059411 fix infini-ARP bug when pinging broadcast/network address 2024-06-14 14:48:28 +00:00
Charlie Crane
923d2629ca Merged PR 418: Update Pre-Commit to check for valid copyright
## Summary
Updates Pre-commit with the a check to ensure valid copyright headers

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [X] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2646
2024-06-13 18:54:48 +00:00
Czar Echavez
4ac352bf3b Merged PR 419: integrate NIC traffic into observation space
## Summary
- Integrates the NIC traffic into the observation space
- Added `monitored_traffic` property into agent config which specifies which traffic is going to be included in observation

## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/419?_a=files&path=/tests/integration_tests/game_layer/observations/test_nic_observations.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2658
2024-06-13 17:53:43 +00:00
Czar Echavez
101fa3ebdb #2658: change key of observation to match what is in CAOS document 2024-06-13 16:29:02 +01:00
Marek Wolan
823e7d81a6 Merged PR 385: #2502 add more info about config placeholders
## Summary
Add to the config placeholder notebook. Now we also explain how to vary the simulation between episodes.

## Test process
precommit pass. notebook runs.

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2502 add more info about config placeholders

Related work items: #2502
2024-06-13 15:19:31 +00:00
Czar Echavez
65d9cfa7dc #2658: I cast uncast on NIC space 2024-06-13 15:11:34 +01:00
Charlie Crane
d095138ef7 Merged PR 413: One line change to try and resolve build docs pipeline failure
## Summary
The build-docs pipeline has been failing to run. This PR corrects it.

## Test process
Link to successfully running:
https://dev.azure.com/ma-dev-uk/PrimAITE/_build/results?buildId=4849&view=results

Previous failures can be seen in the pipeline history.

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

One line change to try and resolve build docs pipeline failure

Related work items: #2659
2024-06-13 13:11:03 +00:00
Charlie Crane
3c8e1d1035 #2646 - Amending a merge conflict resolution in preparation for review 2024-06-13 13:25:29 +01:00
Marek Wolan
d22117a614 Merge remote-tracking branch 'origin/dev' into bugfix/fix_pipeline_bug 2024-06-13 12:53:33 +01:00
Marek Wolan
0bd52b1c15 Merge remote-tracking branch 'origin/dev' into feature/2502-placeholders-docs 2024-06-13 12:52:59 +01:00
Marek Wolan
fad8d936e6 Merge remote-tracking branch 'origin/dev' into feature/2646_Update-pre-commit-to-check-for-valid-copyright 2024-06-13 12:52:09 +01:00
Czar Echavez
1798674d39 #2658: fix space 2024-06-13 12:37:20 +01:00
Charlie Crane
6762fd805e #2646 - Adding back the dns client/server python that was deleted 2024-06-13 12:13:16 +01:00