## Summary
Re-run the benchmarks for v2.0.0rc1 and v2.0.0rc2 using the same config file. As expected, the versions perform almost identically as there's no real logic changes that would affect the agent between the two release candidates
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1574
## Summary
- Turned on the test. Also updated some references to the old primaite paths. Finally, pushed the deployment status classifier to Development Status :: 5 - Production/Stable
## Test process
Yes, turned on the test.
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1650 - Turned on the test. Also updated some references to the old primaite paths. Finally, pushed the deployment status classifier to Development Status :: 5 - Production/Stable
Related work items: #1650
## Summary
- Added _PrimaitePaths class that manages all the primaite locations using PlayformDirs. This class now creates new primaite locations for each version of primaite.
- Rolled the _PrimaitePaths class out throughout the code base.
- Updated the docs to reference the new version paths.
- Updated the author from qinetiq to dstl
- Bumped version number to 2.0.0rc2
## Test process
- Manual checks. Tough to test the install paths.
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
Related work items: #1647
- Rolled the _PrimaitePaths class out throughout the code base.
- Updated the docs to reference the new version paths.
- Updated the author from qinetiq to dstl
- Bumped version number to 2.0.0rc2
## Summary
- Fixed the bug where session gets run twice when loading a session via CLI
- Added a test for the CLI run - xskipped while the bugfix for load session acting odd is tbd
- Fixed a minor bug in PrimAITE session where session_path is overwritten
## Test process
Added a new test for CLI, but xskipped while a different bug is tbd
Ran it locally and no longer runs another session after the loaded session
```
(venv) PS D:\Projects\ARCD\PrimAITE\PrimAITE> primaite session --load [REDACTED for security]\primaite\sessions\2023-07-20\2023-07-20_15-01-11
2023-07-20 15:04:21,320: Using: AgentFramework.SB3, AgentIdentifier.PPO, ActionType.NODE, observation_space=NODE_LINK_TABLE, Training: 5 episodes @ 256 stepsEvaluation: 5 episodes @ 256 steps
2023-07-20 15:04:21,335: Environment configuration loaded
Environment configuration loaded
2023-07-20 15:04:21,775: Welcome to the Primary-level AI Training Environment (PrimAITE) (version: 2.0.0rc1)
2023-07-20 15:04:21,775: The output directory for this session is: C:\Users\czar.echavez\primaite\sessions\2023-07-20\2023-07-20_15-04-21
2023-07-20 15:04:21,779: Beginning learning for 10 episodes @ 256 time steps...
2023-07-20 15:04:22,379: Episode: 1, Average Reward: -0.0020839843750000003
2023-07-20 15:04:23,137: Episode: 2, Average Reward: -0.0021933593750000004
2023-07-20 15:04:23,831: Episode: 3, Average Reward: -0.0022617187500000003
2023-07-20 15:04:24,486: Episode: 4, Average Reward: -0.002373046874999999
2023-07-20 15:04:25,125: Episode: 5, Average Reward: -0.0018066406250000014
2023-07-20 15:04:25,791: Episode: 6, Average Reward: -0.0017597656250000013
2023-07-20 15:04:26,415: Episode: 7, Average Reward: -0.0018437500000000014
2023-07-20 15:04:27,053: Episode: 8, Average Reward: -0.0019101562500000015
2023-07-20 15:04:27,715: Episode: 9, Average Reward: -0.0016777343750000013
2023-07-20 15:04:28,359: Episode: 10, Average Reward: -0.0015976562500000012
2023-07-20 15:04:28,550: Finished learning
2023-07-20 15:04:30,851: Beginning deterministic evaluation for 5 episodes @ 256 time steps...
2023-07-20 15:04:31,243: Episode: 1, Average Reward: -0.0018515625000000014
2023-07-20 15:04:31,663: Episode: 2, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,112: Episode: 3, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,505: Episode: 4, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,904: Episode: 5, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,998: Finished evaluation
```
Also fixed the xskipped tests, since the double running seems to have caused the issue of rewards not matching.
Added a test that runs the PrimAITE in CLI
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
#1595:
- Fixed the...
## Summary
Managed to get the evaluation of rllib agents working. A test has been added to test_primaite_session.py that now tests the full RLlib agent from end-to-end. I've also updated the tests in here to check that the mean reward per episode plot is created for both too. This will need a bit of a re-design further down the line, but for now, it works. Added a custom exception for RLlib eval only error.
Is this a hack? Yes. Does it work? Yes. we'll make this better later.
## Test process
Both a SB3 and Ray RLlib agent is tested now in the test_primaite_session.py module.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1594
## Summary
Added changelog and backfilled with v1.1.0 and v1.1.0 release notes.
Check what's written against notes in: https://nscuk.sharepoint.com/:f:/r/sites/SSE32-ARCDTrainingEnvironments/Shared%20Documents/General/01%20PrimAITE/01.01%20Releases/Release%20Notes?csf=1&web=1&e=uwPsyM
## Test process
N/A
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1639 - Added CHANGELOG.md and backfilled it with v1.1.0 and v1.1.1 release notes.
Related work items: #1639
## Summary
Changed environment config to training config in config.rst as Chris requested.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
feature/1641:
Changed environment config to training config
Related work items: #1641
- Fixed the bug where session gets run twice when loading a session via CLI
- Added a test for the CLI run - xskipped while the bugfix for load session acting odd is tbd
- Fixed a minor bug in PrimAITE session where session_path is overwritten
## Summary
Added to docs as per @<Christopher McCarthy> changes - added prefixes to command line primaite session and explained primate session default no arguments command.
## Test process
*NA*
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
#1640 - Added --ldc and --tc prefixes and added small note about primaite session default run command
Related work items: #1640
## Summary
Updated the UML diagram using puml to help render a better output of the diagram.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [na] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
feature/1637:
Updating the UML diagram using puml instead of mmd.
Related work items: #1637
## Summary
- Added full benchmarking script that included plots and a LaTeX report. Ran the v2.0.0rc1 benchmark. Tidied a few other things up.
The code is a bit scrappy. But it's not released code. I will endeavour to tidy it up at a later date.
## Test process
Manually ran the script. This is the final report -> [PrimAITE v2.0.0rc1 Learning Benchmark.pdf](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/126/attachments/PrimAITE%20v2.0.0rc1%20Learning%20Benchmark.pdf)
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1632
## Summary
Add license file and reference it in the pyproject.toml
## Test process
Running `pip-licenses` shows:
```
pre-commit 2.20.0 MIT License
primaite 2.0.0rc1 MIT
primaite 2.0.0rc1 MIT
prometheus-client 0.17.1 Apache Software License
prompt-toolkit 3.0.39 BSD License
```
## Checklist
- [x This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1638
## Summary
- Updated the session outputs details in primaite_session.rst
- Fixed Logger typehint bugs
- Fixed typing issues in access_control_list.py
## Test process
Build the docs
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1635 - Updated the session outputs details in primaite_session.rst
Related work items: #1635
## Summary
### ACL List
First change was I changed `access_control_list.py` from a `dict` to a `list` so it is now an ordered structure. This was done so I could implement the positions inside the `ACL` and `ANY` action spaces.
From this, some functions have changed such as `add_rule` and `remove_rule`, `is_blocked` and `get_relevant_rules`.
The ACL list is now a fixed size and on initialisation it is filled with `None` types. When a function calls `self.acl` the `implicit rule` (if there is one) is added after the last `ACLRule` object in the list. The remainder of the list (if there is left over space) is padded out with `None`.
As the agent adds rules, the `None` are replaced by `ACLRule` objects and the agent cannot overwrite an existing `ACLRule` with another, it can only write over `None` types.
### ACL Training Config Changes
Changes have been made to the `training_config_main.yaml`. There are 2 new items:
`implicit_acl_rule:` - Implicit ACL firewall rule at end of list to be default action (ALLOW or DENY)
`max_number_acl_rules:` - Total number of ACL rules allowed in the environment
In the `OBSERVATION_SPACE` area of the config, `ACCESS_CONTROL_LIST` can be selected
They have default values if none are specified so for the older configs - these values are in the `TrainingConfig` dataclass.
### ACL and ANY Action Spaces
I changed the ACL space from length of 6 to 7. I have included the `position` of where the agent wants to position the ACL Rule.
`position` = index in `self.acl` with bounds [0 to ...]
As a result, total possible actions have gone up.
### ACL Observation Space
In the observations.py I have made a new observation component: Access Control List.
It has the following mappings/meanings:
[0, 1, 2] - Permission (0 = NA, 1 = DENY, 2 = ALLOW)
[0, num nodes] - Source IP (0 = NA, 1 = any, then 2 -> x resolving to Node IDs)
[0, num nodes] - Dest IP (0 = NA, 1 = any, then 2 -> x resolving to Node IDs)
[0, num services] - Protocol (0 = NA, 1 = any, then 2 -> x resolving to protocol)
[0, num ports] - Port (0 = NA, 1 = any, then 2 -> x resolving to port)
[0, max acl rules - 1] - Position (0 = NA, 1 = first index, then 2 -> x index resolving to acl rule in acl list)
I created a new 0 meaning, which means NA and represents the None objects in the ACLList.
Also, there is no 'flatten' in the observation space components and this has been done in the observations.py now if there are multiple components.
## Test process
I have written tests in a new `TestAccessControlList` object in `test_observations.py`.
I ran a single test which was 1000 episodes, SB3/PPO, Config 5 and ACL Observation Space. I seemed to get some interesting results which may need investigating on Monday.
## Checklist
- ...
