## Summary
Re-run the benchmarks for v2.0.0rc1 and v2.0.0rc2 using the same config file. As expected, the versions perform almost identically as there's no real logic changes that would affect the agent between the two release candidates
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1574
## Summary
- Turned on the test. Also updated some references to the old primaite paths. Finally, pushed the deployment status classifier to Development Status :: 5 - Production/Stable
## Test process
Yes, turned on the test.
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1650 - Turned on the test. Also updated some references to the old primaite paths. Finally, pushed the deployment status classifier to Development Status :: 5 - Production/Stable
Related work items: #1650
## Summary
- Added _PrimaitePaths class that manages all the primaite locations using PlayformDirs. This class now creates new primaite locations for each version of primaite.
- Rolled the _PrimaitePaths class out throughout the code base.
- Updated the docs to reference the new version paths.
- Updated the author from qinetiq to dstl
- Bumped version number to 2.0.0rc2
## Test process
- Manual checks. Tough to test the install paths.
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
Related work items: #1647
- Rolled the _PrimaitePaths class out throughout the code base.
- Updated the docs to reference the new version paths.
- Updated the author from qinetiq to dstl
- Bumped version number to 2.0.0rc2
## Summary
- Fixed the bug where session gets run twice when loading a session via CLI
- Added a test for the CLI run - xskipped while the bugfix for load session acting odd is tbd
- Fixed a minor bug in PrimAITE session where session_path is overwritten
## Test process
Added a new test for CLI, but xskipped while a different bug is tbd
Ran it locally and no longer runs another session after the loaded session
```
(venv) PS D:\Projects\ARCD\PrimAITE\PrimAITE> primaite session --load [REDACTED for security]\primaite\sessions\2023-07-20\2023-07-20_15-01-11
2023-07-20 15:04:21,320: Using: AgentFramework.SB3, AgentIdentifier.PPO, ActionType.NODE, observation_space=NODE_LINK_TABLE, Training: 5 episodes @ 256 stepsEvaluation: 5 episodes @ 256 steps
2023-07-20 15:04:21,335: Environment configuration loaded
Environment configuration loaded
2023-07-20 15:04:21,775: Welcome to the Primary-level AI Training Environment (PrimAITE) (version: 2.0.0rc1)
2023-07-20 15:04:21,775: The output directory for this session is: C:\Users\czar.echavez\primaite\sessions\2023-07-20\2023-07-20_15-04-21
2023-07-20 15:04:21,779: Beginning learning for 10 episodes @ 256 time steps...
2023-07-20 15:04:22,379: Episode: 1, Average Reward: -0.0020839843750000003
2023-07-20 15:04:23,137: Episode: 2, Average Reward: -0.0021933593750000004
2023-07-20 15:04:23,831: Episode: 3, Average Reward: -0.0022617187500000003
2023-07-20 15:04:24,486: Episode: 4, Average Reward: -0.002373046874999999
2023-07-20 15:04:25,125: Episode: 5, Average Reward: -0.0018066406250000014
2023-07-20 15:04:25,791: Episode: 6, Average Reward: -0.0017597656250000013
2023-07-20 15:04:26,415: Episode: 7, Average Reward: -0.0018437500000000014
2023-07-20 15:04:27,053: Episode: 8, Average Reward: -0.0019101562500000015
2023-07-20 15:04:27,715: Episode: 9, Average Reward: -0.0016777343750000013
2023-07-20 15:04:28,359: Episode: 10, Average Reward: -0.0015976562500000012
2023-07-20 15:04:28,550: Finished learning
2023-07-20 15:04:30,851: Beginning deterministic evaluation for 5 episodes @ 256 time steps...
2023-07-20 15:04:31,243: Episode: 1, Average Reward: -0.0018515625000000014
2023-07-20 15:04:31,663: Episode: 2, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,112: Episode: 3, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,505: Episode: 4, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,904: Episode: 5, Average Reward: -0.0018515625000000014
2023-07-20 15:04:32,998: Finished evaluation
```
Also fixed the xskipped tests, since the double running seems to have caused the issue of rewards not matching.
Added a test that runs the PrimAITE in CLI
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
#1595:
- Fixed the...
## Summary
Managed to get the evaluation of rllib agents working. A test has been added to test_primaite_session.py that now tests the full RLlib agent from end-to-end. I've also updated the tests in here to check that the mean reward per episode plot is created for both too. This will need a bit of a re-design further down the line, but for now, it works. Added a custom exception for RLlib eval only error.
Is this a hack? Yes. Does it work? Yes. we'll make this better later.
## Test process
Both a SB3 and Ray RLlib agent is tested now in the test_primaite_session.py module.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1594
## Summary
Added changelog and backfilled with v1.1.0 and v1.1.0 release notes.
Check what's written against notes in: https://nscuk.sharepoint.com/:f:/r/sites/SSE32-ARCDTrainingEnvironments/Shared%20Documents/General/01%20PrimAITE/01.01%20Releases/Release%20Notes?csf=1&web=1&e=uwPsyM
## Test process
N/A
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1639 - Added CHANGELOG.md and backfilled it with v1.1.0 and v1.1.1 release notes.
Related work items: #1639
## Summary
Changed environment config to training config in config.rst as Chris requested.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
feature/1641:
Changed environment config to training config
Related work items: #1641
- Fixed the bug where session gets run twice when loading a session via CLI
- Added a test for the CLI run - xskipped while the bugfix for load session acting odd is tbd
- Fixed a minor bug in PrimAITE session where session_path is overwritten
## Summary
Added to docs as per @<Christopher McCarthy> changes - added prefixes to command line primaite session and explained primate session default no arguments command.
## Test process
*NA*
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
#1640 - Added --ldc and --tc prefixes and added small note about primaite session default run command
Related work items: #1640
## Summary
Updated the UML diagram using puml to help render a better output of the diagram.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [na] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
feature/1637:
Updating the UML diagram using puml instead of mmd.
Related work items: #1637
## Summary
- Added full benchmarking script that included plots and a LaTeX report. Ran the v2.0.0rc1 benchmark. Tidied a few other things up.
The code is a bit scrappy. But it's not released code. I will endeavour to tidy it up at a later date.
## Test process
Manually ran the script. This is the final report -> [PrimAITE v2.0.0rc1 Learning Benchmark.pdf](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/126/attachments/PrimAITE%20v2.0.0rc1%20Learning%20Benchmark.pdf)
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1632
## Summary
Add license file and reference it in the pyproject.toml
## Test process
Running `pip-licenses` shows:
```
pre-commit 2.20.0 MIT License
primaite 2.0.0rc1 MIT
primaite 2.0.0rc1 MIT
prometheus-client 0.17.1 Apache Software License
prompt-toolkit 3.0.39 BSD License
```
## Checklist
- [x This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1638
## Summary
- Updated the session outputs details in primaite_session.rst
- Fixed Logger typehint bugs
- Fixed typing issues in access_control_list.py
## Test process
Build the docs
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
#1635 - Updated the session outputs details in primaite_session.rst
Related work items: #1635
## Summary
### ACL List
First change was I changed `access_control_list.py` from a `dict` to a `list` so it is now an ordered structure. This was done so I could implement the positions inside the `ACL` and `ANY` action spaces.
From this, some functions have changed such as `add_rule` and `remove_rule`, `is_blocked` and `get_relevant_rules`.
The ACL list is now a fixed size and on initialisation it is filled with `None` types. When a function calls `self.acl` the `implicit rule` (if there is one) is added after the last `ACLRule` object in the list. The remainder of the list (if there is left over space) is padded out with `None`.
As the agent adds rules, the `None` are replaced by `ACLRule` objects and the agent cannot overwrite an existing `ACLRule` with another, it can only write over `None` types.
### ACL Training Config Changes
Changes have been made to the `training_config_main.yaml`. There are 2 new items:
`implicit_acl_rule:` - Implicit ACL firewall rule at end of list to be default action (ALLOW or DENY)
`max_number_acl_rules:` - Total number of ACL rules allowed in the environment
In the `OBSERVATION_SPACE` area of the config, `ACCESS_CONTROL_LIST` can be selected
They have default values if none are specified so for the older configs - these values are in the `TrainingConfig` dataclass.
### ACL and ANY Action Spaces
I changed the ACL space from length of 6 to 7. I have included the `position` of where the agent wants to position the ACL Rule.
`position` = index in `self.acl` with bounds [0 to ...]
As a result, total possible actions have gone up.
### ACL Observation Space
In the observations.py I have made a new observation component: Access Control List.
It has the following mappings/meanings:
[0, 1, 2] - Permission (0 = NA, 1 = DENY, 2 = ALLOW)
[0, num nodes] - Source IP (0 = NA, 1 = any, then 2 -> x resolving to Node IDs)
[0, num nodes] - Dest IP (0 = NA, 1 = any, then 2 -> x resolving to Node IDs)
[0, num services] - Protocol (0 = NA, 1 = any, then 2 -> x resolving to protocol)
[0, num ports] - Port (0 = NA, 1 = any, then 2 -> x resolving to port)
[0, max acl rules - 1] - Position (0 = NA, 1 = first index, then 2 -> x index resolving to acl rule in acl list)
I created a new 0 meaning, which means NA and represents the None objects in the ACLList.
Also, there is no 'flatten' in the observation space components and this has been done in the observations.py now if there are multiple components.
## Test process
I have written tests in a new `TestAccessControlList` object in `test_observations.py`.
I ran a single test which was 1000 episodes, SB3/PPO, Config 5 and ACL Observation Space. I seemed to get some interesting results which may need investigating on Monday.
## Checklist
- ...
## Summary
Added typehints to functions/methods, and class attributes.
## Test process
I used flake8-annotations and mypy to verify completeness and correctness. Mypy did throw up a very large number of errors and many of them point to some potential problems in the codebase. To elaborate, there are some places where there has been confusion as to whether objects should be strings, integers, or enums. Resolving this is out of scope of this PR but I will create more tickets with concrete examples.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1623
## Summary
Added the DEFCON 703 header to all possible files
## Test process
Built docs to confirm that the top-of-the-page comment does not break anything
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
#1631 - Added the DEFCON 703 header to all possible files
Related work items: #1631
## Summary
Quick test that uses RLLIB in a session
## Test process
The learning session completes then we check that the number of rows in both the average reward per episode and all transactions csv files.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
#1629 - Added rllib test
Related work items: #1629
## Summary
- Added a feature which allows a user to load a previous SB3 session
- Added a feature which allows a user to load a previous PrimaiteSession
- Added a feature which allows a user to load a previous session via the CLI: `primaite session --load "<SESSION_PATH>"`
- RLlib is TODO in another ticket #1626
- Parallel tests via the [pytest-xdist](https://pypi.org/project/pytest-xdist/) dependency (MIT licensed)
- Moved hardcoded agent into hardcoded_abc.py
- renamed agent.py to agent_abc.py to clarify it is an abstract base class
- Added documentation to clarify how to use the feature via CLI or using the run function via main.py
## Test process
Created [test_session_loading.py](https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/119?_a=files&path=/tests/test_session_loading.py) which loads a previously run session and then performs a learn and evaluation run on the loaded agent/Primate session.
The test copies the saved session into a temporary folder, which is then set as the test session path. Once the test is done, the temporary folder should then be deleted
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1595
## Summary
Add a Getting started page to the docs file.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1597
- Removed bool apply_implicit_rule
- Set default implicit_rule to EXPLICIT DENY
- Added position to ACLs in laydown configs
- Removed apply_implicit_rule from training configs
- Added check in access_control_list.py which sets implicit permission to NA if boolean is False
- Changed the defaults in training_config.py so that each scenario has an EXPLICIT ALLOW rule as default implicit rule
- Updated the test_seeding_and_deterministic_session.py because of change no2 adds an extra rule to that scenario
- Added comments in access_control_list.py
- Changed obs_shape to max_number_acl_rules from max_number_acl_rules + 1 as index starts from 1
- Commented episode and step print line from test_single_action_space.py
- Added ability to load sessions via PrimaiteSession
- PrimaiteSession loading test
- Added a NotImplemented RLlib loading for now
- Added the ability to load sessions for hardcoded agents
- Moved Session metadata parsing to utils
- Re-added the hard-coded mean rewards per episode values from a rpe-trained agent to the deterministic test in test_seeding_and_deterministic_session.py
- Partially tidies up some tests in test_observation_space.py; Still some work to be done on this at a later date.
- SB3 Agent loading
- rename agent.py -> agent_abc.py
- rename hardcoded.py -> hardcoded_abc.py
- Tests
- Added in test asset that is used to load the SB3 Agent
## Summary
* Update observation space information and standardise formatting of code blocks in that section
* Remove non-ascii quotation characters
* Update custom blue agent page to match new AgentSession classes.
* Introduce glossary
* Provide a first draft of migration guide for 1.2 to 2.0 (probably not comprehensive)
## Test process
Sphinx is able to build the documentation as checked on my local machine
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1602
## Summary
Training configs now have 2 different types of episode and step counts - one for train and one for evaluation.
`num_train_episodes`
`num_train_steps`
`num_eval_episodes`
`num_eval_steps`
## Test process
A test file `test_train_eval_episode_steps.py` has been implemented which runs train and evaluation session on two particular configs.
The train and evaluation sessions have different episodes and step count and the test checks that the output log files have the correct number of `total_steps` and `total_episodes`.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1566, #1589
## Summary
Unfortunately, I had to do away with the nice and neat matrix strategy for builds, because they do not support conditionals. Instead, I manually replicated the behaviour of the matrix but added a conditional to run every platform only when the 'build reason' is PR.
## Test process
*How have you tested this (if applicable)?*
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1603
## Summary
Since we added File System State as a new part of the observation space, some of the assumptions made by imported ADSP code were not met. This is addressed by these changes.
The code no longer crashes, but the hardcoded ACL agent doesn't work very well, it keeps returning action 0 and receives a low reward. Also if there are ACL rules with 'ANY' as a source IP, it crashes the function `get_node_of_ip` within the HardCodedACLAgent._calculate_action_full_view() method.
I'm not sure how much effort we need to spend fixing the hardcoded agents as they don't seem like they were delivered in a finished state.
## Test process
Can confirm the hardcoded agent can run within a primaite session now.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
## note
I would appreciate some input about what we should do with hardcoded agents for release 2.0.0, it may require significant effort to get them working correctly.
Related work items: #1587
## Summary
This minor update adds more detail and links to relevant pages within the API docs.
## Test process
Locally built docs in HTML format to verify all content displays correctly.
Related work items: #1596
## Summary
Fixes some incorrectly formatted documentations, such as in the observation module. Also adds some missing module-level docstrings. Also adds a PrimAITE Favicon to docs.
Removed Primaite-dependencies.rst as it's autogenerated.
## Test process
Purely cosmetic, so functionality not tested. I did render the HTML output to observe that some mistakes have been fixed.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
Related work items: #1572
## Summary
- Added type hints and docstrings to functions imported from ADSP.
- Imported `get_relevant_rules` which was referenced but didn't exist.
- Removed duplicated function definitions in `agents.utils`
## Test process
The changes in this PR are almost exclusively cosmetic. I can confirm that after adding/removing functions, the unit tests passed fine. I was also able to run the Hardcoded node and ACL agents without problems.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [na] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1575
- Fixed all errors that were caused b fixing the above.
- Some tests still fail, these are for SS to fix.
- Dropped the old run_generic stuff from conftest.py
## Summary
- Added the fix from #1535 with minor changes to make sure that the `primaite_env.step()` function can properly parse the action
- added the config deterministic and seed to training config
- added the deterministic and seed to the Training config class, with defaults `False` and `None` respectively
- minor fix to `primaite_env.close()` function so that it now works
## Test process
Added e2e tests for generic, ppo and a2c which evaluates a trained agent twice to make sure that the seeding and deterministic action works
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
#1386: added the ability to set deterministic and seeding RNG when training and evaluating + the fix provided in #1535
Related work items: #1386, #1535
## Summary
* Made RLlib and SB3 agents save at the end of each learning session by default using a common file naming format. Also now agents only checkpoint every n and not on the final episode.
## Test process
*Tests saved agent file in the test_primaite_session test.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #1593
## Summary
As per the discussion this morning, this PR reimplements changes that were made by ADSP to make the default rewards smaller. This also adds type hints rewards as floats.
## Test process
I checked that sessions are able to run and that they report values similar to what we are used to but smaller by a factor of 10000. I did not change the reward values in the integration test configs, and the tests still pass.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #889, #1586
- Added test_seeded_learning test and test_deterministic_evaluation test.
- Passed config values seed and deterministic to ppo agent
- Dropped deterministic override in evaluate functions
- TempPrimaiteSession now writes files to a UUID folder rather than datetime
- Added seed to Ray RLlib agent setup in rllib.py
- Added seed to SB3 agent setup in sb3.py
## Summary
* Brought over the RLlib, hardcoded agents, and simple agents from ADSP 1.1.0. This opened a can of worms... ADSP got their stuff working in notebooks (***_stares at data scientists!_** 😂) but hadn't integrated it into the PrimAITE package or made the other PrimAITE functionality work with it.
* RLlib agents have been fully integrated with the wider PrimAITE package. This was done by:
* The creation of an `AgentSessionABC` and `HardCodedAgentSessionABC` classes.
* `SB3Agent` and `RLlibAgent` classes then inherited from `AgentSessionABC`.
* The ADSP hardcoded agents were integrated into subclasses of `HardCodedAgentSessionABC`.
* The random and dummy agents were also integrated into subclasses of `HardCodedagentSessionABC`.
* A set of session output directories were created and managed by the agent session to enable consistent storage of session outputs in a common format regardless of the agent type.
* The main config was rafactored so that it had
* **agent_framework** - To identify whether SB3, RLlib, or Custom.
* **agent_identifier** - To identify whether PPO, A2C, hardcoded, random, or dummy.
* **deep_learning_framework** - To identify which framework to use for RLlib.
* Transactions have been overhauled to simplify the process. It also means that they're written in real time so they're not lost if the agent crashes.
* Tests completely overhauled to use `PrimaiteSession`, or at least a test subclass, `TempPrimaiteSession`. It's temp because it uses temp directory rather than main primaite session directory, and it cleans up after itself.
* All the crap removed from `main.py` and made it so that it just runs `PrimaiteSession`.
Now this is where I went off on a tangent...
* CLI added to just make my life and everyone else's life easier.
* Primaite app config added to hold things like logging format, levels etc.
* A `primaite.data_viz.session_plots` module added so that the average reward per episode for each session is plotted and saves for each session (this helped while we were testing and bug fixing).
## Test process
* All tests use `TempPrimaiteSession`, which uses `PrimaiteSession`.
* I still need to write a tests that runs the RLlib, hardcoded, and random/dummy agents. I'll do that now while this is being reviewed.
## Still to do
* Update docs. I'm getting this PR up now so we can get it in to make use of the features. I'll get the docs updated today either on this branch or another branch (depending on how long this review takes).
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #917, #1563
## Summary
Just splits the install primaite step into two depending if agent is using windows or not.
## Test process
Ran a build successfully.
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
## Summary
Ported over ADSP changes regarding the randomised red agent.
Red agent currently only works on laydown configs which contain links.
Each episode generates random red agent instructions
## Test process
Written a test that ensures that the random red agent produces random red agent instructions
| Random red agent | Laydown | Agent Identifier | Run 1 | Run 2 | Run 3 |
|------------------|------------------------|------------------|------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
| NONE | Very Basic (Laydown 3) | A2C |  |  |  |
| RANDOM | Very Basic (Laydown 3) | A2C |  |  |  |
| NONE | Very Basic (Laydown 3) | PPO |  |  |  |
| RANDOM | Very Basic (Laydown 3) | PPO ...
## Summary
Changed doc-string of test_reward.py to reflect the new test and what it is trying to do rather than the old outdated one.
## Test process
NA - no logic changes
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
1555 - updated doc-string to make test understanding easier
Related work items: #1555, #1556
- Dropped support for overriding the num_episodes and num_steps at the agent level. It's just not needed and will add complexity when overriding and writing output files.
- Dropped support for Python 3.11 due to not supported on Ray RLlib.
- Made release pipeline only run once as we're now no longer using pure path wheels.
## Summary
The code changes are purely cosmetic- the result of applying pre-commit to all our files. I also added a pre-commit step to the build pipeline to reject non-conforming PRs
## Test process
I saw that the build pipeline passes with this new step.
## Checklist
- [ ] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #1557
## Summary
Logic error with negation of booleans.
## Test process
Run with debug logging to verify that no longer getting warnings about reference IERS being blocked.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Fix ier reward calculation
Related work items: #1554
## Summary
As per the ticket and James's explanation, there are now separate reference IERs which are used for the reference environment.
## Test process
I verified that the training can occur.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [n/a] I have written **tests** for any new functionality added with this PR
- [n/a] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Fix reference IERs
Related work items: #1554
## Summary
- Created app dirs and set as constants in the top-level init.
- Renamed _config_values_main to training_config.py and renamed the ConfigValuesMain class to TrainingConfig.
- Moved training_config.py to src/primaite/config/training_config.py
- Renamed all training config yaml file keys to make creating an instance of TrainingConfig easier.
- Moved action_type and num_steps over to the training config.
- Decoupled the training config and lay down config.
- Refactored main.py so that it can be ran from CLI and can take a training config path and a lay down config path.
- Refactored all outputs so that they save to the session dir.
- Added some necessary setup scripts that handle creating app dirs, fronting example config files to the user, fronting demo notebooks to the user, performing clean-up in between installations etc.
- Added functions that attempt to retrieve the file path of users example config files that have been fronted by the primaite setup.
- Added logging config and a getLogger function in the top-level init.
- Refactored all logs entries logged to use a logger using the primaite logging config.
- Added basic typer CLI for doing things like setup, viewing logs, viewing primaite version, running a basic session.
- Updated test to use new features and config structures.
- Made tests log to temp directory
- typer==0.9.0 added to pyproject.toml
- Refactored documentation and included APi docs, dependencies.
- Make files now re-build autosummary and deps file.
- Added typer and platformdirs to deps in pyproject.toml.
- Made root_is_pure = True in setup.py as platform/python specific wheels don't need to be built but the option is there should we need to.
## Test process
- Added an e2e test for primaite.main.run func.
- Added legacy config file conversion tests
- added
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #915
## Summary:
Split the ticket into two task
Task 1: Fixed the resetting operating state to set compromised or overwhelmed services or operating system back to a good state. Added a reset count that switches the node into a good state.
Task 2: Created a "SHUTTING DOWN" operating state to last for a (configurable) and a "BOOTING" operating state to last for a (configurable).
## Test process
First test was to test the reset changes the node to a good state when its set to a COMPROMISED state. The last two test makes sure that the node boots and shutdowns correctly.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
Related work items: #898, #1438
## Summary
This PR implements a new module called `observations` within `primaite.environment`.
The module is able to keep track of the observation space and to generate observations for the blue agent. It builds the observation space from components. Each component can be configured by supplying parameters at instantiation. For example, the Link Traffic Levels component lets the user customise how many levels there should be.
Note: If a space contains multiple components, they are combined into a 'gym.spaces.Tuple' Space. This is not compatible with some learning agents so we may need to add the options to flatten the observation space.
## Test process
I was able to run the main script with a single-component obs space. I also wrote several unit and integration tests for the new functionality.
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style
If you review this, please check the linked tickets and make sure you agree that this PR addresses them fully.
Related work items: #886, #924, #1468, #1469
## Summary
To do this, I have altered `primaite_env` to add the changes from ADSP branch for implementing the `ANY` action space.
It impacts `NODE` and `ACL` action spaces in `primaite_env.py` as all three of them are now discrete action spaces, using dictionary keys to represent different valid actions a node can take on each step.
Previously they were multi-discrete where a single action would look like this `[1,2,1,0]`.
Now an action looks like this, a dictionary entry `{.. 5: [1,2,1,0] ... }` whereby the new action is `5` for example.
It changes the `enums.py` where I added the `ANY` into `ActionType`.
I have also added a package from the ADSP branch agents to add the file utils.py. The file contains functions used by primaite_env.py to decide and check valid actions a node can take and removes the ones which are unnecessary and invalid. This is done for all three types, `NODE`, `ACL` and `ANY`.
## Test process
I have written an unit test in `test_single_action_space.py` which checks the new action space for an `ANY` laydown config has both types of actions in the `action_space` dictionary stored by the environment.
I have written an integration tests to check an agent is carrying out both `NODE` and `ACL` actions in a single episode, where I have hard coded the agent to do two specific things on two different steps.
On one step, I tell the `computer_1` node to turn off one of the nodes and on the other step it creates an ACL rule denying communication between `computer_1` and `switch_1` nodes.
## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style
Related work items: #893, #1429
- make files now re-build autosummary and deps file.
- Added typer and platformdirs to deps in pyproject.toml.
- Made root_is_pure = True in setup.py as platform/python specific wheels don't need to be built but the option is there should we need to.
-
Added an e2e test for primaite.main.run func.
- renamed _config_values_main to training_config.py and renamed the ConfigValuesMain class to TrainingConfig.
Moved training_config.py to src/primaite/config/training_config.py
- Renamed all training config yaml file keys to make creating an instance of TrainingConfig easier.
Moved action_type and num_steps over to the training config.
- Decoupled the training config and lay down config.
- Refactored main.py so that it can be ran from CLI and can take a training config path and a lay down config path.
- refactored all outputs so that they save to the session dir.
- Added some necessary setup scripts that handle creating app dirs, fronting example config files to the user, fronting demo notebooks to the user, performing clean-up in between installations etc.
- Added functions that attempt to retrieve the file path of users example config files that have been fronted by the primaite setup.
- Added logging config and a getLogger function in the top-level init.
- Refactored all logs entries logged to use a logger using the primaite logging config.
- Added basic typer CLI for doing things like setup, viewing logs, viewing primaite version, running a basic session.
- Updated test to use new features and config structures.
- Began updating docs. More to do here.
This PR fixes some minor issues that I found in the main.py script. Namely:
1. The first observation was always all zeroes when using a generic agent. This is because the `update_environment_obs()` method is not called automatically and is only called by `env.reset()`.
2. The config yaml is never closed as the close function of the file reader was only referenced but never called.
Related work items: #1441
Check out the linked bug ticket to understand the issue.
The fix was very simple- just changing which variable is passed to the reward calculation funciton.
Related work items: #1442
Fixed the resetting operating state to set compromised or overwhelmed services or operating system back to a good state. Added a reset count that switches the node into a good state.
Created a "SHUTTING DOWN" operating state to last for a (configurable) and a "BOOTING" operating state to last for a (configurable).
Created a test file to test the reset changes the node to a good state when its set to a COMPROMISED state. The last two test tests makes sure that the node boots and shutdowns correctly.
Lastly, updated the docs file as well.
Fixed the resetting operating state to set compromised or overwhelmed services or operating system back to a good state. Added a reset count that switches the node into a good state.
Created a "SHUTTING DOWN" operating state to last for a (configurable) and a "BOOTING" operating state to last for a (configurable).
Created a test file to test the reset changes the node to a good state when its set to a COMPROMISED state. The last two test tests makes sure that the node boots and shutdowns correctly.
Lastly, updated the docs file as well.
**Summary:**
This adds support for the MultiDiscrete observation spaces, the same as what exists in the ADSP branch. The observation space is now configurable in the same way as the action space- by selecting a config item within the laydown config yaml.
The 'box' option has the same behaviour as before.
**Test Process:**
I added two integration tests to ensure that creating the environment is possible with both types of observation space. I also checked that all existing unit tests run fine as long as I update the observation space in the yaml to box.
**Other comments:**
I also updated the documentation relating to observation spaces, please check if the explanation makes sense.
Related work items: #1463
I wanted to add this pull request template just as a checklist for everyone to ensure they add tests and update documentation.
Do you think it's necessary? Feel free to discuss in the comments of this PR or accept/reject the suggestion.
Related work items: #1467
In reward.py, the comparisons for the IF statements used when assigning config_values reward values currently compares the initial state to the reference state. However, it should be comparing the reference state (What it should be without any blue/red agent interference) and the final state (state after red and blue actions have taken affect).
Change the IF statement logic to say if `reference_node_os_state` and then in the following IF statement if `final_node_os_state` to compare it.
Do this for all reward functions
Write tests to evaluate step rewards
Related work items: #1443
**The following changes are made to constructor params in the Node class and its children (ActiveNode, PassiveNode, and ServiceNode):**
- _id -> node_id
- _name -> name
- _type -> node_type
- _priority -> priority
- _state -> hardware_state
- _ip_address -> ip_address
- _os_state -> software state
- _file_system_state -> file_system_state
- _config_values -> config_values
- Add type hints to all params.
(node_id, name, and ip_address are str, states and other defines types are the respective enums, leave config_values without a type for now.)
**The following changes are made to instance variables in the Node class and its children:**
- self.type -> self.node_type
- self.operating_state -> self.hardware_state
- self.os_state -> self.software_state
- Add type hints to all instance variables.
(node_id, name, and ip_address are str, states and other defines types are the respective enums, leave config_values without a type for now.)
**The following changes are made to the config files where itemType is NODE:**
- itemType -> item_type
- id -> node_id
- portsList -> ports_list
- serviceList -> service_list
- baseType -> base_type
- nodeType -> node_type
- hardwareState -> hardware_state
- ipAddress -> ip_address
- softwareState -> software_state
- fileSystemState -> file_system_state
**The following changes are made in the primaite/environment/primaite_env.py module:
In the create_node function, the id of the node needs to be retrieved using the new "node_id" key.**
- _id -> node_id
- _name -> name
- _type -> node_type
- _priority -> priority
- _state -> hardware_state
- _ip_address -> ip_address
- _os_state -> software state
- _file_system_state -> file_system_state
- _config_values -> config_values
**Few other cosmetic/code style changes too:**
- Enum classes renamed to use CamelCase.
Started refactoring out unnescessary getters and setters by using `@property` and `@<property name>.setter`.
- Have started to add Type Hints.
- Have started to move docstrings over to the Sphinx ReStructured text format.
Related work items: #1355
- Made the same renaming in the yaml laydown config files.
- Added Type hints wherever I've been.
- Added a custom NodeType in custom_typing.py to encompass the Union of ActiveNode, PassiveNode, ServiceNode.
#902 - replaced 'final_node_<placeholder>' with 'reference_node_<placeholder>' in methods for scoring of os_state, file_system_state, service state and operating state. This fixed the reward function so it is checked at each step for node operating system state, operating state, file system state and service state.
- Added unit tests.
Related work items: #902
#1356 - added if statements to set class methods for file system state, os state and service states. Refactored file enums.py
- Added unit tests
Related work items: #1356
#1378 - Re-arranged the action step function in the following order:
1. Implement the Blue Action
2. Perform any time-based activities
3. Apply PoL
4. Implement Red Action
5. Calculate reward signal
6. Output Verbose (currently disabled)
7. Update env_obs
8. Add transaction to the list of transactions
Related work items: #1378
1. Implement the Blue Action
2. Perform any time-based activities
3. Apply PoL
4. Implement Red Action
5. Calculate reward signal
6. Output Verbose (currently disabled)
7. Update env_obs
8. Add transaction to the list of transactions
Initial commit of v1.0.0. Updated the .gitignore for the standard Python gitignore. Added Azure DevOps release pipeline for proper artifact release from the start.
*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?*
## Test process
*How have you tested this (if applicable)?*
## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
* **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues).
* If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=bug&projects=&template=bug_report.md&title=%5BBUG%5D+-+%3Cbug+title+goes+here%3E). Be sure to follow our bug report template with the headers **Describe the bug**, **To Reproduce**, **Expected behaviour**, **Screenshots/Outputs**, **Environment**, and **Additional context**
### **Do you have a solution to fix the bug?**
* [Fork the repository](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/fork).
* Install the pre-commit hook with `pre-commit install`.
* Implement the bug fix.
* Update documentation where applicable.
* Update the **UNRELEASED** section of the [CHANGELOG.md](CHANGELOG.md) file
* Write a suitable test/tests.
* Commit the bug fix to the dev branch on your fork. If the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the commit message (e.g. #1 references issue 1).
* Submit a pull request from your dev branch to the Autonomous-Resilient-Cyber-Defence/PrimAITE dev branch. Again, if the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the pull request description.
### **Did you fix whitespace, format code, or make a purely cosmetic patch?**
Changes that are cosmetic in nature and do not add anything substantial to the stability, functionality, or testability of PrimAITE will generally not be accepted.
### **Do you intend to add a new feature or change an existing one?**
* Submit a [feature request issue](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=feature_request&projects=&template=feature_request.md&title=%5BREQUEST%5D+-+%3Crequest+title+goes+here%3E).
* Know how to implement the new feature or change? Follow the same steps in the bug fix section above to fork, build, document, test, commit, and submit a pull request.
### **Do you have questions about the source code?**
Ask any question about how to use PrimAITE in our discussions section.
### **Do you want to contribute to the PrimAITE documentation?**
Please follow the "Do you intend to add a new feature or change an existing one?" section above and tag your feature request issue and pull request with the documentation tag.
The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effective simulation capability for the purposes of training and evaluating AI in a cyber-defensive role. It incorporates the functionality required of a primary-level ARCD environment, which includes:
- The ability to model a relevant platform / system context;
- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems and services;
- Operates at machine-speed to enable fast training cycles.
PrimAITE presents the following features:
- Highly configurable (via YAML files) to provide the means to model a variety of platform / system laydowns, mission profiles and adversarial attack scenarios;
- A Reinforcement Learning (RL) reward function based on (a) the ability to counter the specific modelled adversarial cyber-attack, and (b) the ability to ensure mission success;
- Provision of logging to support AI evaluation and metrics gathering;
- Uses the concept of Information Exchange Requirements (IERs) to model background pattern of life, adversarial behaviour and mission data (on a sliding scale of criticality);
- An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP address, destination IP address, protocol and port);
- Application of IERs to the platform / system laydown adheres to the ACL ruleset;
- Presents an OpenAI gym or RLLib interface to the environment, allowing integration with any compliant defensive agents;
- Full capture of discrete logs relating to agent training (full system state, agent actions taken, instantaneous and average reward for every step of every episode);
@@ -6,74 +6,20 @@ Welcome to PrimAITE's documentation
====================================
====================================
What is PrimAITE?
What is PrimAITE?
-----------------
------------------------
Overview
PrimAITE (Primary-level AI Training Environment) is a simulation environment for training AI under the ARCD programme. It incorporates the functionality required of a Primary-level environment, as specified in the Dstl ARCD Training Environment Matrix document:
^^^^^^^^
The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effective simulation capability for the purposes of training and evaluating AI in a cyber-defensive role. It incorporates the functionality required of a primary-level ARCD environment, which includes:
- The ability to model a relevant platform / system context;
- Modelling an adversarial agent that the defensive agent can be trained and evaluated against;
- The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, operating systems, services and traffic loading on links;
- Modelling background pattern-of-life;
- Operates at machine-speed to enable fast training cycles.
Features
^^^^^^^^
PrimAITE incorporates the following features:
- Highly configurable (via YAML files) to provide the means to model a variety of platform / system laydowns, mission profiles and adversarial attack scenarios;
- A Reinforcement Learning (RL) reward function based on (a) the ability to counter the modelled adversarial cyber-attack, and (b) the ability to ensure mission success;
- Provision of logging to support AI performance / effectiveness assessment;
- Uses the concept of Information Exchange Requirements (IERs) to model background pattern of life, adversarial behaviour and mission data (on a sliding scale of criticality);
- An Access Control List (ACL) function, mimicking the behaviour of a network firewall, is applied across the model, following standard ACL rule format (e.g. DENY/ALLOW, source IP address, destination IP address, protocol and port);
- Application of traffic to the links of the platform / system laydown adheres to the ACL ruleset;
- Presents both an OpenAI gym and Ray RLLib interface to the environment, allowing integration with any compliant defensive agents;
- Allows for the saving and loading of trained defensive agents;
- Stochastic adversarial agent behaviour;
- Full capture of discrete logs relating to agent training or evaluation (system state, agent actions taken, instantaneous and average reward for every step of every episode);
- Distinct control over running a training and / or evaluation session;
PrimAITE is a Python application and is therefore Operating System agnostic. The OpenAI gym and Ray RLLib frameworks are employed to provide an interface and source for AI agents. Configuration of PrimAITE is achieved via included YAML files which support full control over the platform / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count. NetworkX based nodes and links host Python classes to present attributes and methods, and hence a more representative platform / system can be modelled within the simulation.
Training & Evaluation Capability
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its OpenAI Gym and RLLib compliant interface. Scenarios can be constructed to reflect platform / system laydowns consisting of any configuration of nodes (e.g. PCs, servers, switches etc.) and network links between them. All nodes can be configured to model services (and their status) and the traffic loading between them over the network links. Traffic loading is broken down into a per service granularity, relating directly to a protocol (e.g. Service A would be configured as a TCP service, and TCP traffic then flows between instances of Service A under the direction of a tailored IER). Highlights of PrimAITE’s training and evaluation capability are:
- The scenario is not bound to a representation of any platform, system, technology or mission;
- Fully configurable (network / system laydown, IERs, node pattern-of-life, ACL, number of episodes, steps per episode) and repeatable to suit the requirements of AI agents;
- Can integrate with any OpenAI Gym or RLLib compliant AI agent.
Use of PrimAITE default scenarios within ARCD is supported by a “Use Case Profile” tailored to the scenario.
AI Assessment Capability
^^^^^^^^^^^^^^^^^^^^^^^^
PrimAITE includes the capability to support in-depth assessment of cyber defence AI by outputting logs of the environment state and AI behaviour throughout both training and evaluation sessions. These logs include the following data:
- Timestamp;
- Episode and step number;
- Agent identifier;
- Observation space;
- Action taken (by defensive AI);
- Reward value.
Logs are available in CSV format and provide coverage of the above data for every step of every episode.
* The ability to model a relevant platform / system context;
* The ability to model key characteristics of a platform / system by representing connections, IP addresses, ports, traffic loading, operating systems, file system, services and processes;
* Operates at machine-speed to enable fast training cycles.
PrimAITE aims to evolve into an ARCD environment that could be used as the follow-on from Reception level approaches (e.g. YAWNING TITAN), and help bridge the Sim-to-Real gap into Secondary level environments (e.g. IMAGINARY YAK).
This is similar to the approach taken by FVEY international partners (e.g. AUS CyBORG, US NSA FARLAND and CAN CyGil). These environments are referenced by the Dstl ARCD Agent Training Environments Knowledge Transfer document (TR141342).
What is PrimAITE built with
What is PrimAITE built with
---------------------------
--------------------------------------
*`OpenAI's Gym <https://gym.openai.com/>`_ is used as the basis for AI blue agent interaction with the PrimAITE environment
*`OpenAI's Gym <https://gym.openai.com/>`_ is used as the basis for AI blue agent interaction with the PrimAITE environment
*`Networkx <https://github.com/networkx/networkx>`_ is used as the underlying data structure used for the PrimAITE environment
*`Networkx <https://github.com/networkx/networkx>`_ is used as the underlying data structure used for the PrimAITE environment
@@ -85,8 +31,8 @@ What is PrimAITE built with
*`Plotly <https://github.com/plotly/plotly.py>`_ is used for building high level charts
*`Plotly <https://github.com/plotly/plotly.py>`_ is used for building high level charts
Getting Started with PrimAITE
Where next?
-----------------------------
------------
Head over to the :ref:`getting-started` page to install and setup PrimAITE!
Head over to the :ref:`getting-started` page to install and setup PrimAITE!
In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.10 installed. If you don't already have it, this is how to install it:
In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.10 installed. If you don't already have it, this is how to install it:
training_config = <path to training config yaml file>
lay_down_config = <path to lay down config yaml file>
run(training_config, lay_down_config)
When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/2.0.0/sessions``).
When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/2.0.0/sessions``).
The sub-directory is formatted as such: ``~/primaite/2.0.0/sessions/<yyyy-mm-dd>/<yyyy-mm-dd>_<hh-mm-dd>/``
The sub-directory is formatted as such: ``~/primaite/2.0.0/sessions/<yyyy-mm-dd>/<yyyy-mm-dd>_<hh-mm-dd>/``
@@ -49,36 +49,6 @@ For example, when running a session at 17:30:00 on 31st January 2023, the sessio
``primaite session`` can be ran in the terminal/command prompt without arguments. It will use the default configs in the directory ``primaite/config/example_config``.
``primaite session`` can be ran in the terminal/command prompt without arguments. It will use the default configs in the directory ``primaite/config/example_config``.
To run a PrimAITE session using legacy training or laydown config files, add the ``--legacy-tc`` and/or ``legacy-ldc`` options.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
