Commit Graph

2678 Commits

Author SHA1 Message Date
Marek Wolan
98617981d7 Merged PR 636: Release 4.0.0
# PrimAITE 4.0.0 Release Note

## 📰 Headlines 📰

* **Major Release** - This is the first major release of PrimAITE that has been published since PrimAITE 2.0.0. Users are encouraged to also familiarise themselves with [what's new in PrimAITE 3.0.0](todo: 3.0 release note link).
* **Plugins** - PrimAITE now supports external plugins to extend existing functionality, such as new node types, services, applications, agent types, observations, actions, and rewards.
* **Improved config schema** - The YAML config files have a new schema which simplifies the definition of agents by using named references in actions and observations instead of id references.
* **Use Case 7** - Introduction of much larger UC7 network with example notebooks and config files.

##  What's new 

- **Use Case 7** is a new pre-defined scenario in PrimAITE with a more complex set of nodes that demonstrates some of the new capabilities of PrimAITE 4.
- **Threat Actor Profiles** are a new type of malicious scripted agent:
  - Extensible base class that allows defining a multi-stage kill chain with success conditions and network knowledge tracking.
  - TAP001 agent - agent that exfiltrates and corrupts data from a database.
  - TAP003 agent - agent that maliciously introduces ACL rules to a network to disrupt normal users.
- **Users, Terminals, SSH, and Command&Control** were [introduced in PrimAITE 3.3]().
- **Determinism** Support for determinism by setting and logging randomness seeds.
- **Action Masking** (introduced in 3.2.0).
- **MARL support** (introduced in 3.0.0).
- **Logging** was improved by adding the following:
  - detailed information about agent actions, decisions, and rewards.
  - the full state of the simulation after each environment step.
  - sys logs for each node.
  - pcap logs for each network interface.
- **Domain Randomisation** - the Gym environment can use different variations of the same scenario, alternating between them each episode by providing a folder of YAML files instead of a single file at initialisation.

## 👍 General Improvements 👍
- The organisation of the codebase has been improved by splitting long files into smaller ones.
- Agent logs can now show observation history and more detail about the reward.
- Some classes' `.show()` methods now show more useful or correct information (like agents, and networks).
- More example notebooks and introduction of how-to guides in the Sphinx docs.
- Ability to set scenario-wide default values for certain actions like scan, node power-off, node start-up, etc.
- It's now easier to build complex networks with the new, extensible `NetworkNodeAdder` class.
- NMAP application.
- The YAML config files:
  - support extended classes from plugins
  - information has been deduplicated - actions no longer rely on IDs, instead users can specify meaningful labels.
  - more objects have default values so there is less boilerplate.
  - the way agent settings are defined has been standardised.
  - more data va...
v4.0.0
2025-03-18 11:26:08 +00:00
Marek Wolan
ac4d7e9ecc Merged PR 634: Merge final doc updates into prerelease branch
Related work items: #2936, #3110
2025-03-17 09:20:14 +00:00
Archer Bowen
b48ed25b23 Merged PR 633: #3110 Final user guide comments.
## Summary
Feedback following James' comments

## Test process

## Checklist
- [x ] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [x] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

#3110 Final user guide comments.

Related work items: #3110
2025-03-17 09:09:59 +00:00
Charlie Crane
e9e49ab5f9 Fix some issues with sphinx rendering text in jupyter notebooks 2025-03-14 16:07:08 +00:00
Charlie Crane
e94d7b0b3f Revert "Formatting changes to correct issues with sphinx rendering of documentation"
This reverts commit 42512a2fc2.
2025-03-14 16:01:55 +00:00
Charlie Crane
42512a2fc2 Formatting changes to correct issues with sphinx rendering of documentation 2025-03-14 16:00:30 +00:00
Charlie Crane
b2a097a1cc Merge branch 'bugfix/user-guide-comments' of https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE into bugfix/user-guide-comments 2025-03-14 14:58:36 +00:00
Charlie Crane
67814893e1 Updates to Licence table in primaite-dependencies.rst 2025-03-14 14:57:33 +00:00
Archer Bowen
b5a2234a24 Apply suggestions from code review 2025-03-14 14:38:39 +00:00
Charlie Crane
7f7fa36f14 Merged PR 632: CI pipeline Updates
## Summary
*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?*

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2936
2025-03-14 14:11:06 +00:00
Archer Bowen
a8e64a63f8 #3110 Final user guide comments. 2025-03-14 14:06:03 +00:00
Charlie Crane
cd2da383bf Standardise how python version is declared in ADO pipelines to prevent python version weirdness 2025-03-14 09:59:58 +00:00
Charlie Crane
d1a66c14ae Testing a resolution for pipeline failure seen with using python v3.10 2025-03-14 09:51:59 +00:00
Marek Wolan
20dcc0a987 Updated VERSION 2025-03-13 18:19:49 +00:00
Charlie Crane
062926abd2 Merged PR 629: Minor Userguide changes (Charlie)
## Summary
Minor userguide changes, working from the "bottom" of the comments from 3110

## Test process
User Guide continues to successfully render.

## Checklist
- [X] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #3110
2025-03-13 18:19:06 +00:00
Marek Wolan
0ccf350766 Merge remote-tracking branch 'origin/dev' into feature/3110-userguide-fixes-Charlie 2025-03-13 18:13:07 +00:00
Nick Todd
13fe8f4681 Merged PR 631: #3113: Fix incorrect hostname.
## Summary
*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?*

## Test process
Tested affected notebook.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#3113: Fix incorrect hostname.

Related work items: #3113
2025-03-13 18:12:42 +00:00
Marek Wolan
f4f99d3e67 Merge remote-tracking branch 'origin/dev' into feature/3113_snagging 2025-03-13 18:02:01 +00:00
Marek Wolan
237965823e Merged PR 630: Documentation changes.
Update docs

Related work items: #3110, #3111
2025-03-13 18:01:17 +00:00
Marek Wolan
ec7ee245d9 3110 - appease precommit 2025-03-13 15:59:56 +00:00
Marek Wolan
36a78cc565 Apply suggestions from code review 2025-03-13 15:57:16 +00:00
Charlie Crane
0c2c6bad5d Merge branch 'feature/#3110-userguide-fixes' into feature/3110-userguide-fixes-Charlie 2025-03-13 15:55:31 +00:00
Archer Bowen
6d4be8a653 #3110 UC7 notebook updates:
- Improve clarity of some code cells (narrowed output)
- Reworded some questionably worded sections
- Updated some of the util functionality that using old action names
- Updated a lot of old names into kebab-case
- General tidy up and consistency changes.
2025-03-13 15:21:07 +00:00
Charlie Crane
c3d88c0ce5 #3110 - Merge in changes on target branch 2025-03-13 15:09:49 +00:00
Marek Wolan
7d37ddbb67 3110 - Add some more module level docstrings. 2025-03-13 15:07:38 +00:00
Nick Todd
aca3ff76db Merge branch 'feature/#3110-userguide-fixes' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into feature/#3110-userguide-fixes 2025-03-13 15:01:33 +00:00
Nick Todd
a43c660a83 #3110: pre-commit fixes. 2025-03-13 15:01:23 +00:00
Marek Wolan
8ddb6916fb 3110 - Add module-level docstrings 2025-03-13 14:57:34 +00:00
Nick Todd
8ad350de84 Merge branch 'feature/#3110-userguide-fixes' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into feature/#3110-userguide-fixes 2025-03-13 14:56:20 +00:00
Nick Todd
618e2a2caf #3110: Updates to data_manipulation_bot text. 2025-03-13 14:56:10 +00:00
Charlie Crane
61afaaf8e7 #3110 - Correct some YAML formatting of agent_settings in agents.rst 2025-03-13 14:46:37 +00:00
Marek Wolan
864016b893 3110 - Fix code example and broken references to ipprotocol in c2 suite docs 2025-03-13 14:36:48 +00:00
Marek Wolan
128b459c04 3110 - comment out not-yet-implemented TCP description 2025-03-13 14:21:56 +00:00
Marek Wolan
d4287d6690 3110 Fix formatting and code examples in docs 2025-03-13 14:18:26 +00:00
Marek Wolan
dc65681f2c 3110 Add generated notebooks folder to makefile clean command 2025-03-13 14:18:03 +00:00
Charlie Crane
1433ee08f0 #3110 - Address some incorrect comments in PrimAITE documentation and sort outstanding TODO comments 2025-03-13 13:06:45 +00:00
Marek Wolan
ba3d855b4c 3110 - change notebook title to deduplicate 2025-03-13 11:54:16 +00:00
Marek Wolan
00364c3a9f 3110 - Fix naming schema for rewards in docs 2025-03-13 11:53:44 +00:00
Marek Wolan
9ffd5b90d9 3110 - Remove empty customising scenarios doc page 2025-03-13 11:53:44 +00:00
Nick Todd
e8e8ad7681 Merge branch 'feature/#3110-userguide-fixes' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into feature/#3110-userguide-fixes 2025-03-13 11:36:36 +00:00
Nick Todd
dc3dc5d9c1 #3110: Correct config YAML snippets. 2025-03-13 11:36:24 +00:00
Marek Wolan
3cb9e51084 3110 - docs formatting fixes and updating text 2025-03-13 11:29:03 +00:00
Charlie Crane
2107e2f166 Merge branch 'feature/#3110-userguide-fixes' into feature/3110-userguide-fixes-Charlie 2025-03-13 09:24:02 +00:00
Marek Wolan
8b91c280cb 3110 - remove outdated todos 2025-03-12 17:13:03 +00:00
Charlie Crane
3fa1104d2e #3110 - Amend Documentation pages that have out of date material following updates 2025-03-12 17:10:51 +00:00
Nick Todd
ea7898ce33 #3110: More example config file and domain name changes 2025-03-12 15:47:26 +00:00
Nick Todd
c54007daec Merge branch 'feature/#3110-userguide-fixes' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into feature/#3110-userguide-fixes 2025-03-12 15:23:33 +00:00
Nick Todd
af025d45ff #3110: config file example changes. 2025-03-12 15:23:17 +00:00
Marek Wolan
19d7989c4a Add extra detail and examples to YAML migration guide 2025-03-12 15:21:35 +00:00
Nick Todd
907294992c #3110: Update example schema in terminal.rst 2025-03-12 14:53:34 +00:00