Commit Graph

2601 Commits

Author SHA1 Message Date
Charlie Crane
fcfea3474f #2900 - typo in test_ftp_client and expanded test_terminal_last_response_updates to include a failure scenario 2024-10-01 11:41:42 +01:00
Charlie Crane
1e1d152481 #2900 - Updates to Terminal-Processing jupyter notebook to include a mention of last_response 2024-10-01 11:02:23 +01:00
Charlie Crane
3dafad71b3 #2900 - New test to show that last_response updates as expected. Changelog updated. 2024-10-01 10:45:03 +01:00
Charlie Crane
ac921749a7 #2900 - Changes to terminal to include a last_response attribute, for use in obtaining RequestResponse from remote command executions 2024-09-30 17:38:24 +01:00
Czar Echavez
33c020e671 Merged PR 557: #2897: How to guide on dev mode
## Summary
Added a how to guide which shows the example of the primaite dev-mode commands being run and how to effectively use the commands

## Test process
n/a the notebook is a test itself

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2897: How to guide on dev mode

Related work items: #2897
2024-09-30 10:10:13 +00:00
Marek Wolan
2bd6c69343 Merged PR 549: Align main and dev histories
Related work items: #1847, #2438, #2457, #2472, #2502, #2536, #2552, #2560, #2561, #2563, #2570, #2588, #2606, #2610, #2618, #2620, #2623, #2625, #2626, #2628, #2637, #2639, #2641, #2646, #2656, #2658, #2659, #2660, #2661, #2681, #2682, #2686, #2688, #2689, #2697, #2700, #2701, #2702, #2705, #2716, #2718, #2720, #2721, #2725, #2734, #2736, #2739, #2740, #2745, #2747, #2748, #2757, #2768, #2769, #2772, #2779, #2781, #2799, #2826, #2837, #2844
2024-09-30 09:32:50 +00:00
Marek Wolan
d26fcf01a2 Merged PR 558: Make software manager show method always show all software.
DataFusionService didn't show in software list, because it uses the port_protocol_mapping as the source of truth for installed software instead of the software list itself.

This fixes that :)

Related work items: #2899
2024-09-30 08:16:46 +00:00
Marek Wolan
221e09ba51 Turn AirSpaceFrequency to a schema instead of a dict for validation 2024-09-27 15:06:19 +01:00
Marek Wolan
5282cb0294 #2899 - Make software manager always show() all software 2024-09-27 10:56:48 +01:00
Czar Echavez
17fe5cb043 #2897: How to guide on dev mode 2024-09-27 10:47:38 +01:00
Marek Wolan
521580ea12 Merged PR 555: Refactor Ports and Airspace to be extensible
## Summary
* Ports are no longer enums, they are now plain integers
* Ports are no longer validated against a list of pre-defined ports, any integer from 0-65535 can be used now.
* Port enum was replaced with a convenience port lookup. For example any expression in the form `Port.HTTP` was replaced with `PORT_LOOKUP["HTTP"]` which resolves to the integer 80.
* Some tests were adjusted to use the new syntax for ports
* Backwards compatibility for YAML configs has been retained by adding pydantic validators to automatically convert named ports to integer counterparts, however defining action/observation spaces in code now requires users to specify ports as integers instead of port enum objects. For instance, monitored_ports in the ACL observation space will now look like this: `[53, 80]` instead of `[Port.DNS, Port.HTTP]`
* Plugins can extend the port lookup, however it is not necessary because it's possible to use integer literals.
* Airspace has been treated similarly, except airspace frequencies have multiple attributes, namely max_data_rate_bps. Therefore, the lookup for named Frequencies resolves to a dictionary of freq_hz (float), and max_data_rate_bps (float)
* Airspace logic has been adjusted accordingly to use the new dictionary for tracking bandwidth limits
* A new method for registering new airspace frequencies has been added. Plugins that add new frequencies can call `register_default_frequency`, after which any new airspace will have access to that frequency.

* For consistency, `IPProtocol` was also changed to be a lookup instead of an enum. It is not extendable by plugins as so far we have not needed to model additional protocols in our plugin.

These changes were necessary as it's not possible to extend enums in python, therefore plugins would not have been able to add new ports. There is an added benefit that this is a stepping stone towards support communication on dynamic and ephemeral ports.

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code
2024-09-27 09:19:17 +00:00
Marek Wolan
c74d5ac227 Fix changelog typo and remove repitition in ACL show method 2024-09-27 09:28:26 +01:00
Marek Wolan
203ec5ec85 Add tests for port and protocol validation and update changelog 2024-09-26 16:00:59 +01:00
Marek Wolan
f2b6d68b14 Fix Port scan 2024-09-26 15:35:50 +01:00
Czar Echavez
e7c30d5c8b Merged PR 548: #2445: added the ability to pass the game options thresholds to observations...
## Summary
Added ability to pass the game options thresholds into observation classes

This will allow for NICObservation, FileObservation (and FolderObservation) and ApplicationObservation to be able to get the thresholds for the training session. i.e. Allows for the thresholds for NMNE, file access and application executions to be configurable for training.

## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/548?_a=files&path=/tests/integration_tests/configuration_file_parsing/test_game_options_config.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2445: added the ability to pass the game options thresholds to observations so that relevant observation items can retrieve the thresholds from config

Related work items: #2445
2024-09-26 08:46:36 +00:00
Czar Echavez
603c68acf9 #2445: grammar in changelog 2024-09-26 08:51:30 +01:00
Marek Wolan
50e2234a69 Remove commented out code 2024-09-25 16:51:58 +01:00
Marek Wolan
c3eb093144 remove temporary notebook 2024-09-25 16:50:01 +01:00
Marek Wolan
f1b911bc65 Change port and protocol to annotated validators 2024-09-25 16:28:22 +01:00
Czar Echavez
b9df2bd6a8 #2445: apply PR suggestions 2024-09-25 10:50:26 +01:00
Czar Echavez
171dd83f2f Merge remote-tracking branch 'origin/dev' into feature/2445-make-observation-thresholds-configurable 2024-09-25 09:04:18 +01:00
Archer Bowen
b0363a6b48 Merged PR 534: #2840 NODE_SEND_LOCAL_COMMAND
## Summary
Enables agents to use a new CAOS action ``NODE_SEND_LOCAL_COMMAND``.

## Test process

Added a new unit test as well as tested manually via sandbox notebooks.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [X] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2840
2024-09-24 10:47:08 +00:00
Archer Bowen
e29815305d #2840 Addressing PR comments. 2024-09-24 11:06:38 +01:00
Archer Bowen
88cbb783bc #2840 Fixed sphinx user guide formatting issues. 2024-09-20 13:54:13 +01:00
Archer Bowen
710a4d0e55 Merge remote-tracking branch 'origin/dev' into feature/2840_LOCAL_TERMINAL_COMMAND 2024-09-20 12:38:22 +01:00
Marek Wolan
695891f55c Add port and protocol custom validators 2024-09-20 11:21:28 +01:00
Marek Wolan
08f1cf1fbd Fix airspace and remaining port problems from refactor 2024-09-19 15:06:29 +01:00
Nick Todd
1a800ff83d Merged PR 552: Increment PrimAITE version.
## Summary
Increment PrimAITE version.

## Test process
Ran test suite.

## Checklist
- [X] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2896: Bump version.

Related work items: #2896
2024-09-19 11:27:48 +00:00
Nick Todd
0c576746aa #2896: Bump version. 2024-09-19 11:07:00 +01:00
Marek Wolan
dd931d900b port refactor - eod commit, airspace still broken 2024-09-18 16:02:25 +01:00
Marek Wolan
8c8b3c498a Merged PR 545: Merge release/3.3 into main
Related work items: #2681, #2686, #2689, #2718, #2720, #2721, #2736, #2748, #2768, #2769, #2772, #2779, #2781, #2799, #2826, #2837, #2844
2024-09-18 08:21:58 +00:00
Charlie Crane
6a177caadb Merged PR 544: #2871 - Add show functions to Agents
## Summary
This PR introduces a `show_history` function that can be used to view what actions have been taken by an agent over a training session. Be default, this won't show any `DONOTHING` actions.

## Test process
![image.png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/544/attachments/image.png)

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2871
2024-09-18 07:30:18 +00:00
Charlie Crane
8d3760b5a7 #2871 - Fix notebook failure 2024-09-17 16:19:43 +01:00
Nick Todd
d50c0c8911 Merged PR 538: NMNE config issues
## Summary
Update NMNE and NICObservation code to handle all four permutations of include_nmne and capture_nmne set to True/False.

## Test process
Updated existing NMNE and NICObservation tests.
All existing tests pass.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2829

Related work items: #2829
2024-09-17 15:06:06 +00:00
Nick Todd
335ca9a02f Merged PR 546: Log the random seed used on each run
## Summary
Added changes so that the value of the random number seed is recorded in a log file when it's specified or the user asks for a RNG seed to be automatically generated.

## Test process
Updated existing RNG tests; added new test.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2879
2024-09-17 13:06:10 +00:00
Czar Echavez
4f172634d0 Merge remote-tracking branch 'origin/dev' into feature/2445-make-observation-thresholds-configurable 2024-09-17 12:42:57 +01:00
Czar Echavez
4391d7cdd5 #2445: added the ability to pass the game options thresholds to observations so that relevant observation items can retrieve the thresholds from config 2024-09-17 12:19:35 +01:00
Charlie Crane
3a5b75239d #2871 - Typo in Changelog 2024-09-17 12:05:40 +01:00
Czar Echavez
c589c3d3c2 Merged PR 543: #2880: fix action shape for num_ports + test
## Summary
Resolved a bug where the array for protocols is used to determine the num_ports instead of the array for ports

## Test process
Added a test that checks the shape of the RouterAddACLRule action

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2880: fix action shape for num_ports + test

Related work items: #2880
2024-09-17 09:38:05 +00:00
Charlie Crane
ccb91869c4 #2871 - Minor wording change to description in agents.rst 2024-09-17 10:17:18 +01:00
Nick Todd
67a7142221 Merge branch 'dev' into feature/2879-log-random-seed-value 2024-09-17 10:11:15 +01:00
Charlie Crane
c8f6459af6 #2871 - Changelog and documentation updates, corrected changes in Data manipulation demo notebook 2024-09-17 10:09:10 +01:00
Charlie Crane
cf9646a183 Merge branch 'dev' into feature/2871-Add_show_methods_agents 2024-09-17 09:46:27 +01:00
Charlie Crane
5d7935cde0 #2871 - Changes to notebooks following updates to action history 2024-09-17 09:39:32 +01:00
Nick Todd
078b898565 #2879: Update changelog. 2024-09-17 09:30:14 +01:00
Charlie Crane
78014cd54e Merged PR 540: Sync User Guide updates back into Dev
## Summary
Syncing Dev with release/3.3.0 following changes to the User Guide after review.

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2837
2024-09-17 08:28:06 +00:00
Nick Todd
f3ca9c55c9 #2879: Update tests 2024-09-16 16:38:19 +01:00
Nick Todd
215ceaa6e8 #2879: Fix call to set_random_seed() in reset(). 2024-09-16 10:08:45 +01:00
Nick Todd
e0a1092834 #2879: Pre-commit fix. 2024-09-16 09:04:17 +01:00
Charlie Crane
5006e41546 #2871 - Updated the show_history() function to receive a list of actions to ignore when printing the history. Defaults to ignoring DONOTHING actions 2024-09-13 15:47:59 +01:00