Commit Graph

1317 Commits

Author SHA1 Message Date
Marek Wolan
a7bfc56b98 Apply documentation changes based on PR review. 2024-03-05 11:21:49 +00:00
Marek Wolan
d0225bf3e1 Merge remote-tracking branch 'origin/dev' into feature/2278-log-agent-actions 2024-03-04 19:00:03 +00:00
Marek Wolan
c3010ff816 Update changelog and docs 2024-03-04 18:59:03 +00:00
Marek Wolan
2f456e7ae0 Move IO to environments from session and add agent logging 2024-03-04 18:47:50 +00:00
Marek Wolan
84f5d45540 Merged PR 294: Update actions set in uc2 to match Yak
## Summary
Expand the available actions.

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

Related work items: #2243
2024-03-04 14:39:07 +00:00
Marek Wolan
bc830e6458 Merge remote-tracking branch 'origin/dev' into feature/2243-sync-uc2-action-to-iy 2024-03-04 13:39:22 +00:00
Marek Wolan
c32bd3f941 Merged PR 285: Green agent that sometimes performs database connections
## Summary
* removed `GreenWebBrowsingAgent` because it was replaced by probabilistic agent
* created new 'probabilistic agent' which selects actions randomly from its action map, with configurable probabilities
* slightly refactored action manager to decouple it from `PrimaiteGame` (as a consequence, agents should be given the current timestep if their `get_action()` method is time-dependent)
* refactored `data_manipulation_bot` to use an existing db client on the host rather than inheriting from it
* added new type of SQL query to databases: `"SELECT * FROM pg_stat_activity"` to model checking connection status
* added new execution definition on the `DatabaseClient` app which just performs that new SQL query
* added reward for the green admin being able to connect to the db
* updated uc2 notebook to reflect new changes.
* updated documentation for data manipulation bot
* added new test for probabilistic agent
* added test for new reward

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

Related work items: #2319
2024-03-04 12:06:55 +00:00
Marek Wolan
2c3652979b Add helpful error messages to action index errors 2024-03-04 11:17:54 +00:00
Marek Wolan
ac9d550e9b Change get_action signature for agents 2024-03-04 10:43:38 +00:00
Marek Wolan
d1480e4477 Apply suggestions from PR review. 2024-03-04 09:58:57 +00:00
Marek Wolan
0d490d618c Update MARL config 2024-03-03 16:59:14 +00:00
Marek Wolan
a4c723858b Update action map in second 2024-03-03 16:57:53 +00:00
Marek Wolan
afc3635bfe Update changelog 2024-03-03 16:56:52 +00:00
Marek Wolan
ea1d016bc2 Merge remote-tracking branch 'origin/feature/2319-database-admin' into feature/2243-sync-uc2-action-to-iy 2024-03-03 16:55:42 +00:00
Marek Wolan
0e8c60df4c Update actions 2024-03-03 16:53:18 +00:00
Marek Wolan
a6031d568d Remove unused import 2024-03-03 16:36:08 +00:00
Marek Wolan
ef1a2dc3f4 clear uc2 notebook outputs 2024-03-03 16:00:10 +00:00
Marek Wolan
afa775baff Add test for new reward 2024-03-03 15:52:34 +00:00
Marek Wolan
4d51b1a414 Update configs to new db manipulation bot approach 2024-03-03 14:57:28 +00:00
Marek Wolan
070655cfce Update data manipulation bot documentation 2024-03-03 11:47:50 +00:00
Marek Wolan
9762927289 Update notebook with new changes 2024-03-03 11:43:24 +00:00
Marek Wolan
fe13583d2a Merge remote-tracking branch 'origin/dev' into feature/2319-database-admin 2024-03-03 11:24:09 +00:00
Marek Wolan
4a292a6239 Fix checking connection in db client 2024-03-03 11:23:24 +00:00
Marek Wolan
80158fd9b4 Make db manipulation bot work with db client 2024-03-03 11:18:06 +00:00
Christopher McCarthy
e8b1177c18 Merged PR 291: #2357 - Allow empty nodes, links, and agents arrays in config file
## Summary
Allowed the config to not have nodes, links and agents and still be parsed

## Test process
Tested with config file that doesn't have any simulation key/value pair

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2357 - Allowed the config to not have nodes, links and agents and still be parsed

Related work items: #2357
2024-03-02 19:30:22 +00:00
Christopher McCarthy
e5ebf6a745 Merged PR 292: #2358 - Use class name for node type properties in network container
## Summary
The node-specific properties in Network class now simply use node.__class__.__name__ to check their type for filtering by type

## Test process
Tests updated to use the new property function names

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2358 - the node-specific properties in Network class now simply use node.__class__.__name__ to check their type for filtering by type. Tests updated to use the new property function names

Related work items: #2358
2024-03-02 18:34:51 +00:00
Chris McCarthy
81fd43035d #2358 - the node-specific properties in Network class now simply use node.__class__.__name__ to check their type for filtering by type. Tests updated to use the new property function names 2024-03-01 22:51:01 +00:00
Chris McCarthy
af036f63f1 #2357 - Allowed the config to not have nodes, links and agents and still be parsed 2024-03-01 22:37:51 +00:00
Czar Echavez
ac304c85a3 Merged PR 290: #2356: optional dmz port + optional external acl rules
## Summary
Bug fix to allow firewall dmz port to be optional as well as external acl rules to be optional

## Test process
See https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/290?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_firewall_config.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2356: optional dmz port + optional external acl rules

Related work items: #2356
2024-03-01 17:49:37 +00:00
Czar Echavez
78ff658e30 #2356: optional dmz port + optional external acl rules 2024-03-01 16:48:05 +00:00
Marek Wolan
2a1d99ccce Fix problem with checking connection for db admin 2024-03-01 16:36:41 +00:00
Marek Wolan
ed01293b86 Make db admin reward persistent 2024-03-01 16:02:27 +00:00
Marek Wolan
10a4053887 Fix tests 2024-03-01 15:14:00 +00:00
Marek Wolan
9ff8adab1a Merge remote-tracking branch 'origin/dev' into feature/2319-database-admin 2024-03-01 09:23:00 +00:00
Czar Echavez
b849bfd6e2 Merged PR 280: Router Routes and Configuration documentation updates
## Summary

- Updated Sphinx 6.1.3 => 7.1.2
- Updated furo 2023.3.27 => 2024.01.29
- Added tests to check that firewall, routers and nodes are properly added via config (config parser tests)
- Added some reference points in code comments for Sphinx documentation to reference
- Created a list of System Software so it can be referenced in docs and by code
- Added default values to config within game.py (The defaults are pulled from example_config.yaml)
- Created a section for creating a session via config files:
    - set up a lot of stuff so documentation is easier to maintain
    - Template for things that are repeated in places
    - added how to create nodes via config
    - added how to install applications and services via config
- diagrams to make it easier to understand some stuff e.g. ACL rules for firewall

see https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/CHANGELOG.md

## Test process
- Firewall:
  - Created a DMZ Network example config
  - Tested the creation of Firewall
  - Tested the ACL Rules of Firewall

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_firewall_config.py

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_router_config.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2257
2024-02-29 16:22:55 +00:00
Czar Echavez
69b7ca1703 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-29 15:21:20 +00:00
Czar Echavez
49a4e1fb56 #2257: added common node attributes page + ability to set node operating state via config + tests 2024-02-29 15:20:54 +00:00
Christopher McCarthy
90d51c071c Merged PR 288: #2326 - Network Interface port name/num fixed so that it carries through to s...
## Summary
This pull request fixes the issue o network interface port name and number not being pulled through into sys log and PCAP output files.

## Test process
Manually checked:
![image (4).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%284%29.png)
![image (2).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%282%29.png)
![image (3).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%283%29.png)

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2326 - Network Interface port name/num fixed so that it carries through to sys log and PCAP outputs.

Related work items: #2326
2024-02-29 15:14:08 +00:00
Chris McCarthy
8f0de8521e #2326 - removed port_name print statement 2024-02-29 14:08:42 +00:00
Marek Wolan
bd0b2e0033 Remove redundant notebook cells 2024-02-29 13:22:41 +00:00
Marek Wolan
2f3e40fb6b Fix issue around reset 2024-02-29 13:22:05 +00:00
Marek Wolan
f7c535b557 Merge 'origin/dev' into feature/2319-database-admin 2024-02-29 13:21:44 +00:00
Chris McCarthy
cf0674ce22 #2326 - Network Interface port name/num fixed so that it carries through to sys log and PCAP outputs. 2024-02-29 13:00:27 +00:00
Czar Echavez
eefc2739c8 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-29 11:34:56 +00:00
Marek Wolan
312b5c35bd Merged PR 283: Refactor episode reset
## Summary
Instead of setting all attributes back to a snapshot, simply recreate the entire game and all agents from a cached copy of the config.

This removes the need for `set_original_state` and `reset_component_for_episode` methods on SimComponents.

## Test process
* Unit tests passing
* I've also tried adding a `__del__` method to simcomponent and agent and press env.reset() to verify that the ref count reaches 0, and nothing is secretly keeping using an old part of the simulation.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2317
2024-02-29 11:11:03 +00:00
Czar Echavez
9a4587155b #2257: specifically stating that enpoint refs are node hostnames + remove TODO 2024-02-29 11:07:21 +00:00
Marek Wolan
ce39387f92 Merge 'origin/dev' into feature/2317-refactor-reset 2024-02-29 10:16:42 +00:00
Marek Wolan
8730330f73 Apply PR suggestions 2024-02-29 10:14:31 +00:00
Czar Echavez
13db07dc11 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-28 15:09:23 +00:00
Czar Echavez
6d43c61058 #2257: apply PR suggestions 2024-02-28 15:08:00 +00:00