Commit Graph

188 Commits

Author SHA1 Message Date
Charlie Crane
96f62a3229 Merged PR 436: 2656 - Implement Optional Negative Reward
## Summary
This PR implements an "ActionPenalty" Reward class, that can be used for penalising an agent when it carries out an action other than DONOTHING.

The value for this can be configured within the scenario YAML file.

## Test process
I've written unit tests to check that the action penalty is correctly calculated and applied, another to check that the config is read correctly and loads into PrimaiteGymEnv correctly.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2656
2024-07-01 13:25:05 +00:00
Marek Wolan
7a680678aa #2656 - Make action penalty more configurable 2024-06-27 12:01:32 +01:00
Charlie Crane
7a833afe2d #2656 - Unit tests for new ActionPenalty reward component, testing yaml and some minor changes to the implementation. Need to update Documentation to detail how this is added 2024-06-26 12:20:28 +01:00
Charlie Crane
5ad16fdb7e #2656 - Corrected from_config() for ActionPenalty so that it can pull the negative reward value from YAML and apply, defaulting to 0 still if not found/not configured.
Currerntly prints to terminal when a negative reward is being applied, though this is for implementation and troubleshooting. To be removed before PR is pushed out of draft
2024-06-25 15:36:47 +01:00
Charlie Crane
db27bea4ec #2656 - Committing current state before lunch. New ActionPenalty reward added. Basic implementation returns a -1 reward if last_action_response.action isn't DONOTHING. Minor change in data_manipulation so I can see it working in the data_manipulation notebook. Need to use configured values but so far, promising?. Looks to result in a better average reward than without which is good, I think. 2024-06-25 12:29:01 +01:00
Marek Wolan
28dabad66b #2682 Backport changes to core PrimAITE 2024-06-25 11:04:52 +01:00
Charlie Crane
923d2629ca Merged PR 418: Update Pre-Commit to check for valid copyright
## Summary
Updates Pre-commit with the a check to ensure valid copyright headers

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [X] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2646
2024-06-13 18:54:48 +00:00
Czar Echavez
101fa3ebdb #2658: change key of observation to match what is in CAOS document 2024-06-13 16:29:02 +01:00
Czar Echavez
65d9cfa7dc #2658: I cast uncast on NIC space 2024-06-13 15:11:34 +01:00
Marek Wolan
fad8d936e6 Merge remote-tracking branch 'origin/dev' into feature/2646_Update-pre-commit-to-check-for-valid-copyright 2024-06-13 12:52:09 +01:00
Czar Echavez
1798674d39 #2658: fix space 2024-06-13 12:37:20 +01:00
Czar Echavez
539577ddc3 #2658: added monitored traffic into config + default obs space 2024-06-13 11:48:13 +01:00
Czar Echavez
76e954d1a1 Merge remote-tracking branch 'origin/dev' into feature/1847-more-red-agents 2024-06-11 14:26:11 +01:00
Chris McCarthy
81bcf99855 #2646 - Added a custom pre-commit hook that ensure the copyright clause is added to .py and .rst files. 2024-06-05 09:11:37 +01:00
Marek Wolan
c5f131ece5 fix reward logging 2024-05-31 15:00:18 +01:00
Chris McCarthy
5eea5bf4f9 #2618 - Integrated the NMAP into the action and requests functionality to enable agent usage. added NMAP agents tests. 2024-05-31 13:53:18 +01:00
Czar Echavez
f45df780c9 #2606: correcting the access verb 2024-05-20 15:02:05 +01:00
Czar Echavez
80b1988ab9 #2606: add agent actions + test 2024-05-20 13:10:21 +01:00
Czar Echavez
406d4e889b #2606: pre-commit check 2024-05-17 17:05:14 +01:00
Archer Bowen
13fcda7a26 #2606 - Added initial attempts and created for visibility 2024-05-17 09:28:03 +01:00
Christopher McCarthy
a60e8b2203 Merged PR 349: #2511 - Upgraded pydantic to version 2.7.0
## Summary
- Upgraded pydantic to version 2.7.0.
- Added ipywidgets to the dependencies (for #2300)

## Test process
N/A

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2511 - Upgraded pydantic to version 2.7.0. Added ipywidgets to the dependencies (for #2300)

Related work items: #2300, #2511
2024-04-23 13:43:44 +00:00
Marek Wolan
8e008b6d24 #2511 Appease isort pre commit hook 2024-04-23 13:54:56 +01:00
Chris McCarthy
6060cbbc5b #2511 - Put the ray rlllib dep back to "ray[rllib] == 2.8.0, < 3" 2024-04-23 09:04:17 +01:00
Nick Todd
6726a2da37 #2245: Add comment 2024-04-22 14:09:12 +01:00
Nick Todd
9334b1e79b #2455: Add NICObservation.ConfigSchema to NIC list. 2024-04-19 11:07:32 +01:00
Marek Wolan
72283e6184 #2459 Align whitespace and typing 2024-04-15 12:40:50 +01:00
Marek Wolan
0828f70b4c #2459 back-sync b8 changes into core 2024-04-15 11:50:08 +01:00
Marek Wolan
526dcc7ffe #2450 remove the need to pass Game to observation objects 2024-04-03 22:16:54 +01:00
Marek Wolan
e99d238568 #2450 remove link refs and put nice naming convention instead 2024-04-03 21:39:13 +01:00
Marek Wolan
a4caa3dfe4 #2449 fix observation integration 2024-04-03 15:58:01 +01:00
Marek Wolan
d2c7ae481c #2417 Add categorisation and updated new configs from merge 2024-04-01 22:03:28 +01:00
Marek Wolan
e4300faa1c Merge remote-tracking branch 'origin/dev' into feature/2417-observation-refactor 2024-04-01 19:21:40 +01:00
Marek Wolan
2513646205 #2417 fix last observation tests 2024-04-01 16:50:59 +01:00
Marek Wolan
709486d739 #2417 test firewall and router obs 2024-04-01 16:06:12 +01:00
Cristian-VM2
80c13adfdf #2402 add action maps on all yaml scenarios used for testing 2024-04-01 14:28:41 +00:00
Cristian-VM2
2096b619ec #2402 raise error if action map is not specified. update comment in firewall.py 2024-04-01 12:32:44 +00:00
Marek Wolan
0ba767d2a0 #2417 update observation tests and make old tests pass 2024-04-01 00:54:55 +01:00
Marek Wolan
0e0df1012f #2417 update observations init to autoimport all obs types 2024-03-31 23:39:24 +01:00
Marek Wolan
8da53db822 #2417 Finalise parsing of observation space 2024-03-31 23:20:48 +01:00
Marek Wolan
62ebca8c08 #2417 Remove references to old obs names and add link obs 2024-03-31 21:39:24 +01:00
Marek Wolan
15cb2e6970 #2417 Add NestedObservation 2024-03-31 17:31:10 +01:00
Cristian-VM2
2546f268eb #2402 refactor port actions to take same input params (hostname) as new acl actions for routers and firewalls 2024-03-31 11:59:31 +00:00
Cristian-VM2
8763a27cca Merge remote-tracking branch 'origin/dev' into feature/2402-add-acl-actions-for-routers-and-firewalls 2024-03-29 16:37:37 +00:00
Cristian-VM2
7299a12c64 #2402 add firewall acl actions 2024-03-29 16:30:39 +00:00
Marek Wolan
22e1dfea2f #2417 Move classes to correct files 2024-03-29 14:14:03 +00:00
Marek Wolan
9123aff592 #2417 Add hella docstrings 2024-03-29 13:15:31 +00:00
Marek Wolan
1751714d3d Tidy up node observation file 2024-03-29 12:21:52 +00:00
Marek Wolan
d8a66104f5 Fixed observations 2024-03-29 11:55:22 +00:00
Cristian-VM2
2eb900746b #2402 rename network_acl actions to router_acl and refactor how router_name is given 2024-03-29 11:34:43 +00:00
Marek Wolan
f88b4c0f97 #2417 more observations 2024-03-28 17:40:27 +00:00