## Summary
* add ability for agents to share rewards - the calculated reward value for one agent can be used as a component of another agent's reward.
* Update UC2 configs to use the reward sharing functionality - green agents have a reward based on their two actions. Blue agent reward adds green agent rewards to itself.
* Add agent action history - This allows the rewards to react to agent actions.
* Make action logging use the new agent history.
* Make the webpage and database reward components treat failed requests the same as if the webpage was unavailable / database was unreachable.
* reorder the PrimaiteGame step to be the same as the Gymnasium env step
* update uc2 notebook accordingly.
## Test process
Tested with ad-hoc notebooks and debugging tool to verify correct data is being used. Tested notebooks run properly and pytests pass. Added unit and integration tests.
## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
Related work items: #2372
## Summary
### **work related to v0.7 CAOS**
- Split observations.py into:
- agent_observations.py
- file_system_observations.py
- node_observations.py
- nic_observations.py
- observation_manager.py
- observations.py
- software_observations.py
- added tests to ensure that the observations align with [QTSL-820-2450 - ARCD Track 2 Common Action Observation Space Definition v0.7](https://nscuk.sharepoint.com/❌/r/sites/SSE32ARCDIDT/Shared%20Documents/General/ARCD/Architecture%20%26%20Design%20Documentation/Common/CAOS%20Related%20Documents/QTSL-820-2450%20-%20ARCD%20Track%202%20Common%20Action%20Observation%20Space%20Definition%20v0.7.xlsx?d=wee5713d8640b4b5bb3cb5624936e417e&csf=1&web=1&e=lByVQ5)
### preparation for v0.8 CAOS
WILL NOT AFFECT OBSERVATION SPACE FOR V0.7
**DO NOT PANIC**
these features are needed for v0.8
- integrated `num_access` to file (not used yet in file observations)
- integrated `num_file_deletions` and `num_file_creations` to file_system (not used yet in node observations)
## Test process
*How have you tested this (if applicable)?*
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
Related work items: #2350
this PR is a pre-requisite to the work im doing today where green agents actions that fail to reach their target still incur a negative reward.
## Summary
- Requests now return a response.
- Added the request response to the agent logging and info output of env step.
- Changed all request types in the simulator to return a response
- Added a new 'validator' that allows nodes to execute actions only if the node is on (this check happens at the request level rather than in every single function)
- Fixed NMNE - made nic observation responsible for converting total NMNE this episode to NMNE last step because apply_timestep is applied after actions but before observations in the step method.
- Fixed arcd_uc2_network to configure the db clients and web browsers correctly
- Added tests for request responses
## Test process
New tests, existing tests pass.
## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code
Related work items: #2372
## Summary
Change the default rewards for the data manipulation configs
Also I renamed the config files to make their names more descriptive.
## Test process
Pytests and all notebooks run without fail.
## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code
Related work items: #2365
## Summary
The NMNE counts MNEs since last step rather than since last episode.
## Test process
Updated unit tests to check for the new behaviour and added new test.
## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code
There's no update necessary to the docs or change log because it NMNE is already documented, and the text is still correct given this minor change.
Related work items: #2348
## Summary
Added a new optional capability to create a JSON log each episode with a list of action each agent took each step (including scripted, RL, and red agents).
Also I had to slightly refactor the IO system to not rely on PrimaiteSession, as it's gonna be deprecated soon. Therefore the IO module is now linked to the gym environment. Each time you init a gym environment, it creates a session directory.
## Test process
Tried the SB3, Ray SARL and Ray MARL notebooks to see that the outputs get generated.
## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code
Move IO to environments from session and add agent logging
Related work items: #2278
## Summary
just run the notebook and see if the explanation makes sense.
I also renamed some stuff to make it more user friendly
Related work items: #2343
