DataFusionService didn't show in software list, because it uses the port_protocol_mapping as the source of truth for installed software instead of the software list itself.
This fixes that :)
Related work items: #2899
## Summary
* Ports are no longer enums, they are now plain integers
* Ports are no longer validated against a list of pre-defined ports, any integer from 0-65535 can be used now.
* Port enum was replaced with a convenience port lookup. For example any expression in the form `Port.HTTP` was replaced with `PORT_LOOKUP["HTTP"]` which resolves to the integer 80.
* Some tests were adjusted to use the new syntax for ports
* Backwards compatibility for YAML configs has been retained by adding pydantic validators to automatically convert named ports to integer counterparts, however defining action/observation spaces in code now requires users to specify ports as integers instead of port enum objects. For instance, monitored_ports in the ACL observation space will now look like this: `[53, 80]` instead of `[Port.DNS, Port.HTTP]`
* Plugins can extend the port lookup, however it is not necessary because it's possible to use integer literals.
* Airspace has been treated similarly, except airspace frequencies have multiple attributes, namely max_data_rate_bps. Therefore, the lookup for named Frequencies resolves to a dictionary of freq_hz (float), and max_data_rate_bps (float)
* Airspace logic has been adjusted accordingly to use the new dictionary for tracking bandwidth limits
* A new method for registering new airspace frequencies has been added. Plugins that add new frequencies can call `register_default_frequency`, after which any new airspace will have access to that frequency.
* For consistency, `IPProtocol` was also changed to be a lookup instead of an enum. It is not extendable by plugins as so far we have not needed to model additional protocols in our plugin.
These changes were necessary as it's not possible to extend enums in python, therefore plugins would not have been able to add new ports. There is an added benefit that this is a stepping stone towards support communication on dynamic and ephemeral ports.
## Test process
*How have you tested this (if applicable)?*
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code
## Summary
Added evaluation stage to Training-an-RLLib-Agent and Training-an-RLLIB-MARL-System notebooks.
## Test process
Tested individual notebooks.
Ran all PrimAITE tests.
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
#2844: Added evaluation stage to Ray notebooks.
Related work items: #2844
## Summary
- Added a new Privilege-Escalation-and Data-Loss-Example.ipynb notebook with a more realistic scenario.
- Made some minor changes to multi_lan_internet_network_example.yaml to enable the new scenario.
## Test process
Manual end-to-end run of the notebook.
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
Related work items: #2686
## Summary
Implements the Command and Control applications to the quality and capability needed for the TAP001 expansion and lays the foundations for all the features required for TAP002 (Next Release).
The C2C suite contains three new applications:
**1. Abstract C2**
Base class for the C2 Server and the C2 Beacon. Controls the main internal logic of both applications but with a couple of abstract methods which each class defines differently.
**2. C2 Server**
The C2 Server takes red agent actions and converts the action options into C2 Commands which are then passed to the C2 Beacon.
The output of these commands is sent back to the C2 Server and then returned back to the red agent.
**3. C2 Beacon**
The C2 beacon uses the Terminal and the Ransomware Applications to perform different commands which it receives these commands and executes them and returns the output.
The C2 beacon can also be configured by the Red Agent to configure the current networking behaviour.
For a much more detailed description please refer to the .rst documentation and the notebook which demonstrate and describe the functionality very explicitly.
Lastly the wiki page also provides more information around the design work for this feature.
[Command and Control](/Welcome-to-PrimAITE!/Design/[~In-Progress~]/Command-and-Control)
Worth noting that some changes were needed that were unseen during the design page but the overall goals of the feature have been accomplished.
## Test process
Tested via notebooks and a series of e2e tests.
## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [x] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code (One remaining but unsure if it should be handled in this PR)
Related work items: #2689, #2720, #2721, #2779
## Summary
* Changed how `GreenAdminDatabaseUnreachablePenalty`, `WebpageUnavailablePenalty`, and `WebServer404Penalty` work.
* They can now be configured with `sticky: false` in the yaml
* which means they no longer retain a positive/negative value after a successful/failed request, if the agent goes on to do nothing the next step
* refactored the calculate methods to better align with those rewards depending the previous action
* changed what is returned by some of the `describe_state` methods of sim components. They had legacy methods of returning the most recent success code which is no longer needed since the introduction of agent history
## Test process
Existing tests pass, new tests added
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code
Related work items: #2736
## Summary
This a port of the AgentHistoryItem DB Admin (GreenAdminDatabaseUnreachablePenalty reward) changes that were made to the PrimAITE Internal repo.
See also #2826.
## Test process
Updated tests/integration_tests/game_layer/test_rewards.py.
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
#2748: Port of PrimAITE Internal changes.
Related work items: #2748
## Summary
Adding the Action Space for remote login and remote log out + change password
Updated ray to 2.32.0
## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/484?_a=files&path=/tests/integration_tests/game_layer/actions/user_account_actions/test_remote_user_account_actions.py
## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code
#2769: initial commit of user account actions
Related work items: #2769
