Merged PR 479: merged 3.1.0 into main for git history alignment

merged 3.1.0 into main for git history alignment

Related work items: #1847, #2502, #2610, #2618, #2628, #2646, #2658, #2659, #2660, #2661, #2682

.azure/azure-ci-build-pipeline.yaml

@@ -102,42 +102,16 @@ stages:
               version: '2.1.x'
 
           - script: |
-              python run_test_and_coverage.py
+              coverage run -m --source=primaite pytest -v -o junit_family=xunit2 --junitxml=junit/test-results.xml --cov-fail-under=80
+              coverage xml -o coverage.xml -i
+              coverage html -d htmlcov -i
             displayName: 'Run tests and code coverage'
 
-          # Run the notebooks
-          - script: |
-              pytest --nbmake -n=auto src/primaite/notebooks --junit-xml=./notebook-tests/notebooks.xml
-              notebooks_exit_code=$?
-              pytest --nbmake -n=auto src/primaite/simulator/_package_data --junit-xml=./notebook-tests/package-notebooks.xml
-              package_notebooks_exit_code=$?
-              # Fail step if either of these do not have exit code 0
-              if [ $notebooks_exit_code -ne 0 ] || [ $package_notebooks_exit_code -ne 0 ]; then
-                exit 1
-              fi
-            displayName: 'Run notebooks on Linux and macOS'
-            condition: or(eq(variables['Agent.OS'], 'Linux'), eq(variables['Agent.OS'], 'Darwin'))
-
-          # Run notebooks
-          - script: |
-              pytest --nbmake -n=auto src/primaite/notebooks --junit-xml=./notebook-tests/notebooks.xml
-              set notebooks_exit_code=%ERRORLEVEL%
-              pytest --nbmake -n=auto src/primaite/simulator/_package_data --junit-xml=./notebook-tests/package-notebooks.xml
-              set package_notebooks_exit_code=%ERRORLEVEL%
-              rem Fail step if either of these do not have exit code 0
-              if %notebooks_exit_code% NEQ 0 exit /b 1
-              if %package_notebooks_exit_code% NEQ 0 exit /b 1
-            displayName: 'Run notebooks on Windows'
-            condition: eq(variables['Agent.OS'], 'Windows_NT')
-
           - task: PublishTestResults@2
             condition: succeededOrFailed()
-            displayName: 'Publish Test Results'
             inputs:
               testRunner: JUnit
-              testResultsFiles: |
-                'junit/**.xml'
-                'notebook-tests/**.xml'
+              testResultsFiles: 'junit/**.xml'
               testRunTitle: 'Publish test results'
               failTaskOnFailedTests: true
 

.gitignore

@@ -54,7 +54,6 @@ cover/
 tests/assets/**/*.png
 tests/assets/**/tensorboard_logs/
 tests/assets/**/checkpoints/
-notebook-tests/*.xml
 
 # Translations
 *.mo

CHANGELOG.md

@@ -2,207 +2,240 @@
 
 All notable changes to this project will be documented in this file.
 
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [3.3.0] - 2024-09-04
+## 3.0.0b9
+- Removed deprecated `PrimaiteSession` class.
+- Added ability to set log levels via configuration.
+- Upgraded pydantic to version 2.7.0
+- Upgraded Ray to version >= 2.9
+- Added ipywidgets to the dependencies
+- Added ability to define scenarios that change depending on the episode number.
+- Standardised Environment API by renaming the config parameter of `PrimaiteGymEnv` from `game_config` to `env_config`
+- Database Connection ID's are now created/issued by DatabaseService and not DatabaseClient
+- Updated DatabaseClient so that it can now have a single native DatabaseClientConnection along with a collection of DatabaseClientConnection's.
+- Implemented the uninstall functionality for DatabaseClient so that all connections are terminated at the DatabaseService.
+- Added the ability for a DatabaseService to terminate a connection.
+- Added active_connection to DatabaseClientConnection so that if the connection is terminated active_connection is set to False and the object can no longer be used.
+- Added additional show functions to enable connection inspection.
+- Updates to agent logging, to include the reward both per step and per episode.
+- Introduced Developer CLI tools to assist with developing/debugging PrimAITE
+  - Can be enabled via `primaite dev-mode enable`
+  - Activating dev-mode will change the location where the sessions will be output - by default will output where the PrimAITE repository is located
+- Refactored all air-space usage to that a new instance of AirSpace is created for each instance of Network. This 1:1 relationship between network and airspace will allow parallelization.
+- Added notebook to demonstrate use of SubprocVecEnv from SB3 to vectorise environments to speed up training.
+
+
+
+## [Unreleased]
+- Made requests fail to reach their target if the node is off
+- Added responses to requests
+- Made environment reset completely recreate the game object.
+- Changed the red agent in the data manipulation scenario to randomly choose client 1 or client 2 to start its attack.
+- Changed the data manipulation scenario to include a second green agent on client 1.
+- Refactored actions and observations to be configurable via object name, instead of UUID.
+- Made database patch correctly take 2 timesteps instead of being immediate
+- Made database patch only possible when the software is compromised or good, it's no longer possible when the software is OFF or RESETTING
+- Added a notebook which explains Data manipulation scenario, demonstrates the attack, and shows off blue agent's action space, observation space, and reward function.
+- Made packet capture and system logging optional (off by default). To turn on, change the io_settings.save_pcap_logs and io_settings.save_sys_logs settings in the config.
+- Made observation space flattening optional (on by default). To turn off for an agent, change the `agent_settings.flatten_obs` setting in the config.
+- Added support for SQL INSERT command.
+- Added ability to log each agent's action choices in each step to a JSON file.
+- Removal of Link bandwidth hardcoding. This can now be configured via the network configuraiton yaml. Will default to 100 if not present.
+- Added NMAP application to all host and layer-3 network nodes.
+
+### Bug Fixes
+
+- ACL rules were not resetting on episode reset.
+- ACLs were not showing up correctly in the observation space.
+- Blue agent's ACL actions were being applied against the wrong IP addresses
+- Deleted files and folders did not reset correctly on episode reset.
+- Service health status was using the actual health state instead of the visible health state
+- Database file health status was using the incorrect value for negative rewards
+- Preventing file actions from reaching their intended file
+- The data manipulation attack was triggered at episode start.
+- FTP STOR stored an additional copy on the client machine's filesystem
+- The red agent acted to early
+- Order of service health state
+- Starting a node didn't start the services on it
+- Fixed an issue where the services were still able to run even though the node the service is installed on is turned off
+- The use of NODE_FILE_CHECKHASH and NODE_FOLDER_CHECKHASH in the current release is marked as 'Not Implemented'.
+
 
 ### Added
--   Random Number Generator Seeding by specifying a random number seed in the config file.
--   Implemented Terminal service class, providing a generic terminal simulation.
--   Added `User`, `UserManager` and `UserSessionManager` to enable the creation of user accounts and login on Nodes.
--   Added actions to establish SSH connections, send commands remotely and terminate SSH connections.
--   Added actions to change users' passwords.
--   Added a `listen_on_ports` set in the `IOSoftware` class to enable software listening on ports in addition to the
-    main port they're assigned.
--   Added two new red applications: ``C2Beacon`` and ``C2Server`` which aim to simulate malicious network infrastructure.
-    Refer to the ``Command and Control Application Suite E2E Demonstration`` notebook for more information.
--   Added reward calculation details to AgentHistoryItem.
--   Added a new Privilege-Escalation-and Data-Loss-Example.ipynb notebook with a realistic cyber scenario focusing on
-    internal privilege escalation and data loss through the manipulation of SSH access and Access Control Lists (ACLs).
+- Network Hardware - Added base hardware module with NIC, SwitchPort, Node, and Link. Nodes have
+fundamental services like ARP, ICMP, and PCAP running them by default.
+- Network Transmission - Modelled OSI Model layers 1 through to 5 with various classes for creating network frames and
+transmitting them from a Service/Application, down through the layers, over the wire, and back up through the layers to
+a Service/Application another machine.
+- Introduced `Router` and `Switch` classes to manage networking routes more effectively.
+  - Added `ACLRule` and `RouteTableEntry` classes as part of the `Router`.
+- New `.show()` methods in all network component classes to inspect the state in either plain text or markdown formats.
+- Added `Computer` and `Server` class to better differentiate types of network nodes.
+- Integrated a new Use Case 2 network into the system.
+- New unit tests to verify routing between different subnets using `.ping()`.
+- system - Added the core structure of Application, Services, and Components. Also added a SoftwareManager and
+SessionManager.
+- Permission System - each action can define criteria that will be used to permit or deny agent actions.
+- File System - ability to emulate a node's file system during a simulation
+- Example notebooks - There are 5 jupyter notebook which walk through using PrimAITE
+  1. Training a Stable Baselines 3 agent
+  2. Training a single agent system using Ray RLLib
+  3. Training a multi-agent system Ray RLLib
+  4. Data manipulation end to end demonstration
+  5. Data manipulation scenario with customised red agents
+- Database:
+  - `DatabaseClient` and `DatabaseService` created to allow emulation of database actions
+  - Ability for `DatabaseService` to backup its data to another server via FTP and restore data from backup
+- Red Agent Services:
+  - Data Manipulator Bot - A red agent service which sends a payload to a target machine. (By default this payload is a SQL query that breaks a database). The attack runs in stages with a random, configurable probability of succeeding.
+  - `DataManipulationAgent` runs the Data Manipulator Bot according to a configured start step, frequency and variance.
+- DNS Services: `DNSClient` and `DNSServer`
+- FTP Services: `FTPClient` and `FTPServer`
+- HTTP Services: `WebBrowser` to simulate a web client and `WebServer`
+- NTP Services: `NTPClient` and `NTPServer`
+- **RouterNIC Class**: Introduced a new class `RouterNIC`, extending the standard `NIC` functionality. This class is specifically designed for router operations, optimizing the processing and routing of network traffic.
+  - **Custom Layer-3 Processing**: The `RouterNIC` class includes custom handling for network frames, bypassing standard Node NIC's Layer 3 broadcast/unicast checks. This allows for more efficient routing behavior in network scenarios where router-specific frame processing is required.
+  - **Enhanced Frame Reception**: The `receive_frame` method in `RouterNIC` is tailored to handle frames based on Layer 2 (Ethernet) checks, focusing on MAC address-based routing and broadcast frame acceptance.
+- **Subnet-Wide Broadcasting for Services and Applications**: Implemented the ability for services and applications to conduct broadcasts across an entire IPv4 subnet within the network simulation framework.
+- Introduced the `NetworkInterface` abstract class to provide a common interface for all network interfaces. Subclasses are divided into two main categories: `WiredNetworkInterface` and `WirelessNetworkInterface`, each serving as an abstract base class (ABC) for more specific interface types. Under `WiredNetworkInterface`, the subclasses `NIC` and `SwitchPort` were added. For wireless interfaces, `WirelessNIC` and `WirelessAccessPoint` are the subclasses under `WirelessNetworkInterface`.
+- Added `Layer3Interface` as an abstract base class for networking functionalities at layer 3, including IP addressing and routing capabilities. This class is inherited by `NIC`, `WirelessNIC`, and `WirelessAccessPoint` to provide them with layer 3 capabilities, facilitating their role in both wired and wireless networking contexts with IP-based communication.
+- Created the `ARP` and `ICMP` service classes to handle Address Resolution Protocol operations and Internet Control Message Protocol messages, respectively, with `RouterARP` and `RouterICMP` for router-specific implementations.
+- Created `HostNode` as a subclass of `Node`, extending its functionality with host-specific services and applications. This class is designed to represent end-user devices like computers or servers that can initiate and respond to network communications.
+- Introduced a new `IPV4Address` type in the Pydantic model for enhanced validation and auto-conversion of IPv4 addresses from strings using an `ipv4_validator`.
+- Comprehensive documentation for the Node and its network interfaces, detailing the operational workflow from frame reception to application-level processing.
+- Detailed descriptions of the Session Manager and Software Manager functionalities, including their roles in managing sessions, software services, and applications within the simulation.
+- Documentation for the Packet Capture (PCAP) service and SysLog functionality, highlighting their importance in logging network frames and system events, respectively.
+- Expanded documentation on network devices such as Routers, Switches, Computers, and Switch Nodes, explaining their specific processing logic and protocol support.
+- **Firewall Node**: Introduced the `Firewall` class extending the functionality of the existing `Router` class. The `Firewall` class incorporates advanced features to scrutinize, direct, and filter traffic between various network zones, guided by predefined security rules and policies. Key functionalities include:
+    - Access Control Lists (ACLs) for traffic filtering based on IP addresses, protocols, and port numbers.
+    - Network zone segmentation for managing traffic across external, internal, and DMZ (De-Militarized Zone) networks.
+    - Interface configuration to establish connectivity and define network parameters for external, internal, and DMZ interfaces.
+    - Protocol and service management to oversee traffic and enforce security policies.
+    - Dynamic traffic processing and filtering to ensure network security and integrity.
+- `AirSpace` class to simulate wireless communications, managing wireless interfaces and facilitating the transmission of frames within specified frequencies.
+- `AirSpaceFrequency` enum for defining standard wireless frequencies, including 2.4 GHz and 5 GHz bands, to support realistic wireless network simulations.
+- `WirelessRouter` class, extending the `Router` class, to incorporate wireless networking capabilities alongside traditional wired connections. This class allows the configuration of wireless access points with specific IP settings and operating frequencies.
+- Documentation Updates:
+    - Examples include how to set up PrimAITE session via config
+    - Examples include how to create nodes and install software via config
+    - Examples include how to set up PrimAITE session via Python
+    - Examples include how to create nodes and install software via Python
+    - Added missing ``DoSBot`` documentation page
+    - Added diagrams where needed to make understanding some things easier
+    - Templated parts of the documentation to prevent unnecessary repetition and for easier maintaining of documentation
+    - Separated documentation pages of some items i.e. client and server software were on the same pages - which may make things confusing
+    - Configuration section at the bottom of the software pages specifying the configuration options available (and which ones are optional)
+- Ability to add ``Firewall`` node via config
+- Ability to add ``Router`` routes via config
+- Ability to add ``Router``/``Firewall`` ``ACLRule`` via config
+- NMNE capturing capabilities to `NetworkInterface` class for detecting and logging Malicious Network Events.
+- New `nmne_config` settings in the simulation configuration to enable NMNE capturing and specify keywords such as "DELETE".
+- Router-specific SessionManager Implementation: Introduced a specialized version of the SessionManager tailored for router operations. This enhancement enables the SessionManager to determine the routing path by consulting the route table.
 
 ### Changed
--   File and folder observations can now be configured to always show the true health status, or require scanning like before.
--   It's now possible to disable stickiness on reward components, meaning their value returns to 0 during timesteps where agent don't issue the corresponding action. Affects `GreenAdminDatabaseUnreachablePenalty`, `WebpageUnavailablePenalty`, `WebServer404Penalty`
--   Node observations can now be configured to show the number of active local and remote logins.
-
-### Fixed
--   Folder observations showing the true health state without scanning (the old behaviour can be reenabled via config)
--   Updated `SoftwareManager` `install` and `uninstall` to handle all functionality that was being done at the `install`
-    and `uninstall` methods in the `Node` class.
--   Updated the `receive_payload_from_session_manager` method in `SoftwareManager` so that it now sends a copy of the
-    payload to any software listening on the destination port of the `Frame`.
+- Integrated the RouteTable into the Routers frame processing.
+- Frames are now dropped when their TTL reaches 0
+- **NIC Functionality Update**: Updated the Network Interface Card (`NIC`) functionality to support Layer 3 (L3) broadcasts.
+  - **Layer 3 Broadcast Handling**: Enhanced the existing `NIC` classes to correctly process and handle Layer 3 broadcasts. This update allows devices using standard NICs to effectively participate in network activities that involve L3 broadcasting.
+  - **Improved Frame Reception Logic**: The `receive_frame` method of the `NIC` class has been updated to include additional checks and handling for L3 broadcasts, ensuring proper frame processing in a wider range of network scenarios.
+- Standardised the way network interfaces are accessed across all `Node` subclasses (`HostNode`, `Router`, `Switch`) by maintaining a comprehensive `network_interface` attribute. This attribute captures all network interfaces by their port number, streamlining the management and interaction with network interfaces across different types of nodes.
+- Refactored all tests to utilise new `Node` subclasses (`Computer`, `Server`, `Router`, `Switch`) instead of creating generic `Node` instances and manually adding network interfaces. This change aligns test setups more closely with the intended use cases and hierarchies within the network simulation framework.
+- Updated all tests to employ the `Network()` class for managing nodes and their connections, ensuring a consistent and structured approach to setting up network topologies in testing scenarios.
+- **ACLRule Wildcard Masking**: Updated the `ACLRule` class to support IP ranges using wildcard masking. This enhancement allows for more flexible and granular control over traffic filtering, enabling the specification of broader or more specific IP address ranges in ACL rules.
+- Updated `NetworkInterface` documentation to reflect the new NMNE capturing features and how to use them.
+- Integration of NMNE capturing functionality within the `NICObservation` class.
+- Changed blue action set to enable applying node scan, reset, start, and shutdown to every host in data manipulation scenario
 
 ### Removed
--   Removed the `install` and `uninstall` methods in the `Node` class.
-
-
-## [3.2.0] - 2024-07-18
-
-### Added
--   Action penalty is a reward component that applies a negative reward for doing any action other than DONOTHING
--   Application configuration actions for RansomwareScript, DatabaseClient, and DoSBot applications
--   Ability to configure how long it takes to apply the service fix action
--   Terminal service using SSH
--   Airspaces now track the amount of data being transmitted, viewable using the `show_bandwidth_load` method
--   Tests to verify that airspace bandwidth is applied correctly and can be configured via YAML
--   Agent logging for agents' internal decision logic
--   Action masking in all PrimAITE environments
-### Changed
--   Application registry was moved to the `Application` class and now updates automatically when Application is subclassed
--   Databases can no longer respond to request while performing a backup
--   Application install no longer accepts an `ip_address` parameter
--   Application install action can now be used on all applications
--   Actions have additional logic for checking validity
--   Frame `size` attribute now includes both core size and payload size in bytes
--   The `speed` attribute of `NetworkInterface` has been changed from `int` to `float`
--   Tidied up CHANGELOG
--   Enhanced `AirSpace` logic to block transmissions that would exceed the available capacity.
--   Updated `_can_transmit` function in `Link` to account for current load and total bandwidth capacity, ensuring transmissions do not exceed limits.
+- Removed legacy simulation modules: `acl`, `common`, `environment`, `links`, `nodes`, `pol`
+- Removed legacy training modules
+- Removed tests for legacy code
 
 ### Fixed
--   Links and airspaces can no longer transmit data if this would exceed their bandwidth
-
-
-## [3.1.0] - 2024-06-25
-
-### Added
--   Observations for traffic amounts on host network interfaces
--   NMAP application network discovery, including ping scan and port scan
--   NMAP actions
--   Automated adding copyright notices to source files
--   More file types
--   `show` method to files
--   `model_dump` methods to network enums to enable better logging
-
-### Changed
--   Updated file system actions to stop failures when creating duplicate files
--   Improved parsing of ACL add rule actions to make some parameters optional
-
-### Fixed
--   Fixed database client uninstall failing due to persistent connections
--   Fixed packet storm when pinging broadcast addresses
-
-
-## [3.0.0] - 2024-06-10
-
-### Added
--   New simulation module
--   Multi agent reinforcement learning support
--   File system class to manage files and folders
--   Software for nodes that can have its own behaviour
--   Software classes to model FTP, Postgres databases, web traffic, NTP
--   Much more detailed network simulation including packets, links, and network interfaces
--   More node types: host, computer, server, router, switch, wireless router, and firewalls
--   Network Hardware - NIC, SwitchPort, Node, and Link. Nodes have fundamental services like ARP, ICMP, and PCAP running them by default.
--   Malicious network event detection
--   New `game` module for managing agents
--   ACL rule wildcard masking
--   Network broadcasting
--   Wireless transmission
--   More detailed documentation
--   Example jupyter notebooks to demonstrate new functionality
--   More reward components
--   Packet capture logs
--   Node system logs
--   Per-step full simulation state log
--   Attack randomisation with respect to timing and attack source
--   Ability to set log level via CLI
--   Ability to vary the YAML configuration per-episode
--   Developer CLI tools for enhanced debugging (with `primaite dev-mode enable`)
--   `show` function to many simulation objects to inspect their current state
-
-### Changed
--   Decoupled the environment from the simulation by adding the `game` interface layer
--   Made agents share a common base class
--   Added more actions
--   Made all agents use CAOS actions, including red and green agents
--   Reworked YAML configuration file schema
--   Reworked the reward system to be component-based
--   Changed agent logs to create a JSON output instead of CSV with more detailed action information
--   Made observation space flattening optional
--   Made all logging optional
--   Agent actions now provide responses with a success code
-
-### Removed
--   Legacy simulation modules
--   Legacy training modules
--   Tests for legacy code
--   Hardcoded IERs and PoL, traffic generation is now handled by agents and software
--   Inbuilt agent training scripts
-
+- Addressed network transmission issues that previously allowed ARP requests to be incorrectly routed and repeated across different subnets. This fix ensures ARP requests are correctly managed and confined to their appropriate network segments.
+- Resolved problems in `Node` and its subclasses where the default gateway configuration was not properly utilized for communications across different subnets. This correction ensures that nodes effectively use their configured default gateways for outbound communications to other network segments, thereby enhancing the network's routing functionality and reliability.
+- Network Interface Port name/num being set properly for sys log and PCAP output.
 
 ## [2.0.0] - 2023-07-26
 
 ### Added
--   Command Line Interface (CLI) for easy access and streamlined usage of PrimAITE.
--   Application Directories to enable PrimAITE as a Python package with predefined directories for storage.
--   Support for Ray Rllib, allowing training of PPO and A2C agents using Stable Baselines3 and Ray RLlib.
--   Random Red Agent to train the blue agent against, with options for randomised Red Agent `POL` and `IER`.
--   Repeatability of sessions through seed settings, and deterministic or stochastic evaluation options.
--   Session loading to revisit previously run sessions for SB3 Agents.
--   Agent Session Classes (`AgentSessionABC` and `HardCodedAgentSessionABC`) to standardise agent training with a common interface.
--   Standardised Session Output in a structured format in the user's app sessions directory, providing four types of outputs: Session Metadata, Results, Diagrams, Trained agents.
--   Configurable Observation Space managed by the `ObservationHandler` class for a more flexible observation space setup.
--   Benchmarking of PrimAITE performance, showcasing session and step durations for reference.
--   Documentation overhaul, including automatic API and test documentation with recursive Sphinx auto-summary, using the Furo theme for responsive light/dark theme, and enhanced navigation with `sphinx-code-tabs` and `sphinx-copybutton`.
+- Command Line Interface (CLI) for easy access and streamlined usage of PrimAITE.
+- Application Directories to enable PrimAITE as a Python package with predefined directories for storage.
+- Support for Ray Rllib, allowing training of PPO and A2C agents using Stable Baselines3 and Ray RLlib.
+- Random Red Agent to train the blue agent against, with options for randomised Red Agent `POL` and `IER`.
+- Repeatability of sessions through seed settings, and deterministic or stochastic evaluation options.
+- Session loading to revisit previously run sessions for SB3 Agents.
+- Agent Session Classes (`AgentSessionABC` and `HardCodedAgentSessionABC`) to standardise agent training with a common interface.
+- Standardised Session Output in a structured format in the user's app sessions directory, providing four types of outputs:
+  1. Session Metadata
+  2. Results
+  3. Diagrams
+  4. Saved agents (training checkpoints and a final trained agent).
+- Configurable Observation Space managed by the `ObservationHandler` class for a more flexible observation space setup.
+- Benchmarking of PrimAITE performance, showcasing session and step durations for reference.
+- Documentation overhaul, including automatic API and test documentation with recursive Sphinx auto-summary, using the Furo theme for responsive light/dark theme, and enhanced navigation with `sphinx-code-tabs` and `sphinx-copybutton`.
 
 ### Changed
--   Action Space updated to discrete spaces, introducing a new `ANY` action space option for combined `NODE` and `ACL` actions.
--   Improved `Node` attribute naming convention for consistency, now adhering to `Pascal Case`.
--   Package Structure has been refactored for better build, distribution, and installation, with all source code now in the `src/` directory, and the `PRIMAITE` Python package renamed to `primaite` to adhere to PEP-8 Package & Module Names.
--   Docs and Tests now sit outside the `src/` directory.
--   Non-python files (example config files, Jupyter notebooks, etc.) now sit inside a `*/_package_data/` directory in their respective sub-packages.
--   All dependencies are now defined in the `pyproject.toml` file.
--   Introduced individual configuration for the number of episodes and time steps for training and evaluation sessions, with separate config values for each.
--   Decoupled the lay down config file from the training config, allowing more flexibility in configuration management.
--   Updated `Transactions` to only report pre-action observation, improving the CSV header and providing more human-readable descriptions for columns relating to observations.
--   Changes to `AccessControlList`, where the `acl` dictionary is now a list to accommodate changes to ACL action space and positioning of `ACLRules` inside the list to signal their level of priority.
+- Action Space updated to discrete spaces, introducing a new `ANY` action space option for combined `NODE` and `ACL` actions.
+- Improved `Node` attribute naming convention for consistency, now adhering to `Pascal Case`.
+- Package Structure has been refactored for better build, distribution, and installation, with all source code now in the `src/` directory, and the `PRIMAITE` Python package renamed to `primaite` to adhere to PEP-8 Package & Module Names.
+- Docs and Tests now sit outside the `src/` directory.
+- Non-python files (example config files, Jupyter notebooks, etc.) now sit inside a `*/_package_data/` directory in their respective sub-packages.
+- All dependencies are now defined in the `pyproject.toml` file.
+- Introduced individual configuration for the number of episodes and time steps for training and evaluation sessions, with separate config values for each.
+- Decoupled the lay down config file from the training config, allowing more flexibility in configuration management.
+- Updated `Transactions` to only report pre-action observation, improving the CSV header and providing more human-readable descriptions for columns relating to observations.
+- Changes to `AccessControlList`, where the `acl` dictionary is now a list to accommodate changes to ACL action space and positioning of `ACLRules` inside the list to signal their level of priority.
 
 
 ### Fixed
--   Various bug fixes, including Green IERs separation, correct clearing of links in the reference environment, and proper reward calculation.
--   Logic to check if a node is OFF before executing actions on the node by the blue agent, preventing erroneous state changes.
--   Improved functionality of Resetting a Node, adding "SHUTTING DOWN" and "BOOTING" operating states for more reliable reset commands.
--   Corrected the order of actions in the `Primaite` env to ensure the blue agent uses the current state for decision-making.
-
+- Various bug fixes, including Green IERs separation, correct clearing of links in the reference environment, and proper reward calculation.
+- Logic to check if a node is OFF before executing actions on the node by the blue agent, preventing erroneous state changes.
+- Improved functionality of Resetting a Node, adding "SHUTTING DOWN" and "BOOTING" operating states for more reliable reset commands.
+- Corrected the order of actions in the `Primaite` env to ensure the blue agent uses the current state for decision-making.
 
 ## [1.1.1] - 2023-06-27
 
-### Fixed
--   Fixed bug whereby 'reference' environment links reach bandwidth capacity and are never cleared due to green & red IERs being applied to them. This bug had a knock-on effect that meant IERs were being blocked based on the full capacity of links on the reference environment which was not correct; they should only be based on the link capacity of the 'live' environment. This fix has been addressed by:
-    -   Implementing a reference copy of all green IERs (`self.green_iers_reference`).
-    -   Clearing the traffic on reference IERs at the same time as the live IERs.
-    -   Passing the `green_iers_reference` to the `apply_iers` function at the reference stage.
-    -   Passing the `green_iers_reference` as an additional argument to `calculate_reward_function`.
-    -   Updating the green IERs section of the `calculate_reward_function` to now take into account both the green reference IERs and live IERs. The `green_ier_blocked` reward is only applied if the IER is blocked in the live environment but is running in the reference environment.
-    -   Re-ordering the actions taken as part of the step function to ensure the blue action happens first before other changes.
-    -   Removing the unnecessary "Reapply PoL and IERs" action from the step function.
-    -   Moving the deep-copy of nodes and links to below the "Implement blue action" stage of the step function.
-
+### Bug Fixes
+* Fixed bug whereby 'reference' environment links reach bandwidth capacity and are never cleared due to green & red IERs being applied to them. This bug had a knock-on effect that meant IERs were being blocked based on the full capacity of links on the reference environment which was not correct; they should only be based on the link capacity of the 'live' environment. This fix has been addressed by:
+  * Implementing a reference copy of all green IERs (`self.green_iers_reference`).
+  * Clearing the traffic on reference IERs at the same time as the live IERs.
+  * Passing the `green_iers_reference` to the `apply_iers` function at the reference stage.
+  * Passing the `green_iers_reference` as an additional argument to `calculate_reward_function`.
+  * Updating the green IERs section of the `calculate_reward_function` to now take into account both the green reference IERs and live IERs. The `green_ier_blocked` reward is only applied if the IER is blocked in the live environment but is running in the reference environment.
+  * Re-ordering the actions taken as part of the step function to ensure the blue action happens first before other changes.
+  * Removing the unnecessary "Reapply PoL and IERs" action from the step function.
+  * Moving the deep-copy of nodes and links to below the "Implement blue action" stage of the step function.
 
 ## [1.1.0] - 2023-03-13
 
 ### Added
--   The user can now initiate either a TRAINING session or an EVALUATION (test) session with the Stable Baselines 3 (SB3) agents via the config_main.yaml file. During evaluation/testing, the agent policy will be fixed (no longer learning) and subjected to the SB3 `evaluate_policy()` function.
--   The user can choose whether a saved agent is loaded into the session (with reference to a URL) via the `config_main.yaml` file. They specify a Boolean true/false indicating whether a saved agent should be loaded, and specify the URL and file name.
--   Active and Service nodes now possess a new "File System State" attribute. This attribute is permitted to have the states GOOD, CORRUPT, DESTROYED, REPAIRING, and RESTORING. This new feature affects the following components:
-    -   Blue agent observation space;
-    -   Blue agent action space;
-    -   Reward function;
-    -   Node pattern-of-life.
--   The Red Agent node pattern-of-life has been enhanced so that node PoL is triggered by an 'initiator'. The initiator is either DIRECT (state change is applied to the node without any conditions), IER (state change is applied to the node based on IER entry condition), or SERVICE (state change is applied to the node based on a service state condition on the same node or a different node within the network).
--   New default config named "config_5_DATA_MANIPULATION.yaml" and associated Training Use Case Profile.
--   NodeStateInstruction has been split into `NodeStateInstructionGreen` and `NodeStateInstructionRed` to reflect the changes within the red agent pattern-of-life capability.
--   The reward function has been enhanced so that node attribute states of resetting, patching, repairing, and restarting contribute to the overall reward value.
--   The User Guide has been updated to reflect all the above changes.
+* The user can now initiate either a TRAINING session or an EVALUATION (test) session with the Stable Baselines 3 (SB3) agents via the config_main.yaml file. During evaluation/testing, the agent policy will be fixed (no longer learning) and subjected to the SB3 `evaluate_policy()` function.
+* The user can choose whether a saved agent is loaded into the session (with reference to a URL) via the `config_main.yaml` file. They specify a Boolean true/false indicating whether a saved agent should be loaded, and specify the URL and file name.
+* Active and Service nodes now possess a new "File System State" attribute. This attribute is permitted to have the states GOOD, CORRUPT, DESTROYED, REPAIRING, and RESTORING. This new feature affects the following components:
+  * Blue agent observation space;
+  * Blue agent action space;
+  * Reward function;
+  * Node pattern-of-life.
+* The Red Agent node pattern-of-life has been enhanced so that node PoL is triggered by an 'initiator'. The initiator is either DIRECT (state change is applied to the node without any conditions), IER (state change is applied to the node based on IER entry condition), or SERVICE (state change is applied to the node based on a service state condition on the same node or a different node within the network).
+* New default config named "config_5_DATA_MANIPULATION.yaml" and associated Training Use Case Profile.
+* NodeStateInstruction has been split into `NodeStateInstructionGreen` and `NodeStateInstructionRed` to reflect the changes within the red agent pattern-of-life capability.
+* The reward function has been enhanced so that node attribute states of resetting, patching, repairing, and restarting contribute to the overall reward value.
+* The User Guide has been updated to reflect all the above changes.
 
 ### Changed
--   "config_1_DDOS_BASIC.yaml" modified to make it more simplistic to aid evaluation testing.
--   "config_2_DDOS_BASIC.yaml" updated to reflect the addition of the File System State and the Red Agent node pattern-of-life enhancement.
--   "config_3_DOS_VERY_BASIC.yaml" updated to reflect the addition of the File System State and the Red Agent node pattern-of-life enhancement.
--   "config_UNIT_TEST.yaml" is a copy of the new "config_5_DATA_MANIPULATION.yaml" file.
--   Updates to Transactions.
+* "config_1_DDOS_BASIC.yaml" modified to make it more simplistic to aid evaluation testing.
+* "config_2_DDOS_BASIC.yaml" updated to reflect the addition of the File System State and the Red Agent node pattern-of-life enhancement.
+* "config_3_DOS_VERY_BASIC.yaml" updated to reflect the addition of the File System State and the Red Agent node pattern-of-life enhancement.
+* "config_UNIT_TEST.yaml" is a copy of the new "config_5_DATA_MANIPULATION.yaml" file.
+* Updates to Transactions.
 
 ### Fixed
--   Fixed "config_2_DDOS_BASIC.yaml" by adding another ACL rule to allow traffic to flow from Node 9 to Node 3. Previously, there was no rule, so one of the green IERs could not flow by default.
+* Fixed "config_2_DDOS_BASIC.yaml" by adding another ACL rule to allow traffic to flow from Node 9 to Node 3. Previously, there was no rule, so one of the green IERs could not flow by default.
+
+
+
+[unreleased]: https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/compare/v2.0.0...HEAD
+[2.0.0]: https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/releases/tag/v2.0.0

CONTRIBUTING.md

@@ -13,6 +13,9 @@
 * [Fork the repository](https://github.com/{todo:fill in URL}/PrimAITE/fork).
 * Install the pre-commit hook with `pre-commit install`.
 * Implement the bug fix.
+* Update documentation where applicable.
+* Update the **UNRELEASED** section of the [CHANGELOG.md](CHANGELOG.md) file
+* Write a suitable test/tests.
 * Commit the bug fix to the dev branch on your fork. If the bug has an open issue under [Issues](https://github.com/{todo:fill in URL}/PrimAITE/issues), reference the issue in the commit message (e.g. #1 references issue 1).
 * Submit a pull request from your dev branch to the {todo:fill in URL}/PrimAITE dev branch. Again, if the bug has an open issue under [Issues](https://github.com/{todo:fill in URL}/PrimAITE/issues), reference the issue in the pull request description.
 

README.md

@@ -24,8 +24,6 @@ PrimAITE presents the following features:
 
 - Support for multiple agents, each having their own customisable observation space, action space, and reward function definition, and either deterministic or RL-directed behaviour
 
-Whilst PrimAITE ships with a number of example modelled scenarios (a.k.a. Use Cases), it has not been developed to mandate the solving of a single cyber challenge, and instead provides a highly flexible environment application that can be extended and reconfigured by the user to suit their specific cyber defence training and evaluation needs. PrimAITE provides default networks, red agent and green agent behaviour, reward functions, and action / observation space configuration, all of which can be utilised out of the box, but which ultimately can (and in some instances should) be built upon and / or reconfigured to meet the needs of different defensive agent developers. The PrimAITE user guide provides comprehensive instruction on all PrimAITE features, functionality and components, and can be consulted in order to help guide users in any reconfiguration or enhancements they wish to undertake; a library of example Jupyter notebooks are also provided to support such work.
-
 ## Getting Started with PrimAITE
 
 ### 💫 Installation

_config.yml

@@ -1,3 +0,0 @@
-# Used by nbmake to change build pipeline notebook timeout
-execute:
-  timeout: 600

benchmark/primaite_benchmark.py

@@ -5,7 +5,7 @@ from datetime import datetime
 from pathlib import Path
 from typing import Any, Dict, Final, Tuple
 
-from report import build_benchmark_md_report, md2pdf
+from report import build_benchmark_md_report
 from stable_baselines3 import PPO
 
 import primaite
@@ -159,13 +159,6 @@ def run(
     learning_rate: float = 3e-4,
 ) -> None:
     """Run the PrimAITE benchmark."""
-    # generate report folder
-    v_str = f"v{primaite.__version__}"
-
-    version_result_dir = _RESULTS_ROOT / v_str
-    version_result_dir.mkdir(exist_ok=True, parents=True)
-    output_path = version_result_dir / f"PrimAITE {v_str} Benchmark Report.md"
-
     benchmark_start_time = datetime.now()
 
     session_metadata_dict = {}
@@ -200,12 +193,6 @@ def run(
         session_metadata=session_metadata_dict,
         config_path=data_manipulation_config_path(),
         results_root_path=_RESULTS_ROOT,
-        output_path=output_path,
-    )
-    md2pdf(
-        md_path=output_path,
-        pdf_path=str(output_path).replace(".md", ".pdf"),
-        css_path="static/styles.css",
     )
 
 

benchmark/report.py

@@ -2,7 +2,6 @@
 import json
 import sys
 from datetime import datetime
-from os import PathLike
 from pathlib import Path
 from typing import Dict, Optional
 
@@ -15,9 +14,11 @@ from utils import _get_system_info
 import primaite
 
 PLOT_CONFIG = {
-    "size": {"auto_size": False, "width": 800, "height": 640},
+    "size": {"auto_size": False, "width": 1500, "height": 900},
     "template": "plotly_white",
     "range_slider": False,
+    "av_s_per_100_steps_10_nodes_benchmark_threshold": 5,
+    "benchmark_line_color": "grey",
 }
 
 
@@ -145,20 +146,6 @@ def _plot_benchmark_metadata(
         yaxis={"title": "Total Reward"},
         title=title,
     )
-    fig.update_layout(
-        legend=dict(
-            yanchor="top",
-            y=0.99,
-            xanchor="left",
-            x=0.01,
-            bgcolor="rgba(255,255,255,0.3)",
-        )
-    )
-    for trace in fig["data"]:
-        if trace["name"].startswith("Session"):
-            trace["showlegend"] = False
-    fig["data"][0]["name"] = "Individual Sessions"
-    fig["data"][0]["showlegend"] = True
 
     return fig
 
@@ -209,7 +196,6 @@ def _plot_all_benchmarks_combined_session_av(results_directory: Path) -> Figure:
         title=title,
     )
     fig["data"][0]["showlegend"] = True
-    fig.update_layout(legend=dict(yanchor="top", y=-0.2, xanchor="left", x=0.01, orientation="h"))
 
     return fig
 
@@ -243,14 +229,20 @@ def _plot_av_s_per_100_steps_10_nodes(
     """
     Creates a bar chart visualising the performance of each version of PrimAITE.
 
-    Performance is based on the average training time per 100 steps on 10 nodes.
+    Performance is based on the average training time per 100 steps on 10 nodes. The function also includes a benchmark
+    line indicating the target maximum time.
+
+    Versions that perform under this time are marked in green, and those over are marked in red.
 
     :param version_times_dict: A dictionary with software versions as keys and average times as values.
     :return: A Plotly figure object representing the bar chart of the performance metrics.
     """
     major_v = primaite.__version__.split(".")[0]
     title = f"Performance of Minor and Bugfix Releases for Major Version {major_v}"
-    subtitle = "Average Training Time per 100 Steps on 10 Nodes "
+    subtitle = (
+        f"Average Training Time per 100 Steps on 10 Nodes "
+        f"(target: <= {PLOT_CONFIG['av_s_per_100_steps_10_nodes_benchmark_threshold']} seconds)"
+    )
     title = f"{title} <br><sub>{subtitle}</sub>"
 
     layout = go.Layout(
@@ -263,12 +255,42 @@ def _plot_av_s_per_100_steps_10_nodes(
 
     versions = sorted(list(version_times_dict.keys()))
     times = [version_times_dict[version] for version in versions]
+    av_s_per_100_steps_10_nodes_benchmark_threshold = PLOT_CONFIG["av_s_per_100_steps_10_nodes_benchmark_threshold"]
+    benchmark_line_color = PLOT_CONFIG["benchmark_line_color"]
 
-    fig.add_trace(go.Bar(x=versions, y=times, text=times, textposition="auto", texttemplate="%{y:.3f}"))
+    # Calculate the appropriate maximum y-axis value
+    max_y_axis_value = max(max(times), av_s_per_100_steps_10_nodes_benchmark_threshold) + 1
+
+    fig.add_trace(
+        go.Bar(
+            x=versions,
+            y=times,
+            marker_color=[
+                "green" if time < av_s_per_100_steps_10_nodes_benchmark_threshold else "red" for time in times
+            ],
+            text=times,
+            textposition="auto",
+        )
+    )
+
+    # Add a horizontal line for the benchmark
+    fig.add_shape(
+        type="line",
+        x0=-0.5,  # start slightly before the first bar
+        x1=len(versions) - 0.5,  # end slightly after the last bar
+        y0=av_s_per_100_steps_10_nodes_benchmark_threshold,
+        y1=av_s_per_100_steps_10_nodes_benchmark_threshold,
+        line=dict(
+            color=benchmark_line_color,
+            width=2,
+            dash="dot",
+        ),
+    )
 
     fig.update_layout(
         xaxis_title="PrimAITE Version",
         yaxis_title="Avg Time per 100 Steps on 10 Nodes (seconds)",
+        yaxis=dict(range=[0, max_y_axis_value]),
         title=title,
     )
 
@@ -276,11 +298,7 @@ def _plot_av_s_per_100_steps_10_nodes(
 
 
 def build_benchmark_md_report(
-    benchmark_start_time: datetime,
-    session_metadata: Dict,
-    config_path: Path,
-    results_root_path: Path,
-    output_path: PathLike,
+    benchmark_start_time: datetime, session_metadata: Dict, config_path: Path, results_root_path: Path
 ) -> None:
     """
     Generates a Markdown report for a benchmarking session, documenting performance metrics and graphs.
@@ -332,7 +350,7 @@ def build_benchmark_md_report(
     data = benchmark_metadata_dict
     primaite_version = data["primaite_version"]
 
-    with open(output_path, "w") as file:
+    with open(version_result_dir / f"PrimAITE v{primaite_version} Benchmark Report.md", "w") as file:
         # Title
         file.write(f"# PrimAITE v{primaite_version} Learning Benchmark\n")
         file.write("## PrimAITE Dev Team\n")
@@ -406,15 +424,3 @@ def build_benchmark_md_report(
             f"![Performance of Minor and Bugfix Releases for Major Version {major_v}]"
             f"({performance_benchmark_plot_path.name})\n"
         )
-
-
-def md2pdf(md_path: PathLike, pdf_path: PathLike, css_path: PathLike) -> None:
-    """Generate PDF version of Markdown report."""
-    from md2pdf.core import md2pdf
-
-    md2pdf(
-        pdf_file_path=pdf_path,
-        md_file_path=md_path,
-        base_url=Path(md_path).parent,
-        css_file_path=css_path,
-    )

benchmark/results/v3/v3.0.0/PrimAITE v3.0.0 Benchmark Report.md

@@ -1,10 +1,10 @@
-# PrimAITE v3.2.0 Learning Benchmark
+# PrimAITE v3.0.0 Learning Benchmark
 ## PrimAITE Dev Team
-### 2024-07-21
+### 2024-07-20
 
 ---
 ## 1 Introduction
-PrimAITE v3.2.0 was benchmarked automatically upon release. Learning rate metrics were captured to be referenced during system-level testing and user acceptance testing (UAT).
+PrimAITE v3.0.0 was benchmarked automatically upon release. Learning rate metrics were captured to be referenced during system-level testing and user acceptance testing (UAT).
 The benchmarking process consists of running 5 training session using the same config file. Each session trains an agent for 1000 episodes, with each episode consisting of 128 steps.
 The total reward per episode from each session is captured. This is then used to calculate an caverage total reward per episode from the 5 individual sessions for smoothing. Finally, a 25-widow rolling average of the average total reward per session is calculated for further smoothing.
 ## 2 System Information
@@ -26,12 +26,12 @@ The total reward per episode from each session is captured. This is then used to
 - **Total Sessions:** 5
 - **Total Episodes:** 5005
 - **Total Steps:** 640000
-- **Av Session Duration (s):** 1691.5034
-- **Av Step Duration (s):** 0.0529
-- **Av Duration per 100 Steps per 10 Nodes (s):** 5.2859
+- **Av Session Duration (s):** 1452.5910
+- **Av Step Duration (s):** 0.0454
+- **Av Duration per 100 Steps per 10 Nodes (s):** 4.5393
 ## 4 Graphs
-### 4.1 v3.2.0 Learning Benchmark Plot
-![PrimAITE 3.2.0 Learning Benchmark Plot](PrimAITE v3.2.0 Learning Benchmark.png)
+### 4.1 v3.0.0 Learning Benchmark Plot
+![PrimAITE 3.0.0 Learning Benchmark Plot](PrimAITE v3.0.0 Learning Benchmark.png)
 ### 4.2 Learning Benchmark of Minor and Bugfix Releases for Major Version 3
 ![Learning Benchmark of Minor and Bugfix Releases for Major Version 3](PrimAITE Learning Benchmark of Minor and Bugfix Releases for Major Version 3.png)
 ### 4.3 Performance of Minor and Bugfix Releases for Major Version 3

benchmark/results/v3/v3.1.0/session_metadata/1.json

@@ -1006,4 +1006,4 @@
         "999": 112.00000000000009,
         "1000": 102.55000000000008
     }
-}
+}

benchmark/results/v3/v3.1.0/session_metadata/2.json

@@ -1006,4 +1006,4 @@
         "999": 110.30000000000007,
         "1000": 118.05000000000011
     }
-}
+}

benchmark/results/v3/v3.1.0/session_metadata/3.json

@@ -1006,4 +1006,4 @@
         "999": 118.44999999999996,
         "1000": 108.20000000000029
     }
-}
+}

benchmark/results/v3/v3.1.0/session_metadata/4.json

@@ -1006,4 +1006,4 @@
         "999": 99.50000000000009,
         "1000": 98.70000000000006
     }
-}
+}

benchmark/results/v3/v3.1.0/session_metadata/5.json

@@ -1006,4 +1006,4 @@
         "999": 118.45000000000007,
         "1000": 116.3000000000001
     }
-}
+}

benchmark/results/v3/v3.1.0/v3.1.0_benchmark_metadata.json

@@ -7441,4 +7441,4 @@
             }
         }
     }
-}
+}

benchmark/results/v3/v3.3.0/PrimAITE v3.3.0 Benchmark Report.md

@@ -1,38 +0,0 @@
-# PrimAITE v3.3.0 Learning Benchmark
-## PrimAITE Dev Team
-### 2024-09-02
-
----
-## 1 Introduction
-PrimAITE v3.3.0 was benchmarked automatically upon release. Learning rate metrics were captured to be referenced during system-level testing and user acceptance testing (UAT).
-The benchmarking process consists of running 5 training session using the same config file. Each session trains an agent for 1000 episodes, with each episode consisting of 128 steps.
-The total reward per episode from each session is captured. This is then used to calculate an caverage total reward per episode from the 5 individual sessions for smoothing. Finally, a 25-widow rolling average of the average total reward per session is calculated for further smoothing.
-## 2 System Information
-### 2.1 Python
-**Version:** 3.10.14 (main, Apr  6 2024, 18:45:05) [GCC 9.4.0]
-### 2.2 System
-- **OS:** Linux
-- **OS Version:** #76~20.04.1-Ubuntu SMP Thu Jun 13 18:00:23 UTC 2024
-- **Machine:** x86_64
-- **Processor:** x86_64
-### 2.3 CPU
-- **Physical Cores:** 2
-- **Total Cores:** 4
-- **Max Frequency:** 0.00Mhz
-### 2.4 Memory
-- **Total:** 15.62GB
-- **Swap Total:** 0.00B
-## 3 Stats
-- **Total Sessions:** 5
-- **Total Episodes:** 5005
-- **Total Steps:** 640000
-- **Av Session Duration (s):** 1458.2831
-- **Av Step Duration (s):** 0.0456
-- **Av Duration per 100 Steps per 10 Nodes (s):** 4.5571
-## 4 Graphs
-### 4.1 v3.3.0 Learning Benchmark Plot
-![PrimAITE 3.3.0 Learning Benchmark Plot](PrimAITE v3.3.0 Learning Benchmark.png)
-### 4.2 Learning Benchmark of Minor and Bugfix Releases for Major Version 3
-![Learning Benchmark of Minor and Bugfix Releases for Major Version 3](PrimAITE Learning Benchmark of Minor and Bugfix Releases for Major Version 3.png)
-### 4.3 Performance of Minor and Bugfix Releases for Major Version 3
-![Performance of Minor and Bugfix Releases for Major Version 3](PrimAITE Performance of Minor and Bugfix Releases for Major Version 3.png)

benchmark/static/styles.css

@@ -1,34 +0,0 @@
-body {
-    font-family: 'Arial', sans-serif;
-    line-height: 1.6;
-    /* margin: 1cm; */
-}
-h1, h2, h3, h4, h5, h6 {
-    font-weight: bold;
-    /* margin: 1em 0; */
-}
-p {
-    /* margin: 0.5em 0; */
-}
-ul, ol {
-    margin: 1em 0;
-    padding-left: 1.5em;
-}
-pre {
-    background: #f4f4f4;
-    padding: 0.5em;
-    overflow-x: auto;
-}
-img {
-    max-width: 100%;
-    height: auto;
-}
-table {
-    width: 100%;
-    border-collapse: collapse;
-    margin: 1em 0;
-}
-th, td {
-    padding: 0.5em;
-    border: 1px solid #ddd;
-}

docs/index.rst

@@ -8,47 +8,6 @@ Welcome to PrimAITE's documentation
 What is PrimAITE?
 -----------------
 
-.. toctree::
-   :maxdepth: 8
-   :caption: About PrimAITE:
-   :hidden:
-
-   source/about
-   source/dependencies
-   source/glossary
-
-.. toctree::
-   :caption: Usage:
-   :hidden:
-
-   source/getting_started
-   source/game_layer
-   source/simulation
-   source/config
-   source/rewards
-   source/customising_scenarios
-   source/varying_config_files
-   source/environment
-   source/action_masking
-
-.. toctree::
-   :caption: Notebooks:
-   :hidden:
-
-   source/example_notebooks
-   source/notebooks/executed_notebooks
-
-.. toctree::
-   :caption: Developer information:
-   :hidden:
-
-   source/developer_tools
-   source/state_system
-   source/request_system
-   PrimAITE API <source/_autosummary/primaite>
-   PrimAITE Tests <source/_autosummary/tests>
-
-
 Overview
 ^^^^^^^^
 
@@ -60,8 +19,6 @@ The ARCD Primary-level AI Training Environment (**PrimAITE**) provides an effect
 - Modelling background (green) pattern-of-life;
 - Operates at machine-speed to enable fast training cycles via Reinforcement Learning (RL).
 
-PrimAITE has been designed as an extensible environment and toolkit to support the development, test, training and evaluation of AI-based cyber defensive agents. Whilst PrimAITE ships with a number of example modelled scenarios (a.k.a. Use Cases), it has not been developed to mandate the solving of a single cyber challenge, and instead provides a highly flexible environment application that can be extended and reconfigured by the user to suit their specific cyber defence training and evaluation needs. PrimAITE provides default networks, red agent and green agent behaviour, reward functions, and action / observation space configuration, all of which can be utilised out of the box, but which ultimately can (and in some instances should) be built upon and / or reconfigured to meet the needs of different defensive agent developers. The PrimAITE user guide provides comprehensive instruction on all PrimAITE features, functionality and components, and can be consulted in order to help guide users in any reconfiguration or enhancements they wish to undertake; a library of example Jupyter notebooks are also provided to support such work.
-
 Features
 ^^^^^^^^
 
@@ -79,6 +36,107 @@ PrimAITE incorporates the following features:
 - A PCAP service is seamlessly integrated within the simulation, automatically capturing and logging frames for both
   inbound and outbound traffic at the network interface level. This automatic functionality, combined with the ability
   to separate traffic directions, significantly enhances network analysis and troubleshooting capabilities;
-- Agent action logs provide a description of every action taken by each agent during the episode. This includes timestep, action, parameters, request and response, for all Blue agent activity, which is aligned with the Track 2 Common Action / Observation Space (CAOS) format. Action logs also detail all scripted / stochastic red / green agent actions;
+- Agent action logs provide a description of every action taken by each agent during the episode. This includes timestep, action, parameters, request and response, for all Blue agent activity, which is aligned with the Track 2 Common Action / Observation Space (CAOS) format. Action logs also details of all scripted / stochastic red / green agent actions;
 - Environment ground truth is provided at every timestep, providing a full description of the environment’s true state;
 - Alignment with CAOS provides the ability to transfer agents between CAOS compliant environments.
+
+Architecture
+^^^^^^^^^^^^
+
+PrimAITE is a Python application and will operate on multiple Operating Systems (Windows, Linux and Mac);
+a comprehensive installation and user guide is provided with each release to support its usage.
+
+Configuration of PrimAITE is achieved via included YAML files which support full control over the network / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count.
+A Simulation Controller layer manages the overall running of the simulation, keeping track of all low-level objects.
+
+It is agnostic to the number of agents, their action / observation spaces, and the RL library being used.
+
+It presents a public API providing a method for describing the current state of the simulation, a method that accepts action requests and provides responses, and a method that triggers a timestep advancement.
+The Game Layer converts the simulation into a playable game for the agent(s).
+
+It translates between simulation state and Gymnasium.Spaces to pass action / observation data between the agent(s) and the simulation. It is responsible for calculating rewards, managing Multi-Agent RL (MARL) action turns, and via a single agent interface can interact with Blue, Red and Green agents.
+
+Agents can either generate their own scripted behaviour or accept input behaviour from an RL agent.
+
+Finally, a Gymnasium / Ray RLlib Environment Layer forwards requests to the Game Layer as the agent sends them. This layer also manages most of the I/O, such as reading in the configuration files and saving agent logs.
+
+.. image:: ../../_static/primAITE_architecture.png
+    :width: 500
+    :align: center
+
+
+Training & Evaluation Capability
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its Gymnasium / Ray RLlib compliant interface.
+
+Scenarios can be constructed to reflect network / system laydowns consisting of any configuration of nodes (e.g., PCs, servers etc.) and the networking equipment and links between them.
+
+All nodes can be configured to contain applications, services, folders and files (and their status).
+
+Traffic flows between services and applications as directed by an ‘execution definition,’ with the traffic flow on the network governed by the network equipment (switches, routers and firewalls) and the ACL rules and routing tables they employ.
+
+Highlights of PrimAITE’s training and evaluation capability are:
+
+- The scenario is not bound to a representation of any platform, system, or technology;
+- Fully configurable (network / system laydown, green pattern-of-life, red personas, reward function, ACL rules for each device, number of episodes / steps, action / observation space) and repeatable to suit the requirements of AI agents;
+- Can integrate with any Gymnasium / Ray RLlib compliant AI agent .
+
+
+PrimAITE provides a number of use cases (network and red/green action configurations) by default which the user is able to extend and modify as required.
+
+What is PrimAITE built with
+---------------------------
+
+* `Gymnasium <https://gymnasium.farama.org/>`_ is used as the basis for AI blue agent interaction with the PrimAITE environment
+* `Networkx <https://github.com/networkx/networkx>`_ is used as the underlying data structure used for the PrimAITE environment
+* `Stable Baselines 3 <https://github.com/DLR-RM/stable-baselines3>`_ is used as a default source of RL algorithms (although PrimAITE is not limited to SB3 agents)
+* `Ray RLlib <https://github.com/ray-project/ray>`_ is used as an additional source of RL algorithms
+* `Typer <https://github.com/tiangolo/typer>`_ is used for building CLIs (Command Line Interface applications)
+* `Jupyterlab <https://github.com/jupyterlab/jupyterlab>`_ is used as an extensible environment for interactive and reproducible computing, based on the Jupyter Notebook Architecture
+* `Platformdirs <https://github.com/platformdirs/platformdirs>`_ is used for finding the right location to store user data and configuration but varies per platform
+* `Plotly <https://github.com/plotly/plotly.py>`_ is used for building high level charts
+
+
+Getting Started with PrimAITE
+-----------------------------
+
+Head over to the :ref:`getting-started` page to install and setup PrimAITE!
+
+.. toctree::
+   :maxdepth: 8
+   :caption: About PrimAITE:
+   :hidden:
+
+   source/about
+   source/dependencies
+   source/glossary
+
+.. toctree::
+   :caption: Usage:
+   :hidden:
+
+   source/getting_started
+   source/simulation
+   source/game_layer
+   source/config
+   source/environment
+   source/customising_scenarios
+   source/varying_config_files
+
+.. toctree::
+   :caption: Notebooks:
+   :hidden:
+
+   source/example_notebooks
+   source/notebooks/executed_notebooks
+
+.. toctree::
+   :caption: Developer information:
+   :hidden:
+
+   source/developer_tools
+   source/state_system
+   source/request_system
+   PrimAITE API <source/_autosummary/primaite>
+   PrimAITE Tests <source/_autosummary/tests>

docs/source/about.rst

@@ -7,68 +7,27 @@
 About PrimAITE
 ==============
 
-Architecture
-^^^^^^^^^^^^
+PrimAITE is a simulation environment for training agents to protect a computer network from cyber attacks.
 
-PrimAITE is a Python application and will operate on multiple Operating Systems (Windows, Linux and Mac);
-a comprehensive installation and user guide is provided with each release to support its usage.
+Features
+********
 
-Configuration of PrimAITE is achieved via included YAML files which support full control over the network / system laydown being modelled, background pattern of life, adversarial (red agent) behaviour, and step and episode count.
-A Simulation Controller layer manages the overall running of the simulation, keeping track of all low-level objects.
+PrimAITE provides the following features:
 
-It is agnostic to the number of agents, their action / observation spaces, and the RL library being used.
-
-It presents a public API providing a method for describing the current state of the simulation, a method that accepts action requests and provides responses, and a method that triggers a timestep advancement.
-The Game Layer converts the simulation into a playable game for the agent(s).
-
-It translates between simulation state and Gymnasium.Spaces to pass action / observation data between the agent(s) and the simulation. It is responsible for calculating rewards, managing Multi-Agent RL (MARL) action turns, and via a single agent interface can interact with Blue, Red and Green agents.
-
-Agents can either generate their own scripted behaviour or accept input behaviour from an RL agent.
-
-Finally, a Gymnasium / Ray RLlib Environment Layer forwards requests to the Game Layer as the agent sends them. This layer also manages most of the I/O, such as reading in the configuration files and saving agent logs.
-
-.. image:: ../../_static/primAITE_architecture.png
-    :width: 500
-    :align: center
+* A flexible system for defining network layouts and host configurations
+* Highly configurable network hosts, including definition of software, file system, and network interfaces,
+* Realistic network traffic simulation, including address and sending packets via internet protocols like TCP, UDP, ICMP, etc.
+* Routers with traffic routing and firewall capabilities
+* Simulation of customisable deterministic agents
+* Support for multiple agents, each having their own customisable observation space, action space, and reward function definition.
 
 
-Training & Evaluation Capability
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Structure
+*********
 
-PrimAITE provides a training and evaluation capability to AI agents in the context of cyber-attack, via its Gymnasium / Ray RLlib compliant interface.
+PrimAITE consists of a simulator and a 'game' layer that allows agents to interact with the simulator. The simulator is built in a modular way where each component such as network hosts, links, networking devices, softwares, etc. are implemented as instances of a base class, meaning they all support the same interface. This allows for standardised configuration using either the Python API or YAML files.
+The game layer is built on top of the simulator and it consumes the simulation action/state interface to allow agents to interact with the simulator. The game layer is also responsible for defining the reward function and observation space for the agents.
 
-Scenarios can be constructed to reflect network / system laydowns consisting of any configuration of nodes (e.g., PCs, servers etc.) and the networking equipment and links between them.
-
-All nodes can be configured to contain applications, services, folders and files (and their status).
-
-Traffic flows between services and applications as directed by an ‘execution definition’, with the traffic flow on the network governed by the network equipment (switches, routers and firewalls) and the ACL rules and routing tables they employ.
-
-Highlights of PrimAITE’s training and evaluation capability are:
-
-- The scenario is not bound to a representation of any platform, system, or technology;
-- Fully configurable (network / system laydown, green pattern-of-life, red personas, reward function, ACL rules for each device, number of episodes / steps, action / observation space) and repeatable to suit the requirements of AI agents;
-- Can integrate with any Gymnasium / Ray RLlib compliant AI agent.
-
-
-PrimAITE provides a number of use cases (network and red/green action configurations) by default which the user is able to extend and modify as required.
-
-What is PrimAITE built with
----------------------------
-
-* `Gymnasium <https://gymnasium.farama.org/>`_ is used as the basis for AI blue agent interaction with the PrimAITE environment
-* `Networkx <https://github.com/networkx/networkx>`_ is used as the underlying data structure used for the PrimAITE environment
-* `Stable Baselines 3 <https://github.com/DLR-RM/stable-baselines3>`_ is used as a default source of RL algorithms (although PrimAITE is not limited to SB3 agents)
-* `Ray RLlib <https://github.com/ray-project/ray>`_ is used as an additional source of RL algorithms
-* `Typer <https://github.com/tiangolo/typer>`_ is used for building CLIs (Command Line Interface applications)
-* `Jupyterlab <https://github.com/jupyterlab/jupyterlab>`_ is used as an extensible environment for interactive and reproducible computing, based on the Jupyter Notebook Architecture
-* `Platformdirs <https://github.com/platformdirs/platformdirs>`_ is used for finding the right location to store user data and configuration but varies per platform
-* `Plotly <https://github.com/plotly/plotly.py>`_ is used for building high level charts
-
-
-Getting Started with PrimAITE
------------------------------
-
-Head over to the :ref:`getting-started` page to install and setup PrimAITE!
 
 ..
   Architecture - Nodes and Links

docs/source/action_masking.rst

@@ -1,157 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-Action Masking
-**************
-The PrimAITE simulation is able to provide action masks in the environment output. These action masks let the agents know
-about which actions are invalid based on the current environment state. For instance, it's not possible to install
-software on a node that is turned off. Therefore, if an agent has a NODE_SOFTWARE_INSTALL in it's action map for that node,
-the action mask will show `0` in the corresponding entry.
-
-*Note: just because an action is available in the action mask does not mean it will be successful when executed. It just means it's possible to try to execute the action at this time.*
-
-Configuration
-=============
-Action masking is supported for agents that use the `ProxyAgent` class (the class used for connecting to RL algorithms).
-In order to use action masking, set the agent_settings.action_masking parameter to True in the config file.
-
-Masking Logic
-=============
-The following logic is applied:
-
-+------------------------------------------+---------------------------------------------------------------------+
-| Action                                   | Action Mask Logic                                                   |
-+==========================================+=====================================================================+
-| **DONOTHING**                            | Always Possible.                                                    |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_SCAN**                    | Node is on. Service is running.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_STOP**                    | Node is on. Service is running.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_START**                   | Node is on. Service is stopped.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_PAUSE**                   | Node is on. Service is running.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_RESUME**                  | Node is on. Service is paused.                                      |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_RESTART**                 | Node is on. Service is running.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_DISABLE**                 | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_ENABLE**                  | Node is on. Service is disabled.                                    |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SERVICE_FIX**                     | Node is on. Service is running.                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_EXECUTE**             | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_SCAN**                | Node is on. Application is running.                                 |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_CLOSE**               | Node is on. Application is running.                                 |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_FIX**                 | Node is on. Application is running.                                 |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_INSTALL**             | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_APPLICATION_REMOVE**              | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_SCAN**                       | Node is on. File exists. File not deleted.                          |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_CREATE**                     | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_CHECKHASH**                  | Node is on. File exists. File not deleted.                          |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_DELETE**                     | Node is on. File exists.                                            |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_REPAIR**                     | Node is on. File exists. File not deleted.                          |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_RESTORE**                    | Node is on. File exists. File is deleted.                           |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_CORRUPT**                    | Node is on. File exists. File not deleted.                          |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FILE_ACCESS**                     | Node is on. File exists. File not deleted.                          |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FOLDER_CREATE**                   | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FOLDER_SCAN**                     | Node is on. Folder exists. Folder not deleted.                      |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FOLDER_CHECKHASH**                | Node is on. Folder exists. Folder not deleted.                      |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FOLDER_REPAIR**                   | Node is on. Folder exists. Folder not deleted.                      |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_FOLDER_RESTORE**                  | Node is on. Folder exists. Folder is deleted.                       |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_OS_SCAN**                         | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **HOST_NIC_ENABLE**                      | NIC is disabled. Node is on.                                        |
-+------------------------------------------+---------------------------------------------------------------------+
-| **HOST_NIC_DISABLE**                     | NIC is enabled. Node is on.                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SHUTDOWN**                        | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_STARTUP**                         | Node is off.                                                        |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_RESET**                           | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_PING_SCAN**                  | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_PORT_SCAN**                  | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_NETWORK_SERVICE_RECON**      | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NETWORK_PORT_ENABLE**                  | Node is on. Router is on.                                           |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NETWORK_PORT_DISABLE**                 | Router is on.                                                       |
-+------------------------------------------+---------------------------------------------------------------------+
-| **ROUTER_ACL_ADDRULE**                   | Router is on.                                                       |
-+------------------------------------------+---------------------------------------------------------------------+
-| **ROUTER_ACL_REMOVERULE**                | Router is on.                                                       |
-+------------------------------------------+---------------------------------------------------------------------+
-| **FIREWALL_ACL_ADDRULE**                 | Firewall is on.                                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **FIREWALL_ACL_REMOVERULE**              | Firewall is on.                                                     |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_PING_SCAN**                  | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_PORT_SCAN**                  | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_NMAP_NETWORK_SERVICE_RECON**      | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **CONFIGURE_DATABASE_CLIENT**            | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **CONFIGURE_RANSOMWARE_SCRIPT**          | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **CONFIGURE_DOSBOT**                     | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **CONFIGURE_C2_BEACON**                  | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **C2_SERVER_RANSOMWARE_LAUNCH**          | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **C2_SERVER_RANSOMWARE_CONFIGURE**       | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **C2_SERVER_TERMINAL_COMMAND**           | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **C2_SERVER_DATA_EXFILTRATE**            | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_ACCOUNTS_CHANGE_PASSWORD**        | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **SSH_TO_REMOTE**                        | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **SESSIONS_REMOTE_LOGOFF**               | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-| **NODE_SEND_REMOTE_COMMAND**             | Node is on.                                                         |
-+------------------------------------------+---------------------------------------------------------------------+
-
-
-Mechanism
-=========
-The environment iterates over the RL agent's ``action_map`` and generates the corresponding simulator request string.
-It uses the ``RequestManager.check_valid()`` method to invoke the relevant ``RequestPermissionValidator`` without
-actually running the request on the simulation.
-
-Current Limitations
-===================
-Currently, action masking only considers whether the action as a whole is possible, it doesn't verify that the exact
-parameter combination passed to the action make sense in the current context. For instance, if ACL rule 3 on router_1 is
-already populated, the action for adding another rule at position 3 will be available regardless, as long as that router
-is turned on. This will never block valid actions. It will just occasionally allow invalid actions.

docs/source/configuration/agents.rst

@@ -172,8 +172,3 @@ The amount of timesteps that the frequency can randomly change.
 ---------------
 
 If ``True``, gymnasium flattening will be performed on the observation space before sending to the agent. Set this to ``True`` if your agent does not support nested observation spaces.
-
-``Agent History``
------------------
-
-Agents will record their action log for each step. This is a summary of what the agent did, along with response information from requests within the simulation.

docs/source/configuration/game.rst

@@ -28,7 +28,6 @@ This section defines high-level settings that apply across the game, currently i
                 high: 10
                 medium: 5
                 low: 0
-        seed: 1
 
 ``max_episode_length``
 ----------------------
@@ -55,8 +54,3 @@ See :ref:`List of IPProtocols <List of IPProtocols>` for a list of protocols.
 --------------
 
 These are used to determine the thresholds of high, medium and low categories for counted observation occurrences.
-
-``seed``
---------
-
-Used to configure the random seeds used within PrimAITE, ensuring determinism within episode/session runs. If empty or set to -1, no seed is set.

docs/source/configuration/io_settings.rst

@@ -18,11 +18,8 @@ This section configures how PrimAITE saves data during simulation and training.
         save_step_metadata: False
         save_pcap_logs: False
         save_sys_logs: False
-        save_agent_logs: False
         write_sys_log_to_terminal: False
-        write_agent_log_to_terminal: False
         sys_log_level: WARNING
-        agent_log_level: INFO
 
 
 ``save_logs``
@@ -60,12 +57,6 @@ Optional. Default value is ``False``.
 
 If ``True``, then the log files which contain all node actions during the simulation will be saved.
 
-``save_agent_logs``
------------------
-
-Optional. Default value is ``False``.
-
-If ``True``, then the log files which contain all human readable agent behaviour during the simulation will be saved.
 
 ``write_sys_log_to_terminal``
 -----------------------------
@@ -74,25 +65,16 @@ Optional. Default value is ``False``.
 
 If ``True``, PrimAITE will print sys log to the terminal.
 
-``write_agent_log_to_terminal``
------------------------------
 
-Optional. Default value is ``False``.
-
-If ``True``, PrimAITE will print all human readable agent behaviour logs to the terminal.
-
-
-``sys_log_level & agent_log_level``
----------------------------------
+``sys_log_level``
+-------------
 
 Optional. Default value is ``WARNING``.
 
-The level of logging that should be visible in the syslog, agent logs or the logs output to the terminal.
+The level of logging that should be visible in the sys logs or the logs output to the terminal.
 
 ``save_sys_logs`` or ``write_sys_log_to_terminal`` has to be set to ``True`` for this setting to be used.
 
-This is also true for agent behaviour logging.
-
 Available options are:
 
 - ``DEBUG``: Debug level items and the items below

docs/source/configuration/simulation.rst

@@ -7,7 +7,7 @@
 ==============
 In this section the network layout is defined. This part of the config follows a hierarchical structure. Almost every component defines a ``ref`` field which acts as a human-readable unique identifier, used by other parts of the config, such as agents.
 
-At the top level of the network are ``nodes``, ``links`` and ``airspace``.
+At the top level of the network are ``nodes`` and ``links``.
 
 e.g.
 
@@ -19,9 +19,6 @@ e.g.
             ...
             links:
             ...
-            airspace:
-            ...
-
 
 ``nodes``
 ---------
@@ -104,27 +101,3 @@ This accepts an integer value e.g. if port 1 is to be connected, the configurati
 ``bandwidth``
 
 This is an integer value specifying the allowed bandwidth across the connection. Units are in Mbps.
-
-``airspace``
-------------
-
-This section configures settings specific to the wireless network's virtual airspace.
-
-``frequency_max_capacity_mbps``
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-This setting allows the user to override the default maximum bandwidth capacity set for each frequency. The key should
-be the AirSpaceFrequency name and the value be the desired maximum bandwidth capacity in mbps (megabits per second) for
-a single timestep.
-
-The below example would permit 123.45 megabits to be transmit across the WiFi 2.4 GHz frequency in a single timestep.
-Setting a frequencies max capacity to 0.0 blocks that frequency on the airspace.
-
-.. code-block:: yaml
-
-    simulation:
-      network:
-        airspace:
-          frequency_max_capacity_mbps:
-            WIFI_2_4: 123.45
-            WIFI_5: 0.0

docs/source/configuration/simulation/nodes/common/common_node_attributes.rst

@@ -53,27 +53,3 @@ The number of time steps required to occur in order for the node to cycle from `
 Optional. Default value is ``3``.
 
 The number of time steps required to occur in order for the node to cycle from ``ON`` to ``SHUTTING_DOWN`` and then finally ``OFF``.
-
-``users``
----------
-
-The list of pre-existing users that are additional to the default admin user (``username=admin``, ``password=admin``).
-Additional users are configured as an array and must contain a ``username``, ``password``, and can contain an optional
-boolean ``is_admin``.
-
-Example of adding two additional users to a node:
-
-.. code-block:: yaml
-
-    simulation:
-      network:
-        nodes:
-        - hostname: [hostname]
-          type: [Node Type]
-          users:
-            - username: jane.doe
-              password: '1234'
-              is_admin: true
-            - username: john.doe
-              password: password_1
-              is_admin: false

docs/source/developer_tools.rst

@@ -49,68 +49,36 @@ dev-mode configuration
 
 The following configures some specific items that the dev-mode overrides, if enabled.
 
-`--sys-log-level` or `-slevel`
------------------------------
+`--sys-log-level` or `-level`
+----------------------------
 
 The level of system logs can be overridden by dev-mode.
 
 By default, this is set to DEBUG
 
-The available options for both system and agent logs are:
+The available options are [DEBUG|INFO|WARNING|ERROR|CRITICAL]
 
-+-------------------+
-| Log Level         |
-+===================+
-| DEBUG             |
-+-------------------+
-| INFO              |
-+-------------------+
-| WARNING           |
-+-------------------+
-| ERROR             |
-+-------------------+
-| CRITICAL          |
-+-------------------+
+.. code-block::
+
+    primaite dev-mode config -level INFO
+
+or
 
 .. code-block::
 
     primaite dev-mode config --sys-log-level INFO
 
-or
-
-.. code-block::
-
-    primaite dev-mode config -slevel INFO
-
-
-`--agent-log-level` or `-alevel`
---------------------------------
-
-The level of agent logs can be overridden by dev-mode.
-
-By default, this is set to DEBUG.
-
-.. code-block::
-
-    primaite dev-mode config --agent-log-level INFO
-
-or
-
-.. code-block::
-
-    primaite dev-mode config -alevel INFO
-
 `--output-sys-logs` or `-sys`
 -----------------------------
 
-The output of system logs can be overridden by dev-mode.
+The outputting of system logs can be overridden by dev-mode.
 
 By default, this is set to False
 
 Enabling system logs
 """"""""""""""""""""
 
-To enable output of system logs
+To enable outputting of system logs
 
 .. code-block::
 
@@ -125,7 +93,7 @@ or
 Disabling system logs
 """""""""""""""""""""
 
-To disable output of system logs
+To disable outputting of system logs
 
 .. code-block::
 
@@ -137,47 +105,17 @@ or
 
     primaite dev-mode config -nsys
 
-Enabling agent logs
-""""""""""""""""""""
-
-To enable output of system logs
-
-.. code-block::
-
-    primaite dev-mode config --output-agent-logs
-
-or
-
-.. code-block::
-
-    primaite dev-mode config -agent
-
-Disabling system logs
-"""""""""""""""""""""
-
-To disable output of system logs
-
-.. code-block::
-
-    primaite dev-mode config --no-agent-logs
-
-or
-
-.. code-block::
-
-    primaite dev-mode config -nagent
-
 `--output-pcap-logs` or `-pcap`
 -------------------------------
 
-The output of packet capture logs can be overridden by dev-mode.
+The outputting of packet capture logs can be overridden by dev-mode.
 
 By default, this is set to False
 
 Enabling PCAP logs
 """"""""""""""""""
 
-To enable output of packet capture logs
+To enable outputting of packet capture logs
 
 .. code-block::
 
@@ -192,7 +130,7 @@ or
 Disabling PCAP logs
 """""""""""""""""""
 
-To disable output of packet capture logs
+To disable outputting of packet capture logs
 
 .. code-block::
 
@@ -207,14 +145,14 @@ or
 `--output-to-terminal` or `-t`
 ------------------------------
 
-The output of system logs to the terminal can be overridden by dev-mode.
+The outputting of system logs to the terminal can be overridden by dev-mode.
 
 By default, this is set to False
 
 Enabling system log output to terminal
 """"""""""""""""""""""""""""""""""""""
 
-To enable output of system logs to terminal
+To enable outputting of system logs to terminal
 
 .. code-block::
 
@@ -229,7 +167,7 @@ or
 Disabling system log output to terminal
 """""""""""""""""""""""""""""""""""""""
 
-To disable output of system logs to terminal
+To disable outputting of system logs to terminal
 
 .. code-block::
 

docs/source/example_notebooks.rst

@@ -2,8 +2,6 @@
 
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
-.. _example jupyter notebooks:
-
 Example Jupyter Notebooks
 =========================
 
@@ -20,7 +18,6 @@ Running Jupyter Notebooks
 -------------------------
 
 1. Navigate to the PrimAITE directory
-"""""""""""""""""""""""""""""""""""""
 
 .. code-block:: bash
     :caption: Unix
@@ -32,10 +29,7 @@ Running Jupyter Notebooks
 
     cd ~\primaite\{VERSION}
 
-2. Run jupyter notebook
-"""""""""""""""""""""""
-
-**Please note that the python environment to which you installed PrimAITE must be active.**
+2. Run jupyter notebook (the python environment to which you installed PrimAITE must be active)
 
 .. code-block:: bash
     :caption: Unix
@@ -48,13 +42,11 @@ Running Jupyter Notebooks
     jupyter notebook
 
 3. Opening the jupyter webpage (optional)
-"""""""""""""""""""""""""""""""""""""""""
 
 The default web browser may automatically open the webpage. However, if that is not the case, click the link shown in your command prompt output. It should look like this: ``http://localhost:8888/?token=0123456798abc0123456789abc``
 
 
 4. Navigate to the list of notebooks
-"""""""""""""""""""""""""""""""""""""""""
 
 The example notebooks are located in ``notebooks/example_notebooks/``. The file system shown in the jupyter webpage is relative to the location in which the ``jupyter notebook`` command was used.
 
@@ -85,6 +77,6 @@ The following extensions should now be installed
     :width: 300
     :align: center
 
-VSCode will then ask for a Python environment version to use. PrimAITE is compatible with Python versions 3.8 - 3.11
+VSCode will then ask for a Python environment version to use. PrimAITE is compatible with Python versions 3.9 - 3.11
 
 You should now be able to interact with the notebook.

docs/source/game_layer.rst

@@ -42,50 +42,49 @@ An agent's reward function is managed by the ``RewardManager``. It calculates re
 Reward Components
 -----------------
 
-Currently implemented are reward components tailored to the data manipulation scenario. View the full API and description of how they work here: :py:modules:`primaite.game.agent.rewards`.
+Currently implemented are reward components tailored to the data manipulation scenario. View the full API and description of how they work here: :py:module:`primaite.game.agent.reward`.
 
 Reward Sharing
 --------------
 
 An agent's reward can be based on rewards of other agents. This is particularly useful for modelling a situation where the blue agent's job is to protect the ability of green agents to perform their pattern-of-life. This can be configured in the YAML file this way:
 
-.. code-block:: yaml
+```yaml
+green_agent_1: # this agent sometimes tries to access the webpage, and sometimes the database
+    # actions, observations, and agent settings go here
+    reward_function:
+      reward_components:
 
-  green_agent_1: # this agent sometimes tries to access the webpage, and sometimes the database
-      # actions, observations, and agent settings go here
-      reward_function:
-        reward_components:
+        # When the webpage loads, the reward goes up by 0.25 when it fails to load, it goes down to -0.25
+        - type: WEBPAGE_UNAVAILABLE_PENALTY
+          weight: 0.25
+          options:
+            node_hostname: client_2
 
-          # When the webpage loads, the reward goes up by 0.25 when it fails to load, it goes down to -0.25
-          - type: WEBPAGE_UNAVAILABLE_PENALTY
-            weight: 0.25
-            options:
-              node_hostname: client_2
+        # When the database is reachable, the reward goes up by 0.05, when it is unreachable it goes down to -0.05
+        - type: GREEN_ADMIN_DATABASE_UNREACHABLE_PENALTY
+          weight: 0.05
+          options:
+            node_hostname: client_2
 
-          # When the database is reachable, the reward goes up by 0.05, when it is unreachable it goes down to -0.05
-          - type: GREEN_ADMIN_DATABASE_UNREACHABLE_PENALTY
-            weight: 0.05
-            options:
-              node_hostname: client_2
+blue_agent:
+    # actions, observations, and agent settings go here
+    reward_function:
+      reward_components:
 
-  blue_agent:
-      # actions, observations, and agent settings go here
-      reward_function:
-        reward_components:
-
-          # When the database file is in a good state, blue's reward is 0.4, when it's in a corrupted state the reward is -0.4
-          - type: DATABASE_FILE_INTEGRITY
-            weight: 0.40
-            options:
-              node_hostname: database_server
-              folder_name: database
-              file_name: database.db
-
-          # The green's reward is added onto the blue's reward.
-          - type: SHARED_REWARD
-            weight: 1.0
-            options:
-              agent_name: client_2_green_user
+        # When the database file is in a good state, blue's reward is 0.4, when it's in a corrupted state the reward is -0.4
+        - type: DATABASE_FILE_INTEGRITY
+          weight: 0.40
+          options:
+            node_hostname: database_server
+            folder_name: database
+            file_name: database.db
 
+        # The green's reward is added onto the blue's reward.
+        - type: SHARED_REWARD
+          weight: 1.0
+          options:
+            agent_name: client_2_green_user
+```
 
 When defining agent reward sharing, users must be careful to avoid circular references, as that would lead to an infinite calculation loop. PrimAITE will prevent circular dependencies and provide a helpful error message if they are detected in the yaml.

docs/source/primaite-dependencies.rst

@@ -2,44 +2,40 @@
 
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| Name              | Version | License                            | Description                                                                                           | URL                                                                |
-+===================+=========+====================================+=======================================================================================================+====================================================================+
-| gymnasium         | 0.28.1  | MIT License                        | A standard API for reinforcement learning and a diverse set of reference environments (formerly Gym). | https://farama.org                                                 |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| ipywidgets        | 8.1.5   | BSD License                        | Jupyter interactive widgets                                                                           | http://jupyter.org                                                 |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| jupyterlab        | 3.6.1   | BSD License                        | JupyterLab computational environment                                                                  | https://jupyter.org                                                |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| kaleido           | 0.2.1   | MIT                                | Static image export for web-based visualization libraries with zero dependencies                      | https://github.com/plotly/Kaleido                                  |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| matplotlib        | 3.7.1   | Python Software Foundation License | Python plotting package                                                                               | https://matplotlib.org                                             |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| networkx          | 3.1     | BSD License                        | Python package for creating and manipulating graphs and networks                                      | https://networkx.org/                                              |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| numpy             | 1.23.5  | BSD License                        | NumPy is the fundamental package for array computing with Python.                                     | https://www.numpy.org                                              |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| platformdirs      | 3.5.1   | MIT License                        | A small Python package for determining appropriate platform-specific dirs, e.g. a "user data dir".    | https://github.com/platformdirs/platformdirs                       |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| plotly            | 5.15.0  | MIT License                        | An open-source, interactive data visualization library for Python                                     | https://plotly.com/python/                                         |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| polars            | 0.20.30 | MIT License                        | Blazingly fast DataFrame library                                                                      | https://www.pola.rs/                                               |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| prettytable       | 3.8.0   | BSD License (BSD (3 clause))       | A simple Python library for easily displaying tabular data in a visually appealing ASCII table format | https://github.com/jazzband/prettytable                            |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| pydantic          | 2.7.0   | MIT License                        | Data validation using Python type hints                                                               | https://github.com/pydantic/pydantic                               |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| PyYAML            | 6.0     | MIT License                        | YAML parser and emitter for Python                                                                    | https://pyyaml.org/                                                |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| ray               | 2.32.0  | Apache 2.0                         | Ray provides a simple, universal API for building distributed applications.                           | https://github.com/ray-project/ray                                 |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| stable-baselines3 | 2.1.0   | MIT                                | Pytorch version of Stable Baselines, implementations of reinforcement learning algorithms.            | https://github.com/DLR-RM/stable-baselines3                        |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| tensorflow        | 2.12.0  | Apache Software License            | TensorFlow is an open source machine learning framework for everyone.                                 | https://www.tensorflow.org/                                        |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| typer             | 0.9.0   | MIT License                        | Typer, build great CLIs. Easy to code. Based on Python type hints.                                    | https://github.com/tiangolo/typer                                  |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| Deepdiff          | 8.0.1   | MIT License                        | Deep difference of dictionaries, iterables, strings, and any other object objects.                    | https://github.com/seperman/deepdiff                               |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
-| sb3_contrib       | 2.1.0   | MIT License                        | Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code (Action Masking)| https://github.com/Stable-Baselines-Team/stable-baselines3-contrib |
-+-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| Name              | Version | License                            | Description                                                                                           | URL                                          |
++===================+=========+====================================+=======================================================================================================+==============================================+
+| gymnasium         | 0.28.1  | MIT License                        | A standard API for reinforcement learning and a diverse set of reference environments (formerly Gym). | https://farama.org                           |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| ipywidgets        | 8.1.3   | BSD License                        | Jupyter interactive widgets                                                                           | http://jupyter.org                           |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| jupyterlab        | 3.6.1   | BSD License                        | JupyterLab computational environment                                                                  | https://jupyter.org                          |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| kaleido           | 0.2.1   | MIT                                | Static image export for web-based visualization libraries with zero dependencies                      | https://github.com/plotly/Kaleido            |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| matplotlib        | 3.7.1   | Python Software Foundation License | Python plotting package                                                                               | https://matplotlib.org                       |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| networkx          | 3.1     | BSD License                        | Python package for creating and manipulating graphs and networks                                      | https://networkx.org/                        |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| numpy             | 1.23.5  | BSD License                        | NumPy is the fundamental package for array computing with Python.                                     | https://www.numpy.org                        |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| platformdirs      | 3.5.1   | MIT License                        | A small Python package for determining appropriate platform-specific dirs, e.g. a "user data dir".    | https://github.com/platformdirs/platformdirs |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| plotly            | 5.15.0  | MIT License                        | An open-source, interactive data visualization library for Python                                     | https://plotly.com/python/                   |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| polars            | 0.18.4  | MIT License                        | Blazingly fast DataFrame library                                                                      | https://www.pola.rs/                         |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| prettytable       | 3.8.0   | BSD License (BSD (3 clause))       | A simple Python library for easily displaying tabular data in a visually appealing ASCII table format | https://github.com/jazzband/prettytable      |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| pydantic          | 2.7.0   | MIT License                        | Data validation using Python type hints                                                               | https://github.com/pydantic/pydantic         |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| PyYAML            | 6.0     | MIT License                        | YAML parser and emitter for Python                                                                    | https://pyyaml.org/                          |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| ray               | 2.23.0  | Apache 2.0                         | Ray provides a simple, universal API for building distributed applications.                           | https://github.com/ray-project/ray           |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| stable-baselines3 | 2.1.0   | MIT                                | Pytorch version of Stable Baselines, implementations of reinforcement learning algorithms.            | https://github.com/DLR-RM/stable-baselines3  |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| tensorflow        | 2.12.0  | Apache Software License            | TensorFlow is an open source machine learning framework for everyone.                                 | https://www.tensorflow.org/                  |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+
+| typer             | 0.9.0   | MIT License                        | Typer, build great CLIs. Easy to code. Based on Python type hints.                                    | https://github.com/tiangolo/typer            |
++-------------------+---------+------------------------------------+-------------------------------------------------------------------------------------------------------+----------------------------------------------+

docs/source/request_system.rst

@@ -9,55 +9,49 @@ Request System
 
 Just like other aspects of SimComponent, the request types are not managed centrally for the whole simulation, but instead they are dynamically created and updated based on the nodes, links, and other components that currently exist in the simulation. This is achieved in the following way:
 
-When requesting an action within the simulation, these two arguments must be provided:
+- API
+    When requesting an action within the simulation, these two arguments must be provided:
 
-1. ``request`` - selects which action you want to take on this ``SimComponent``. This is formatted as a list of strings such as ``['network', 'node', '<node-name>', 'service', '<service-name>', 'restart']``.
-2. ``context`` - optional extra information that can be used to decide how to process the request. This is formatted as a dictionary. For example, if the request requires authentication, the context can include information about the user that initiated the request to decide if their permissions are sufficient.
+    1. ``request`` - selects which action you want to take on this ``SimComponent``. This is formatted as a list of strings such as ``['network', 'node', '<node-name>', 'service', '<service-name>', 'restart']``.
+    2. ``context`` - optional extra information that can be used to decide how to process the request. This is formatted as a dictionary. For example, if the request requires authentication, the context can include information about the user that initiated the request to decide if their permissions are sufficient.
 
-When a request is resolved, it returns a success status, and optional additional data about the request.
+    When a request is resolved, it returns a success status, and optional additional data about the request.
 
-``status`` can be one of:
+    ``status`` can be one of:
 
-* ``success``: the request was executed
-* ``failure``: the request could not be executed
-* ``unreachable``: the target for the request was not found
-* ``pending``: the request was initiated, but has not finished during this step
+    * ``success``: the request was executed
+    * ``failure``: the request could not be executed
+    * ``unreachable``: the target for the request was not found
+    * ``pending``: the request was initiated, but has not finished during this step
 
-``data`` can be a dictionary with any arbitrary JSON-like data to describe the outcome of the request.
+    ``data`` can be a dictionary with any arbitrary JSON-like data to describe the outcome of the request.
 
-Requests:
-"""""""""
+- ``request`` detail
+    The request is a list of strings which help specify who should handle the request. The strings in the request list help RequestManagers traverse the 'ownership tree' of SimComponent. The example given above would be handled in the following way:
 
-Request Syntax
----------------
+    1. ``Simulation`` receives ``['network', 'node', 'computer_1', 'service', 'DNSService', 'restart']``.
+        The first element of the request is ``network``, therefore it passes the request down to its network.
+    2. ``Network`` receives ``['node', 'computer_1', 'service', 'DNSService', 'restart']``.
+        The first element of the request is ``node``, therefore the network looks at the node name and passes the request down to the node with that name.
+    3. ``computer_1`` receives ``['service', 'DNSService', 'restart']``.
+        The first element of the request is ``service``, therefore the node looks at the service name and passes the rest of the request to the service with that name.
+    4. ``DNSService`` receives ``['restart']``.
+        Since ``restart`` is a defined request type in the service's own RequestManager, the service performs a restart.
 
-The request is a list of strings which help specify who should handle the request. The strings in the request list help RequestManagers traverse the 'ownership tree' of SimComponent. The example given above would be handled in the following way:
-
-1. ``Simulation`` receives ``['network', 'node', 'computer_1', 'service', 'DNSService', 'restart']``.
-    The first element of the request is ``network``, therefore it passes the request down to its network.
-2. ``Network`` receives ``['node', 'computer_1', 'service', 'DNSService', 'restart']``.
-    The first element of the request is ``node``, therefore the network looks at the node name and passes the request down to the node with that name.
-3. ``computer_1`` receives ``['service', 'DNSService', 'restart']``.
-    The first element of the request is ``service``, therefore the node looks at the service name and passes the rest of the request to the service with that name.
-4. ``DNSService`` receives ``['restart']``.
-    Since ``restart`` is a defined request type in the service's own RequestManager, the service performs a restart.
-
-- ``context``
+- ``context`` detail
     The context is not used by any of the currently implemented components or requests.
 
-Request responses
------------------
+- Request response
+    When the simulator receives a request, it returns a response with a success status. The possible statuses are:
 
-When the simulator receives a request, it returns a response with a success status. The possible statuses are:
+    * **success**: The request was received and successfully executed.
+        * For example, the agent tries to add an ACL rule and specifies correct parameters, and the ACL rule is added successfully.
 
-* **success**: The request was received and successfully executed.
-    * For example, the agent tries to add an ACL rule and specifies correct parameters, and the ACL rule is added successfully.
+    * **failure**: The request was received, but it could not be executed, or it failed while executing.
+        * For example, the agent tries to execute the ``WebBrowser`` application, but the webpage wasn't retrieved because the DNS server is not setup on the node.
 
-* **failure**: The request was received, but it could not be executed, or it failed while executing.
-    * For example, the agent tries to execute the ``WebBrowser`` application, but the webpage wasn't retrieved because the DNS server is not setup on the node.
-
-* **unreachable**: The request was sent to a simulation component that does not exist.
-    * For example, the agent tries to scan a file that has not been created yet.
+    * **unreachable**: The request was sent to a simulation component that does not exist.
+        * For example, the agent tries to scan a file that has not been created yet.
 
 For more information, please refer to the ``Requests-and-Responses.ipynb`` jupyter notebook
 

docs/source/rewards.rst

@@ -1,126 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-Rewards
-#######
-
-Rewards in PrimAITE are based on a system of individual components that react to events in the simulation. An agent's reward function is calculated as the weighted sum of several reward components.
-
-Some rewards, such as the ``GreenAdminDatabaseUnreachablePenalty``, can be marked as 'sticky' in their configuration. Setting this to ``True`` will mean that they continue to output the same value after an event until another event of that type.
-In the instance of the ``GreenAdminDatabaseUnreachablePenalty``, the database admin reward will stay negative until the next successful database request is made, even if the database admin agents do nothing and the database returns a good state.
-
-Components
-**********
-The following API pages describe the use of each reward component and the possible configuration options. An example of configuring each via yaml is also provided.
-
-:py:class:`primaite.game.agent.rewards.DummyReward`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: DUMMY
-              weight: 1.0
-
-
-:py:class:`primaite.game.agent.rewards.DatabaseFileIntegrity`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: DATABASE_FILE_INTEGRITY
-              weight: 1.0
-              options:
-                node_hostname: server_1
-                folder_name: database
-                file_name: database.db
-
-
-:py:class:`primaite.game.agent.rewards.WebServer404Penalty`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: WEB_SERVER_404_PENALTY
-              node_hostname: web_server
-              weight: 1.0
-              options:
-                service_name: WebService
-                sticky: false
-
-
-:py:class:`primaite.game.agent.rewards.WebpageUnavailablePenalty`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: WEBPAGE_UNAVAILABLE_PENALTY
-              node_hostname: computer_1
-              weight: 1.0
-              options:
-                sticky: false
-
-
-:py:class:`primaite.game.agent.rewards.GreenAdminDatabaseUnreachablePenalty`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: GREEN_ADMIN_DATABASE_UNREACHABLE_PENALTY
-              weight: 1.0
-              options:
-                node_hostname: admin_pc_1
-                sticky: false
-
-
-:py:class:`primaite.game.agent.rewards.SharedReward`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: scripted_agent
-        # ...
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: SHARED_REWARD
-              weight: 1.0
-              options:
-                agent_name: scripted_agent
-
-
-:py:class:`primaite.game.agent.rewards.ActionPenalty`
-
-.. code-block:: yaml
-
-    agents:
-      - ref: agent_name
-        # ...
-        reward_function:
-          reward_components:
-            - type: ACTION_PENALTY
-              weight: 1.0
-              options:
-                  action_penalty: -0.3
-                  do_nothing_penalty: 0.0

docs/source/simulation.rst

@@ -27,7 +27,6 @@ Contents
    simulation_components/network/nodes/firewall
    simulation_components/network/switch
    simulation_components/network/network
-   simulation_components/network/airspace
    simulation_components/system/internal_frame_processing
    simulation_components/system/sys_log
    simulation_components/system/pcap

docs/source/simulation_components/network/airspace.rst

@@ -1,42 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-.. _airspace:
-
-AirSpace
-========
-
-
-1. Introduction
----------------
-
-The AirSpace class is the central component for wireless networks in PrimAITE and is designed to model and manage the behavior and interactions of wireless network interfaces within a simulated wireless network environment. This documentation provides a detailed overview of the AirSpace class, its components, and how they interact to create a realistic simulation of wireless network dynamics.
-
-2. Overview of the AirSpace System
-----------------------------------
-
-The AirSpace is a virtual representation of a physical wireless environment, managing multiple wireless network interfaces that simulate devices connected to the wireless network. These interfaces communicate over radio frequencies, with their interactions influenced by various factors modeled within the AirSpace.
-
-2.1 Key Components
-^^^^^^^^^^^^^^^^^^
-
-- **Wireless Network Interfaces**: Representations of network interfaces connected physical devices like routers, computers, or IoT devices that can send and receive data wirelessly.
-- **Bandwidth Management**: Tracks data transmission over frequencies to prevent overloading and simulate real-world network congestion.
-
-
-3. Managing Wireless Network Interfaces
----------------------------------------
-
-- Interfaces can be dynamically added or removed.
-- Configurations can be changed in real-time.
-- The AirSpace handles data transmissions, ensuring data sent by an interface is received by all other interfaces on the same frequency.
-
-
-4. AirSpace Inspection
-----------------------
-
-The AirSpace class provides methods for visualizing network behavior:
-
-- ``show_wireless_interfaces()``: Displays current state of all interfaces
-- ``show_bandwidth_load()``: Shows bandwidth utilisation

docs/source/simulation_components/network/base_hardware.rst

@@ -97,8 +97,8 @@ Node Behaviours/Functions
 - **receive_frame()**: Handles the processing of incoming network frames.
 - **apply_timestep()**: Advances the state of the node according to the simulation timestep.
 - **power_on()**: Initiates the node, enabling all connected Network Interfaces and starting all Services and
-  Applications, taking into account the ``start_up_duration``.
-- **power_off()**: Stops the node's operations, adhering to the ``shut_down_duration``.
+  Applications, taking into account the `start_up_duration`.
+- **power_off()**: Stops the node's operations, adhering to the `shut_down_duration`.
 - **ping()**: Sends ICMP echo requests to a specified IP address to test connectivity.
 - **has_enabled_network_interface()**: Checks if the node has any network interfaces enabled, facilitating network
   communication.
@@ -109,205 +109,3 @@ Node Behaviours/Functions
 The Node class handles installation of system software, network connectivity, frame processing, system logging, and
 power states. It establishes baseline functionality while allowing subclassing to model specific node types like hosts,
 routers, firewalls etc. The flexible architecture enables composing complex network topologies.
-
-User, UserManager, and UserSessionManager
-=========================================
-
-The ``base.py`` module also includes essential classes for managing users and their sessions within the PrimAITE
-simulation. These are the ``User``, ``UserManager``, and ``UserSessionManager`` classes. The base ``Node`` class comes
-with ``UserManager``, and ``UserSessionManager`` classes pre-installed.
-
-User Class
-----------
-
-The ``User`` class represents a user in the system. It includes attributes such as ``username``, ``password``,
-``disabled``, and ``is_admin`` to define the user's credentials and status.
-
-Example Usage
-^^^^^^^^^^^^^
-
-Creating a user:
-    .. code-block:: python
-
-        user = User(username="john_doe", password="12345")
-
-UserManager Class
------------------
-
-The ``UserManager`` class handles user management tasks such as creating users, authenticating them, changing passwords,
-and enabling or disabling user accounts. It maintains a dictionary of users and provides methods to manage them
-effectively.
-
-Example Usage
-^^^^^^^^^^^^^
-
-Creating a ``UserManager`` instance and adding a user:
-    .. code-block:: python
-
-        user_manager = UserManager()
-        user_manager.add_user(username="john_doe", password="12345")
-
-Authenticating a user:
-    .. code-block:: python
-
-        user = user_manager.authenticate_user(username="john_doe", password="12345")
-
-UserSessionManager Class
-------------------------
-
-The ``UserSessionManager`` class manages user sessions, including local and remote sessions. It handles session creation,
-timeouts, and provides methods for logging users in and out.
-
-Example Usage
-^^^^^^^^^^^^^
-
-Creating a ``UserSessionManager`` instance and logging a user in locally:
-    .. code-block:: python
-
-        session_manager = UserSessionManager()
-        session_id = session_manager.local_login(username="john_doe", password="12345")
-
-Logging a user out:
-    .. code-block:: python
-
-        session_manager.local_logout()
-
-Practical Examples
-------------------
-
-Below are unit tests which act as practical examples illustrating how to use the ``User``, ``UserManager``, and
-``UserSessionManager`` classes within the context of a client-server network simulation.
-
-Setting up a Client-Server Network
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    from typing import Tuple
-    from uuid import uuid4
-
-    import pytest
-
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.network.hardware.nodes.host.server import Server
-
-    @pytest.fixture(scope="function")
-    def client_server_network() -> Tuple[Computer, Server, Network]:
-        network = Network()
-
-        client = Computer(
-            hostname="client",
-            ip_address="192.168.1.2",
-            subnet_mask="255.255.255.0",
-            default_gateway="192.168.1.1",
-            start_up_duration=0,
-        )
-        client.power_on()
-
-        server = Server(
-            hostname="server",
-            ip_address="192.168.1.3",
-            subnet_mask="255.255.255.0",
-            default_gateway="192.168.1.1",
-            start_up_duration=0,
-        )
-        server.power_on()
-
-        network.connect(client.network_interface[1], server.network_interface[1])
-
-        return client, server, network
-
-Local Login Success
-^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    def test_local_login_success(client_server_network):
-        client, server, network = client_server_network
-
-        assert not client.user_session_manager.local_user_logged_in
-
-        client.user_session_manager.local_login(username="admin", password="admin")
-
-        assert client.user_session_manager.local_user_logged_in
-
-Local Login Failure
-^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    def test_local_login_failure(client_server_network):
-        client, server, network = client_server_network
-
-        assert not client.user_session_manager.local_user_logged_in
-
-        client.user_session_manager.local_login(username="jane.doe", password="12345")
-
-        assert not client.user_session_manager.local_user_logged_in
-
-Adding a New User and Successful Local Login
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    def test_new_user_local_login_success(client_server_network):
-        client, server, network = client_server_network
-
-        assert not client.user_session_manager.local_user_logged_in
-
-        client.user_manager.add_user(username="jane.doe", password="12345")
-
-        client.user_session_manager.local_login(username="jane.doe", password="12345")
-
-        assert client.user_session_manager.local_user_logged_in
-
-Clearing Previous Login on New Local Login
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    def test_new_local_login_clears_previous_login(client_server_network):
-        client, server, network = client_server_network
-
-        assert not client.user_session_manager.local_user_logged_in
-
-        current_session_id = client.user_session_manager.local_login(username="admin", password="admin")
-
-        assert client.user_session_manager.local_user_logged_in
-
-        assert client.user_session_manager.local_session.user.username == "admin"
-
-        client.user_manager.add_user(username="jane.doe", password="12345")
-
-        new_session_id = client.user_session_manager.local_login(username="jane.doe", password="12345")
-
-        assert client.user_session_manager.local_user_logged_in
-
-        assert client.user_session_manager.local_session.user.username == "jane.doe"
-
-        assert new_session_id != current_session_id
-
-Persistent Login for the Same User
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: python
-
-    def test_new_local_login_attempt_same_uses_persists(client_server_network):
-        client, server, network = client_server_network
-
-        assert not client.user_session_manager.local_user_logged_in
-
-        current_session_id = client.user_session_manager.local_login(username="admin", password="admin")
-
-        assert client.user_session_manager.local_user_logged_in
-
-        assert client.user_session_manager.local_session.user.username == "admin"
-
-        new_session_id = client.user_session_manager.local_login(username="admin", password="admin")
-
-        assert client.user_session_manager.local_user_logged_in
-
-        assert client.user_session_manager.local_session.user.username == "admin"
-
-        assert new_session_id == current_session_id

docs/source/simulation_components/network/nodes/host_node.rst

@@ -49,5 +49,3 @@ fundamental network operations:
 5. **NTP (Network Time Protocol) Client:** Synchronises the host's clock with network time servers.
 
 6. **Web Browser:** A simulated application that allows the host to request and display web content.
-
-7. **Terminal:** A simulated service that allows the host to connect to remote hosts and execute commands.

docs/source/simulation_components/network/nodes/wireless_router.rst

@@ -3,7 +3,7 @@
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
 ######
-Wireless Router
+Router
 ######
 
 The ``WirelessRouter`` class extends the functionality of the standard ``Router`` class within PrimAITE,
@@ -37,7 +37,7 @@ additional steps to configure wireless settings:
 .. code-block:: python
 
     from primaite.simulator.network.hardware.nodes.network.wireless_router import WirelessRouter
-    from primaite.simulator.network.airspace import AirSpaceFrequency, ChannelWidth
+    from primaite.simulator.network.airspace import AirSpaceFrequency
 
     # Instantiate the WirelessRouter
     wireless_router = WirelessRouter(hostname="MyWirelessRouter")
@@ -49,7 +49,7 @@ additional steps to configure wireless settings:
     wireless_router.configure_wireless_access_point(
         port=1, ip_address="192.168.2.1",
         subnet_mask="255.255.255.0",
-        frequency=AirSpaceFrequency.WIFI_2_4,
+        frequency=AirSpaceFrequency.WIFI_2_4
     )
 
 
@@ -71,7 +71,7 @@ ICMP traffic, ensuring basic network connectivity and ping functionality.
 
 .. code-block:: python
 
-    from primaite.simulator.network.airspace import AirSpaceFrequency, ChannelWidth
+    from primaite.simulator.network.airspace import AIR_SPACE, AirSpaceFrequency
     from primaite.simulator.network.container import Network
     from primaite.simulator.network.hardware.nodes.host.computer import Computer
     from primaite.simulator.network.hardware.nodes.network.router import ACLAction
@@ -130,13 +130,13 @@ ICMP traffic, ensuring basic network connectivity and ping functionality.
         port=1,
         ip_address="192.168.1.1",
         subnet_mask="255.255.255.0",
-        frequency=AirSpaceFrequency.WIFI_2_4,
+        frequency=AirSpaceFrequency.WIFI_2_4
     )
     router_2.configure_wireless_access_point(
         port=1,
         ip_address="192.168.1.2",
         subnet_mask="255.255.255.0",
-        frequency=AirSpaceFrequency.WIFI_2_4,
+        frequency=AirSpaceFrequency.WIFI_2_4
     )
 
     # Configure routes for inter-router communication

docs/source/simulation_components/system/applications/c2_suite.rst

@@ -1,319 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-.. _C2_Suite:
-
-Command and Control Application Suite
-#####################################
-
-Comprising of two applications, the Command and Control (C2) suite intends to introduce
-malicious network architecture and further the realism of red agents within PrimAITE.
-
-Overview:
-=========
-
-These two new classes give red agents a cyber realistic way of leveraging the capabilities of the ``Terminal`` application whilst introducing more opportunities for the blue agent(s) to notice and subvert a red agent during an episode.
-
-For a more in-depth look at the command and control applications then please refer to the ``C2-E2E-Notebook``.
-
-``C2 Server``
-"""""""""""""
-
-The C2 Server application is intended to represent the malicious infrastructure already under the control of an adversary.
-
-The C2 Server is configured to listen and await ``keep alive`` traffic from a C2 beacon. Once received the C2 Server is able to send and receive C2 commands.
-
-Currently, the C2 Server offers four commands:
-
-+---------------------+---------------------------------------------------------------------------+
-|C2 Command           | Meaning                                                                   |
-+=====================+===========================================================================+
-|RANSOMWARE_CONFIGURE | Configures an installed ransomware script based on the passed parameters. |
-+---------------------+---------------------------------------------------------------------------+
-|RANSOMWARE_LAUNCH    | Launches the installed ransomware script.                                 |
-+---------------------+---------------------------------------------------------------------------+
-|DATA_EXFILTRATION    | Copies a target file from a remote node to the C2 Beacon & Server via FTP |
-+---------------------+---------------------------------------------------------------------------+
-|TERMINAL             | Executes a command via the terminal installed on the C2 Beacons Host.     |
-+---------------------+---------------------------------------------------------------------------+
-
-
-It's important to note that in order to keep PrimAITE realistic from a cyber perspective,
-the C2 Server application should never be visible or actionable upon directly by the blue agent.
-
-This is because in the real world, C2 servers are hosted on ephemeral public domains that would not be accessible by private network blue agent.
-Therefore granting blue agent(s) the ability to perform counter measures directly against the application would be unrealistic.
-
-It is more accurate to see the host that the C2 Beacon is installed on as being able to route to the C2 Server (Internet Access).
-
-``C2 Beacon``
-"""""""""""""
-
-The C2 Beacon application is intended to represent malware that is used to establish and maintain contact to a C2 Server within a compromised network.
-
-A C2 Beacon will need to be first configured with the C2 Server IP Address which can be done via the ``configure`` method.
-
-Once installed and configured; the C2 beacon can establish connection with the C2 Server via executing the application.
-
-This will send an initial ``keep alive`` to the given C2 Server (The C2 Server IPv4Address must be given upon C2 Beacon configuration).
-Which is then resolved and responded by another ``Keep Alive`` by the C2 server back to the C2 beacon to confirm connection.
-
-The C2 Beacon will send out periodic keep alive based on its configuration parameters to configure it's active connection with the C2 server.
-
-It's recommended that a C2 Beacon is installed and configured mid episode by a Red Agent for a more cyber realistic simulation.
-
-Usage
-=====
-
-As mentioned, the C2 Suite is intended to grant Red Agents further flexibility whilst also expanding a blue agent's observation space.
-
-Adding to this, the following behaviour of the C2 beacon can be configured by users for increased domain randomisation:
-
-+---------------------+---------------------------------------------------------------------------+
-|Configuration Option | Option Meaning                                                            |
-+=====================+===========================================================================+
-|c2_server_ip_address | The IP Address of the C2 Server. (The C2 Server must be running)          |
-+---------------------+---------------------------------------------------------------------------+
-|keep_alive_frequency | How often should the C2 Beacon confirm it's connection in timesteps.      |
-+---------------------+---------------------------------------------------------------------------+
-|masquerade_protocol  | What protocol should the C2 traffic masquerade as? (HTTP, FTP or DNS)     |
-+---------------------+---------------------------------------------------------------------------+
-|masquerade_port      | What port should the C2 traffic use? (TCP or UDP)                         |
-+---------------------+---------------------------------------------------------------------------+
-
-
-Implementation
-==============
-
-Both applications inherit from an abstract C2 which handles the keep alive functionality and main logic.
-However, each host implements it's own receive methods.
-
-- The ``C2 Beacon`` is responsible for the following logic:
-    - Establishes and confirms connection to the C2 Server via sending ``C2Payload.KEEP_ALIVE``.
-    - Receives and executes C2 Commands given by the C2 Server via ``C2Payload.INPUT``.
-    - Returns the RequestResponse of the C2 Commands executed back the C2 Server via ``C2Payload.OUTPUT``.
-
-- The ``C2 Server`` is responsible for the following logic:
-    - Listens and resolves connection to a C2 Beacon via responding to ``C2Payload.KEEP_ALIVE``.
-    - Sends C2 Commands to the C2 Beacon via ``C2Payload.INPUT``.
-    - Receives the RequestResponse of the C2 Commands executed by C2 Beacon via ``C2Payload.OUTPUT``.
-
-The sequence diagram below clarifies the functionality of both applications:
-
-.. image:: ../../../../_static/c2_sequence.png
-    :width: 1000
-    :align: center
-
-
-For further details and more in-depth examples please refer to the ``Command-&-Control notebook``
-
-Examples
-========
-
-Python
-""""""
-.. code-block:: python
-
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.network.hardware.nodes.network.switch import Switch
-    from primaite.simulator.system.applications.database_client import DatabaseClient
-    from primaite.simulator.system.applications.red_applications.ransomware_script import RansomwareScript
-    from primaite.simulator.system.services.database.database_service import DatabaseService
-    from primaite.simulator.system.applications.red_applications.c2.c2_server import C2Command, C2Server
-    from primaite.simulator.system.applications.red_applications.c2.c2_beacon import C2Beacon
-
-    # Network Setup
-    network = Network()
-
-
-    switch = Switch(hostname="switch", start_up_duration=0, num_ports=4)
-    switch.power_on()
-
-    node_a = Computer(hostname="node_a", ip_address="192.168.0.10", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_a.power_on()
-    network.connect(node_a.network_interface[1], switch.network_interface[1])
-
-    node_b = Computer(hostname="node_b", ip_address="192.168.0.11", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_b.power_on()
-
-    network.connect(node_b.network_interface[1], switch.network_interface[2])
-
-    node_c = Computer(hostname="node_c", ip_address="192.168.0.12", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_c.power_on()
-    network.connect(node_c.network_interface[1], switch.network_interface[3])
-
-    node_c.software_manager.install(software_class=DatabaseService)
-    node_b.software_manager.install(software_class=DatabaseClient)
-    node_b.software_manager.install(software_class=RansomwareScript)
-    node_a.software_manager.install(software_class=C2Server)
-
-    # C2 Application objects
-
-    c2_server_host: Computer = network.get_node_by_hostname("node_a")
-    c2_beacon_host: Computer = network.get_node_by_hostname("node_b")
-
-    c2_server: C2Server = c2_server_host.software_manager.software["C2Server"]
-    c2_beacon: C2Beacon = c2_beacon_host.software_manager.software["C2Beacon"]
-
-    # Configuring the C2 Beacon
-    c2_beacon.configure(c2_server_ip_address="192.168.0.10", keep_alive_frequency=5)
-
-    # Launching the C2 Server (Needs to be running in order to listen for connections)
-    c2_server.run()
-
-    # Establishing connection
-    c2_beacon.establish()
-
-    # Example command: Creating a file
-
-    file_create_command = {
-        "commands": [
-            ["file_system", "create", "folder", "test_folder"],
-            ["file_system", "create", "file", "test_folder", "example_file", "True"],
-        ],
-        "username": "admin",
-        "password": "admin",
-        "ip_address": None,
-    }
-
-    c2_server.send_command(C2Command.TERMINAL, command_options=file_create_command)
-
-    # Example command: Installing and configuring Ransomware:
-
-    ransomware_installation_command = { "commands": [
-            ["software_manager","application","install","RansomwareScript"],
-        ],
-        "username": "admin",
-        "password": "admin",
-        "ip_address": None,
-    }
-    c2_server.send_command(given_command=C2Command.TERMINAL, command_options=ransomware_installation_command)
-
-    ransomware_config = {"server_ip_address": "192.168.0.12"}
-
-    c2_server.send_command(given_command=C2Command.RANSOMWARE_CONFIGURE, command_options=ransomware_config)
-
-    c2_beacon_host.software_manager.show()
-
-    # Example command: File Exfiltration
-
-    data_exfil_options = {
-        "username": "admin",
-        "password": "admin",
-        "ip_address": None,
-        "target_ip_address": "192.168.0.12",
-        "target_file_name": "database.db",
-        "target_folder_name": "database",
-    }
-
-    c2_server.send_command(given_command=C2Command.DATA_EXFILTRATION, command_options=data_exfil_options)
-
-    # Example command: Launching Ransomware
-
-    c2_server.send_command(given_command=C2Command.RANSOMWARE_LAUNCH, command_options={})
-
-
-
-Via Configuration
-"""""""""""""""""
-
-.. code-block:: yaml
-
-    simulation:
-        network:
-            nodes:
-                - ref: example_computer_1
-                hostname: computer_a
-                type: computer
-                ...
-                applications:
-                    type: C2Server
-            ...
-                hostname: computer_b
-                type: computer
-                ...
-                # A C2 Beacon will not automatically connection to a C2 Server.
-                # Either an agent must use application_execute.
-                # Or a if using the simulation layer - .establish().
-                applications:
-                    type: C2Beacon
-                    options:
-                        c2_server_ip_address: ...
-                        keep_alive_frequency: 5
-                        masquerade_protocol: tcp
-                        masquerade_port: http
-                        listen_on_ports:
-                            - 80
-                            - 53
-                            - 21
-
-
-
-C2 Beacon Configuration
-=======================
-
-``c2_server_ip_address``
-""""""""""""""""""""""""
-
-IP address of the ``C2Server`` that the C2 Beacon will use to establish connection.
-
-This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
-
-
-``Keep Alive Frequency``
-""""""""""""""""""""""""
-
-How often should the C2 Beacon confirm it's connection in timesteps.
-
-For example, if the keep alive Frequency is set to one then every single timestep
-the C2 connection will be confirmed.
-
-It's worth noting that this may be a useful option when investigating
-network blue agent observation space.
-
-This must be a valid integer i.e ``10``. Defaults to ``5``.
-
-
-``Masquerade Protocol``
-"""""""""""""""""""""""
-
-The protocol that the C2 Beacon will use to communicate to the C2 Server with.
-
-Currently only ``TCP`` and ``UDP`` are valid masquerade protocol options.
-
-It's worth noting that this may be a useful option to bypass ACL rules.
-
-This must be a string i.e *UDP*. Defaults to ``TCP``.
-
-*Please refer to the ``IPProtocol`` class for further reference.*
-
-``Masquerade Port``
-"""""""""""""""""""
-
-What port that the C2 Beacon will use to communicate to the C2 Server with.
-
-Currently only ``FTP``, ``HTTP`` and ``DNS`` are valid masquerade port options.
-
-It's worth noting that this may be a useful option to bypass ACL rules.
-
-This must be a string i.e ``DNS``. Defaults to ``HTTP``.
-
-*Please refer to the ``IPProtocol`` class for further reference.*
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`
-
-
-C2 Server Configuration
-=======================
-
-*The C2 Server does not currently offer any unique configuration options and will configure itself to match the C2 beacon's network behaviour.*
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/data_manipulation_bot.rst

@@ -158,6 +158,10 @@ If not using the data manipulation bot manually, it needs to be used with a data
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: DataManipulationBot
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DataManipulationBot``
 
 ``server_ip``
 """""""""""""
@@ -199,8 +203,3 @@ Optional. Default value is ``0.1``.
 The chance of the ``DataManipulationBot`` to succeed with a data manipulation attack.
 
 This must be a float value between ``0`` and ``1``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/database_client.rst

@@ -90,6 +90,11 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: DatabaseClient
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DatabaseClient``
+
 
 ``db_server_ip``
 """"""""""""""""
@@ -104,8 +109,3 @@ This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.25
 Optional. Default value is ``None``.
 
 The password that the ``DatabaseClient`` will use to access the :ref:`DatabaseService`.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/dos_bot.rst

@@ -7,10 +7,7 @@
 DoSBot
 ######
 
-The ``DoSBot`` is an implementation of a Denial of Service attack within the PrimAITE simulation.
-This specifically simulates a `Slow Loris attack`_.
-
-.. _Slow Loris Attack: https://en.wikipedia.org/wiki/Slowloris_(computer_security)
+The ``DoSBot`` is an implementation of a Denial of Service attack within the PrimAITE simulation. This specifically simulates a `Slow Loris attack <https://en.wikipedia.org/wiki/Slowloris_(computer_security)>`.
 
 Key features
 ============
@@ -98,6 +95,11 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: DoSBot
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DoSBot``
+
 ``target_ip_address``
 """""""""""""""""""""
 
@@ -156,8 +158,3 @@ Optional. Default value is ``1000``.
 The maximum number of sessions the ``DoSBot`` is able to make.
 
 This must be an integer value equal to or greater than ``0``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/nmap.rst

@@ -5,10 +5,10 @@
 .. _NMAP:
 
 NMAP
-####
+====
 
 Overview
-========
+--------
 
 The NMAP application is used to simulate network scanning activities. NMAP is a powerful tool that helps in discovering
 hosts and services on a network. It provides functionalities such as ping scans to discover active hosts and port scans
@@ -19,8 +19,8 @@ structure, identify active devices, and find potential vulnerabilities by discov
 However, it is also a tool frequently used by attackers during the reconnaissance stage of a cyber attack to gather
 information about the target network.
 
-Scan Type
-=========
+Scan Types
+----------
 
 Ping Scan
 ^^^^^^^^^
@@ -46,7 +46,7 @@ identifying potential entry points for attacks. There are three types of port sc
   It gives a comprehensive view of the network's service landscape.
 
 Example Usage
-^^^^^^^^^^^^^
+-------------
 
 The network we use for these examples is defined below:
 
@@ -345,9 +345,3 @@ Perform a full box scan on all ports, over both TCP and UDP, on a whole subnet:
    | 192.168.1.13 | 123  | NTP             | UDP      |
    | 192.168.1.13 | 219  | ARP             | UDP      |
    +--------------+------+-----------------+----------+
-
-
-``Common Attributes``
-"""""""""""""""""""""
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/ransomware_script.rst

@@ -1,86 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-.. _RansomwareScript:
-
-RansomwareScript
-###################
-
-The RansomwareScript class provides functionality to connect to a :ref:`DatabaseService` and set a database's database.db into a ``CORRUPTED`` state.
-
-Overview
-========
-
-The ransomware script intends to simulate a generic implementation of ransomware.
-
-Currently, due to simulation restraints, the ransomware script is unable to attack a host without an active database service.
-
-The ransomware script is similar to that of the data_manipulation_bot but does not have any separate stages or configurable probabilities.
-
-Additionally, similar to the data_manipulation_bot, the ransomware script must be installed on a host with a pre-existing :ref:`DatabaseClient` application installed.
-
-Usage
-=====
-
-- Create an instance and call ``configure`` to set:
-    - Target Database IP
-    - Database password (if needed)
-- Call ``Execute`` to connect and execute the ransomware script.
-
-This application handles connections to the database server and the connection made to encrypt the database but it does not handle disconnections.
-
-Implementation
-==============
-
-Currently, the ransomware script connects to a :ref:`DatabaseClient` and leverages its connectivity. The host running ``RansomwareScript`` must also have a :ref:`DatabaseClient` installed on it.
-
-- Uses the Application base class for lifecycle management.
-- Target IP and other options set via ``configure``.
-- ``execute`` handles connecting and encrypting.
-
-
-Examples
-========
-
-Python
-""""""
-.. code-block:: python
-
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
-    from primaite.simulator.system.applications.red_applications.RansomwareScript import RansomwareScript
-    from primaite.simulator.system.applications.database_client import DatabaseClient
-
-    client_1 = Computer(
-        hostname="client_1",
-        ip_address="192.168.10.21",
-        subnet_mask="255.255.255.0",
-        default_gateway="192.168.10.1",
-        operating_state=NodeOperatingState.ON # initialise the computer in an ON state
-    )
-    network.connect(endpoint_b=client_1.network_interface[1], endpoint_a=switch_2.network_interface[1])
-    client_1.software_manager.install(DatabaseClient)
-    client_1.software_manager.install(RansomwareScript)
-    RansomwareScript: RansomwareScript = client_1.software_manager.software.get("RansomwareScript")
-    RansomwareScript.configure(server_ip_address=IPv4Address("192.168.1.14"))
-    RansomwareScript.execute()
-
-
-Configuration
-=============
-
-The RansomwareScript inherits configuration options such as ``fix_duration`` from its parent class. However, for the ``RansomwareScript`` the most relevant option is ``server_ip``.
-
-
-``server_ip``
-"""""""""""""
-
-IP address of the :ref:`DatabaseService` which the ``RansomwareScript`` will encrypt.
-
-This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/applications/web_browser.rst

@@ -23,7 +23,7 @@ Usage
 =====
 
 - Install on a Node via the ``SoftwareManager`` to start the ``WebBrowser``.
-- Service runs on HTTP port 80 by default.
+- Service runs on HTTP port 80 by default. (TODO: HTTPS)
 - Execute sending an HTTP GET request with ``get_webpage``
 
 Implementation
@@ -92,6 +92,10 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: WebBrowser
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``WebBrowser``
 
 ``target_url``
 """"""""""""""
@@ -105,9 +109,3 @@ The domain ``arcd.com`` can be matched by
 - http://arcd.com/
 - http://arcd.com/users/
 - arcd.com
-
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/common/common_configuration.rst

@@ -2,56 +2,17 @@
 
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
-.. _Common Configuration:
+``ref``
+=======
 
-Common Configuration
-""""""""""""""""""""
+Human readable name used as reference for the |SOFTWARE_NAME_BACKTICK|. Not used in code.
 
-ref
-"""
+``type``
+========
 
-Human readable name used as reference for the software class. Not used in code.
+The type of software that should be added. To add |SOFTWARE_NAME| this must be |SOFTWARE_NAME_BACKTICK|.
 
-type
-""""
+``options``
+===========
 
-The type of software that should be added. To add the required software, this must be it's name.
-
-options
-"""""""
-
-The configuration options are the attributes that fall under the options for an application or service.
-
-fix_duration
-""""""""""""
-
-Optional. Default value is ``2``.
-
-The number of timesteps the software will remain in a ``FIXING`` state before going into a ``GOOD`` state.
-
-
-listen_on_ports
-^^^^^^^^^^^^^^^
-
-Optional. The set of ports to listen on. This is in addition to the main port the software is designated. This can either be
-the string name of ports or the port integers
-
-Example:
-
-.. code-block:: yaml
-
-    simulation:
-      network:
-        nodes:
-        - hostname: [hostname]
-          type: [Node Type]
-          services:
-            - type: [Service Type]
-              options:
-                listen_on_ports:
-                  - 631
-          applications:
-              - type: [Application Type]
-                options:
-                  listen_on_ports:
-                    - SMB
+The configuration options are the attributes that fall under the options for an application.

docs/source/simulation_components/system/list_of_applications.rst

@@ -8,7 +8,7 @@
 
     applications/*
 
-More info :py:mod:`primaite.simulator.system.applications.application.Application`
+More info :py:mod:`primaite.game.game.APPLICATION_TYPES_MAPPING`
 
 .. include:: list_of_system_applications.rst
 

docs/source/simulation_components/system/services/database_service.rst

@@ -94,6 +94,11 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: DatabaseService
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DatabaseService``
+
 ``backup_server_ip``
 """"""""""""""""""""
 
@@ -109,8 +114,3 @@ This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.25
 Optional. Default value is ``None``.
 
 The password that needs to be provided by connecting clients in order to create a successful connection.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/dns_client.rst

@@ -84,6 +84,10 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: DNSClient
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DNSClient``
 
 ``dns_server``
 """"""""""""""
@@ -93,8 +97,3 @@ Optional. Default value is ``None``.
 The IP Address of the :ref:`DNSServer`.
 
 This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/dns_server.rst

@@ -83,17 +83,16 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
 
-``domain_mapping``
-""""""""""""""""""
+.. |SOFTWARE_NAME| replace:: DNSServer
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``DNSServer``
+
+domain_mapping
+""""""""""""""
 
 Domain mapping takes the domain and IP Addresses as a key-value pairs i.e.
 
 If the domain is "arcd.com" and the IP Address attributed to the domain is 192.168.0.10, then the value should be ``arcd.com: 192.168.0.10``
 
 The key must be a string and the IP Address must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/ftp_client.rst

@@ -83,7 +83,9 @@ Via Configuration
 Configuration
 =============
 
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
+.. include:: ../common/common_configuration.rst
 
-See :ref:`Common Configuration`
+.. |SOFTWARE_NAME| replace:: FTPClient
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``FTPClient``
+
+**FTPClient has no configuration options**

docs/source/simulation_components/system/services/ftp_server.rst

@@ -81,14 +81,14 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: FTPServer
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``FTPServer``
+
 ``server_password``
 """""""""""""""""""
 
 Optional. Default value is ``None``.
 
 The password that needs to be provided by a connecting :ref:`FTPClient` in order to create a successful connection.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/ntp_client.rst

@@ -80,6 +80,11 @@ Via Configuration
 Configuration
 =============
 
+.. include:: ../common/common_configuration.rst
+
+.. |SOFTWARE_NAME| replace:: NTPClient
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``NTPClient``
+
 ``ntp_server_ip``
 """""""""""""""""
 
@@ -88,8 +93,3 @@ Optional. Default value is ``None``.
 The IP address of an NTP Server which provides a time that the ``NTPClient`` can synchronise to.
 
 This must be a valid octet i.e. in the range of ``0.0.0.0`` and ``255.255.255.255``.
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/ntp_server.rst

@@ -75,8 +75,12 @@ Via Configuration
                     - ref: ntp_server
                     type: NTPServer
 
+Configuration
+=============
 
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
+.. include:: ../common/common_configuration.rst
 
-See :ref:`Common Configuration`
+.. |SOFTWARE_NAME| replace:: NTPServer
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``NTPServer``
+
+**NTPServer has no configuration options**

docs/source/simulation_components/system/services/terminal.rst

@@ -1,181 +0,0 @@
-.. only:: comment
-
-    © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-
-.. _Terminal:
-
-Terminal
-########
-
-The ``Terminal.py`` class provides a generic terminal simulation, by extending the base Service class within PrimAITE. The aim of this is to act as the primary entrypoint for Nodes within the environment.
-
-
-Overview
-========
-
-The Terminal service uses Secure Socket (SSH) as the communication method between terminals. They operate on port 22, and are part of the services automatically
-installed on Nodes when they are instantiated.
-
-Key capabilities
-""""""""""""""""
-
- - Ensures packets are matched to an existing session
- - Simulates common Terminal processes/commands.
- - Leverages the Service base class for install/uninstall, status tracking etc.
-
-
-Implementation
-""""""""""""""
-
- - Manages remote connections in a dictionary by session ID.
- - Processes commands, forwarding to the ``RequestManager`` or ``SessionManager`` where appropriate.
- - Extends Service class.
-  - A detailed guide on the implementation and functionality of the Terminal class can be found in the "Terminal-Processing" jupyter notebook.
-
-
-Usage
-"""""
-
- - Pre-Installs on all ``Nodes`` (with the exception of ``Switches``).
- - Terminal Clients connect, execute commands and disconnect from remote nodes.
- - Ensures that users are logged in to the component before executing any commands.
- - Service runs on SSH port 22 by default.
-
-Usage
-=====
-
-The below code examples demonstrate how to create a terminal, a remote terminal, and how to send a basic application install command to a remote node.
-
-Python
-""""""
-
-.. code-block:: python
-
-    from ipaddress import IPv4Address
-
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.system.services.terminal.terminal import Terminal
-    from primaite.simulator.network.hardware.node_operating_state import NodeOperatingState
-
-    client = Computer(
-        hostname="client",
-        ip_address="192.168.10.21",
-        subnet_mask="255.255.255.0",
-        default_gateway="192.168.10.1",
-        operating_state=NodeOperatingState.ON,
-    )
-
-    terminal: Terminal = client.software_manager.software.get("Terminal")
-
-Creating Remote Terminal Connection
-"""""""""""""""""""""""""""""""""""
-
-
-.. code-block:: python
-
-    from primaite.simulator.system.services.terminal.terminal import Terminal
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection
-
-
-    network = Network()
-    node_a = Computer(hostname="node_a", ip_address="192.168.0.10", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_a.power_on()
-    node_b = Computer(hostname="node_b", ip_address="192.168.0.11", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_b.power_on()
-    network.connect(node_a.network_interface[1], node_b.network_interface[1])
-
-    terminal_a: Terminal = node_a.software_manager.software.get("Terminal")
-
-
-    term_a_term_b_remote_connection: RemoteTerminalConnection = terminal_a.login(username="admin", password="Admin123!", ip_address="192.168.0.11")
-
-
-
-Executing a basic application install command
-"""""""""""""""""""""""""""""""""""""""""""""
-
-.. code-block:: python
-
-    from primaite.simulator.system.services.terminal.terminal import Terminal
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection
-    from primaite.simulator.system.applications.red_applications.ransomware_script import RansomwareScript
-
-
-    network = Network()
-    node_a = Computer(hostname="node_a", ip_address="192.168.0.10", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_a.power_on()
-    node_b = Computer(hostname="node_b", ip_address="192.168.0.11", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_b.power_on()
-    network.connect(node_a.network_interface[1], node_b.network_interface[1])
-
-    terminal_a: Terminal = node_a.software_manager.software.get("Terminal")
-
-
-    term_a_term_b_remote_connection: RemoteTerminalConnection = terminal_a.login(username="admin", password="Admin123!", ip_address="192.168.0.11")
-
-    term_a_term_b_remote_connection.execute(["software_manager", "application", "install", "RansomwareScript"])
-
-
-
-Creating a folder on a remote node
-""""""""""""""""""""""""""""""""""
-
-.. code-block:: python
-
-    from primaite.simulator.system.services.terminal.terminal import Terminal
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection
-    from primaite.simulator.system.applications.red_applications.ransomware_script import RansomwareScript
-
-
-    network = Network()
-    node_a = Computer(hostname="node_a", ip_address="192.168.0.10", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_a.power_on()
-    node_b = Computer(hostname="node_b", ip_address="192.168.0.11", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_b.power_on()
-    network.connect(node_a.network_interface[1], node_b.network_interface[1])
-
-    terminal_a: Terminal = node_a.software_manager.software.get("Terminal")
-
-
-    term_a_term_b_remote_connection: RemoteTerminalConnection = terminal_a.login(username="admin", password="Admin123!", ip_address="192.168.0.11")
-
-    term_a_term_b_remote_connection.execute(["file_system", "create", "folder", "downloads"])
-
-
-Disconnect from Remote Node
-"""""""""""""""""""""""""""
-
-.. code-block:: python
-
-    from primaite.simulator.system.services.terminal.terminal import Terminal
-    from primaite.simulator.network.container import Network
-    from primaite.simulator.network.hardware.nodes.host.computer import Computer
-    from primaite.simulator.system.services.terminal.terminal import RemoteTerminalConnection
-    from primaite.simulator.system.applications.red_applications.ransomware_script import RansomwareScript
-
-
-    network = Network()
-    node_a = Computer(hostname="node_a", ip_address="192.168.0.10", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_a.power_on()
-    node_b = Computer(hostname="node_b", ip_address="192.168.0.11", subnet_mask="255.255.255.0", start_up_duration=0)
-    node_b.power_on()
-    network.connect(node_a.network_interface[1], node_b.network_interface[1])
-
-    terminal_a: Terminal = node_a.software_manager.software.get("Terminal")
-
-
-    term_a_term_b_remote_connection: RemoteTerminalConnection = terminal_a.login(username="admin", password="Admin123!", ip_address="192.168.0.11")
-
-    term_a_term_b_remote_connection.disconnect()
-
-
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
-
-See :ref:`Common Configuration`

docs/source/simulation_components/system/services/web_server.rst

@@ -75,8 +75,12 @@ Via Configuration
                     - ref: web_server
                     type: WebServer
 
+Configuration
+=============
 
-``Common Attributes``
-^^^^^^^^^^^^^^^^^^^^^
+.. include:: ../common/common_configuration.rst
 
-See :ref:`Common Configuration`
+.. |SOFTWARE_NAME| replace:: WebServer
+.. |SOFTWARE_NAME_BACKTICK| replace:: ``WebServer``
+
+**WebServer has no configuration options**

docs/source/simulation_components/system/software.rst

@@ -2,8 +2,6 @@
 
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
-.. _software:
-
 
 Software
 ========
@@ -65,10 +63,3 @@ Processes
 #########
 
 `To be implemented`
-
-Common Software Configuration
-#############################
-
-Below is a list of the common configuration items within Software components of PrimAITE:
-
-.. include:: common/common_configuration.rst

docs/source/simulation_structure.rst

@@ -15,7 +15,7 @@ when a component's ``describe_state()`` method is called, it will include the st
 ``apply_request()`` method can be used to act on a component or one of its descendants. The diagram below shows the
 relationship between components.
 
-.. image:: ../_static/component_relationship.png
+.. image:: ../../_static/component_relationship.png
     :width: 500
     :align: center
     :alt: ::    The top level simulation object owns a NetworkContainer and a DomainController. The DomainController has a

docs/source/varying_config_files.rst

@@ -3,7 +3,7 @@
     © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
 
 Defining variations in the config files
-=======================================
+================
 
 PrimAITE supports the ability to use different variations on a scenario at different episodes. This can be used to increase domain randomisation to prevent overfitting, or to set up curriculum learning to train agents to perform more complicated tasks.
 
@@ -15,7 +15,7 @@ Base scenario
 
 The base scenario is essentially the same as a fixed YAML configuration, but it can contain placeholders that are populated with episode-specific data at runtime. The base scenario contains any network, agent, or settings that remain fixed for the entire training/evaluation session.
 
-The placeholders are defined as YAML Aliases and they are denoted by an asterisk (* *placeholder*)
+The placeholders are defined as YAML Aliases and they are denoted by an asterisk (*placeholder).
 
 Variations
 **********
@@ -46,4 +46,4 @@ It takes the following format:
 
 For more information please refer to the ``Using Episode Schedules`` notebook in either :ref:`Executed Notebooks` or run the notebook interactively in ``notebooks/example_notebooks/``.
 
-For further information around notebooks in general refer to the :ref:`example_notebooks` page.
+For further information around notebooks in general refer to the :ref:`Example Jupyter Notebooks`.

pyproject.toml

@@ -52,10 +52,9 @@ license-files = ["LICENSE"]
 
 [project.optional-dependencies]
 rl = [
-    "ray[rllib] >= 2.20.0, <2.33",
+    "ray[rllib] >= 2.20.0, < 3",
     "tensorflow==2.12.0",
     "stable-baselines3[extra]==2.1.0",
-    "sb3-contrib==2.1.0",
 ]
 dev = [
     "build==0.10.0",
@@ -65,6 +64,7 @@ dev = [
     "gputil==1.4.0",
     "pip-licenses==4.3.0",
     "pre-commit==2.20.0",
+    "pylatex==1.4.1",
     "pytest==7.2.0",
     "pytest-xdist==3.3.1",
     "pytest-cov==4.0.0",
@@ -73,10 +73,7 @@ dev = [
     "Sphinx==7.1.2",
     "sphinx-copybutton==0.5.2",
     "wheel==0.38.4",
-    "nbsphinx==0.9.4",
-    "nbmake==1.5.4",
-    "pytest-xdist==3.3.1",
-    "md2pdf",
+    "nbsphinx==0.9.4"
 ]
 
 [project.scripts]

run_test_and_coverage.py

@@ -1,22 +0,0 @@
-# © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-import subprocess
-import sys
-from typing import Any
-
-
-def run_command(command: Any):
-    """Runs a command and returns the exit code."""
-    result = subprocess.run(command, shell=True)
-    if result.returncode != 0:
-        sys.exit(result.returncode)
-
-
-# Run pytest with coverage
-run_command(
-    "coverage run -m --source=primaite pytest -v -o junit_family=xunit2 "
-    "--junitxml=junit/test-results.xml --cov-fail-under=80"
-)
-
-# Generate coverage reports if tests passed
-run_command("coverage xml -o coverage.xml -i")
-run_command("coverage html -d htmlcov -i")

src/primaite/VERSION

@@ -1 +1 @@
-3.3.0
+3.1.0

src/primaite/config/_package_data/data_manipulation.yaml

@@ -739,9 +739,10 @@ agents:
           options:
             agent_name: client_2_green_user
 
+
+
     agent_settings:
       flatten_obs: true
-      action_masking: true
 
 
 

src/primaite/config/_package_data/data_manipulation_marl.yaml

@@ -733,7 +733,6 @@ agents:
 
     agent_settings:
       flatten_obs: true
-      action_masking: true
 
   - ref: defender_2
     team: BLUE
@@ -1317,7 +1316,6 @@ agents:
 
     agent_settings:
       flatten_obs: true
-      action_masking: true
 
 
 

src/primaite/config/_package_data/multi_lan_internet_network_example.yaml

@@ -25,7 +25,7 @@ simulation:
               db_server_ip: 10.10.1.11
           - type: WebBrowser
             options:
-              target_url: http://sometech.ai/users/
+              target_url: http://sometech.ai
 
       - hostname: pc_2
         type: computer
@@ -39,7 +39,7 @@ simulation:
               db_server_ip: 10.10.1.11
           - type: WebBrowser
             options:
-              target_url: http://sometech.ai/users/
+              target_url: http://sometech.ai
 
       - hostname: server_1
         type: server
@@ -221,7 +221,7 @@ simulation:
             subnet_mask: 255.255.255.0
 
         acl:
-          11: # Allow the some_tech_web_srv to connect to the Database Service on some_tech_db_srv
+          2: # Allow the some_tech_web_srv to connect to the Database Service on some_tech_db_srv
             action: PERMIT
             src_ip: 94.10.180.6
             src_wildcard_mask: 0.0.0.0
@@ -229,7 +229,7 @@ simulation:
             dst_ip: 10.10.1.11
             dst_wildcard_mask: 0.0.0.0
             dst_port: POSTGRES_SERVER
-          12: # Allow the Database Service on some_tech_db_srv to respond to some_tech_web_srv
+          3: # Allow the Database Service on some_tech_db_srv to respond to some_tech_web_srv
             action: PERMIT
             src_ip: 10.10.1.11
             src_wildcard_mask: 0.0.0.0
@@ -237,7 +237,7 @@ simulation:
             dst_ip: 94.10.180.6
             dst_wildcard_mask: 0.0.0.0
             dst_port: POSTGRES_SERVER
-          13: # Prevent the Junior engineer from downloading files from the some_tech_storage_srv over FTP
+          4: # Prevent the Junior engineer from downloading files from the some_tech_storage_srv over FTP
             action: DENY
             src_ip: 10.10.2.12
             src_wildcard_mask: 0.0.0.0
@@ -245,41 +245,33 @@ simulation:
             dst_ip: 10.10.1.12
             dst_wildcard_mask: 0.0.0.0
             dst_port: FTP
-          14: # Prevent the Junior engineer from connecting to some_tech_storage_srv over SSH
-            action: DENY
-            src_ip: 10.10.2.12
-            src_wildcard_mask: 0.0.0.0
-            src_port: SSH
-            dst_ip: 10.10.1.12
-            dst_wildcard_mask: 0.0.0.0
-            dst_port: SSH
-          15: # Allow communication between Engineering and the DB & Storage subnet
+          5: # Allow communication between Engineering and the DB & Storage subnet
             action: PERMIT
             src_ip: 10.10.2.0
             src_wildcard_mask: 0.0.0.255
             dst_ip: 10.10.1.0
             dst_wildcard_mask: 0.0.0.255
-          16: # Allow communication between the DB & Storage subnet and Engineering
+          6: # Allow communication between the DB & Storage subnet and Engineering
             action: PERMIT
             src_ip: 10.10.1.0
             src_wildcard_mask: 0.0.0.255
             dst_ip: 10.10.2.0
             dst_wildcard_mask: 0.0.0.255
-          17: # Allow the SomeTech network to use HTTP
+          7: # Allow the SomeTech network to use HTTP
             action: PERMIT
             src_port: HTTP
             dst_port: HTTP
-          18: # Allow the SomeTech internal network to use ARP
+          8: # Allow the SomeTech internal network to use ARP
             action: PERMIT
             src_ip: 10.10.0.0
             src_wildcard_mask: 0.0.255.255
             src_port: ARP
-          19: # Allow the SomeTech internal network to use ICMP
+          9: # Allow the SomeTech internal network to use ICMP
             action: PERMIT
             src_ip: 10.10.0.0
             src_wildcard_mask: 0.0.255.255
             protocol: ICMP
-          21:
+          10:
             action: PERMIT
             src_ip: 94.10.180.6
             src_wildcard_mask: 0.0.0.0
@@ -287,14 +279,10 @@ simulation:
             dst_ip: 10.10.0.0
             dst_wildcard_mask: 0.0.255.255
             dst_port: HTTP
-          22: # Permit SomeTech to use DNS
+          11: # Permit SomeTech to use DNS
             action: PERMIT
             src_port: DNS
             dst_port: DNS
-          23: # Permit SomeTech to use SSH
-            action: PERMIT
-            src_port: SSH
-            dst_port: SSH
         default_route: # Default route to all external networks
           next_hop_ip_address: 10.10.4.2 # NI int on some_tech_fw
 
@@ -344,7 +332,7 @@ simulation:
               db_server_ip: 10.10.1.11
           - type: WebBrowser
             options:
-              target_url: http://sometech.ai/users/
+              target_url: http://sometech.ai
 
       - hostname: some_tech_snr_dev_pc
         type: computer
@@ -358,7 +346,7 @@ simulation:
               db_server_ip: 10.10.1.11
           - type: WebBrowser
             options:
-              target_url: http://sometech.ai/users/
+              target_url: http://sometech.ai
 
       - hostname: some_tech_jnr_dev_pc
         type: computer
@@ -372,7 +360,7 @@ simulation:
               db_server_ip: 10.10.1.11
           - type: WebBrowser
             options:
-              target_url: http://sometech.ai/users/
+              target_url: http://sometech.ai
 
     links:
       # Home/Office Lan Links

src/primaite/config/_package_data/scenario_with_placeholders/scenario.yaml

@@ -129,10 +129,6 @@ agents:
 
 simulation:
   network:
-    nmne_config:
-      capture_nmne: true
-      nmne_capture_keywords:
-        - DELETE
     nodes:
     - hostname: client
       type: computer

src/primaite/config/load.py

@@ -44,18 +44,3 @@ def data_manipulation_config_path() -> Path:
         _LOGGER.error(msg)
         raise FileNotFoundError(msg)
     return path
-
-
-def data_manipulation_marl_config_path() -> Path:
-    """
-    Get the path to the MARL example config.
-
-    :return: Path to yaml config file for the MARL scenario.
-    :rtype: Path
-    """
-    path = _EXAMPLE_CFG / "data_manipulation_marl.yaml"
-    if not path.exists():
-        msg = f"Example config does not exist: {path}. Have you run `primaite setup`?"
-        _LOGGER.error(msg)
-        raise FileNotFoundError(msg)
-    return path

src/primaite/game/agent/actions.py

@@ -14,10 +14,9 @@ from abc import ABC, abstractmethod
 from typing import Dict, List, Literal, Optional, Tuple, TYPE_CHECKING, Union
 
 from gymnasium import spaces
-from pydantic import BaseModel, ConfigDict, Field, field_validator, ValidationInfo
+from pydantic import BaseModel, Field, field_validator, ValidationInfo
 
 from primaite import getLogger
-from primaite.interface.request import RequestFormat
 
 _LOGGER = getLogger(__name__)
 
@@ -49,7 +48,7 @@ class AbstractAction(ABC):
         objects."""
 
     @abstractmethod
-    def form_request(self) -> RequestFormat:
+    def form_request(self) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         return []
 
@@ -67,7 +66,7 @@ class DoNothingAction(AbstractAction):
         # i.e. a choice between one option. To make enumerating this action easier, we are adding a 'dummy' paramter
         # with one option. This just aids the Action Manager to enumerate all possibilities.
 
-    def form_request(self, **kwargs) -> RequestFormat:
+    def form_request(self, **kwargs) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         return ["do_nothing"]
 
@@ -86,7 +85,7 @@ class NodeServiceAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes, "service_id": num_services}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int, service_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, service_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         service_name = self.manager.get_service_name_by_idx(node_id, service_id)
@@ -181,7 +180,7 @@ class NodeApplicationAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes, "application_id": num_applications}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int, application_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, application_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         application_name = self.manager.get_application_name_by_idx(node_id, application_id)
@@ -229,7 +228,7 @@ class NodeApplicationInstallAction(AbstractAction):
         super().__init__(manager=manager)
         self.shape: Dict[str, int] = {"node_id": num_nodes}
 
-    def form_request(self, node_id: int, application_name: str) -> RequestFormat:
+    def form_request(self, node_id: int, application_name: str, ip_address: str) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         if node_name is None:
@@ -242,81 +241,10 @@ class NodeApplicationInstallAction(AbstractAction):
             "application",
             "install",
             application_name,
+            ip_address,
         ]
 
 
-class ConfigureDatabaseClientAction(AbstractAction):
-    """Action which sets config parameters for a database client on a node."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this action."""
-
-        model_config = ConfigDict(extra="forbid")
-        server_ip_address: Optional[str] = None
-        server_password: Optional[str] = None
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, config: Dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        ConfigureDatabaseClientAction._Opts.model_validate(config)  # check that options adhere to schema
-        return ["network", "node", node_name, "application", "DatabaseClient", "configure", config]
-
-
-class ConfigureRansomwareScriptAction(AbstractAction):
-    """Action which sets config parameters for a ransomware script on a node."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this option."""
-
-        model_config = ConfigDict(extra="forbid")
-        server_ip_address: Optional[str] = None
-        server_password: Optional[str] = None
-        payload: Optional[str] = None
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, config: Dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        ConfigureRansomwareScriptAction._Opts.model_validate(config)  # check that options adhere to schema
-        return ["network", "node", node_name, "application", "RansomwareScript", "configure", config]
-
-
-class ConfigureDoSBotAction(AbstractAction):
-    """Action which sets config parameters for a DoS bot on a node."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this action."""
-
-        model_config = ConfigDict(extra="forbid")
-        target_ip_address: Optional[str] = None
-        target_port: Optional[str] = None
-        payload: Optional[str] = None
-        repeat: Optional[bool] = None
-        port_scan_p_of_success: Optional[float] = None
-        dos_intensity: Optional[float] = None
-        max_sessions: Optional[int] = None
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, config: Dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        self._Opts.model_validate(config)  # check that options adhere to schema
-        return ["network", "node", node_name, "application", "DoSBot", "configure", config]
-
-
 class NodeApplicationRemoveAction(AbstractAction):
     """Action which removes/uninstalls an application."""
 
@@ -324,7 +252,7 @@ class NodeApplicationRemoveAction(AbstractAction):
         super().__init__(manager=manager)
         self.shape: Dict[str, int] = {"node_id": num_nodes}
 
-    def form_request(self, node_id: int, application_name: str) -> RequestFormat:
+    def form_request(self, node_id: int, application_name: str) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         if node_name is None:
@@ -346,7 +274,7 @@ class NodeFolderAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes, "folder_id": num_folders}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int, folder_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, folder_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         folder_name = self.manager.get_folder_name_by_idx(node_idx=node_id, folder_idx=folder_id)
@@ -394,9 +322,7 @@ class NodeFileCreateAction(AbstractAction):
         super().__init__(manager, num_nodes=num_nodes, num_folders=num_folders, **kwargs)
         self.verb: str = "create"
 
-    def form_request(
-        self, node_id: int, folder_name: str, file_name: str, force: Optional[bool] = False
-    ) -> RequestFormat:
+    def form_request(self, node_id: int, folder_name: str, file_name: str, force: Optional[bool] = False) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         if node_name is None or folder_name is None or file_name is None:
@@ -411,7 +337,7 @@ class NodeFolderCreateAction(AbstractAction):
         super().__init__(manager, num_nodes=num_nodes, num_folders=num_folders, **kwargs)
         self.verb: str = "create"
 
-    def form_request(self, node_id: int, folder_name: str) -> RequestFormat:
+    def form_request(self, node_id: int, folder_name: str) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         if node_name is None or folder_name is None:
@@ -432,7 +358,7 @@ class NodeFileAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes, "folder_id": num_folders, "file_id": num_files}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int, folder_id: int, file_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, folder_id: int, file_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         folder_name = self.manager.get_folder_name_by_idx(node_idx=node_id, folder_idx=folder_id)
@@ -465,7 +391,7 @@ class NodeFileDeleteAction(NodeFileAbstractAction):
         super().__init__(manager, num_nodes=num_nodes, num_folders=num_folders, num_files=num_files, **kwargs)
         self.verb: str = "delete"
 
-    def form_request(self, node_id: int, folder_id: int, file_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, folder_id: int, file_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         folder_name = self.manager.get_folder_name_by_idx(node_idx=node_id, folder_idx=folder_id)
@@ -506,7 +432,7 @@ class NodeFileAccessAction(AbstractAction):
         super().__init__(manager, num_nodes=num_nodes, num_folders=num_folders, **kwargs)
         self.verb: str = "access"
 
-    def form_request(self, node_id: int, folder_name: str, file_name: str) -> RequestFormat:
+    def form_request(self, node_id: int, folder_name: str, file_name: str) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         if node_name is None or folder_name is None or file_name is None:
@@ -527,7 +453,7 @@ class NodeAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int) -> RequestFormat:
+    def form_request(self, node_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_id)
         return ["network", "node", node_name, self.verb]
@@ -742,7 +668,7 @@ class RouterACLRemoveRuleAction(AbstractAction):
         super().__init__(manager=manager)
         self.shape: Dict[str, int] = {"position": max_acl_rules}
 
-    def form_request(self, target_router: str, position: int) -> RequestFormat:
+    def form_request(self, target_router: str, position: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         return ["network", "node", target_router, "acl", "remove_rule", position]
 
@@ -925,7 +851,7 @@ class HostNICAbstractAction(AbstractAction):
         self.shape: Dict[str, int] = {"node_id": num_nodes, "nic_id": max_nics_per_node}
         self.verb: str  # define but don't initialise: defends against children classes not defining this
 
-    def form_request(self, node_id: int, nic_id: int) -> RequestFormat:
+    def form_request(self, node_id: int, nic_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         node_name = self.manager.get_node_name_by_idx(node_idx=node_id)
         nic_num = self.manager.get_nic_num_by_idx(node_idx=node_id, nic_idx=nic_id)
@@ -962,7 +888,7 @@ class NetworkPortEnableAction(AbstractAction):
         super().__init__(manager=manager)
         self.shape: Dict[str, int] = {"port_id": max_nics_per_node}
 
-    def form_request(self, target_nodename: str, port_id: int) -> RequestFormat:
+    def form_request(self, target_nodename: str, port_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         if target_nodename is None or port_id is None:
             return ["do_nothing"]
@@ -981,7 +907,7 @@ class NetworkPortDisableAction(AbstractAction):
         super().__init__(manager=manager)
         self.shape: Dict[str, int] = {"port_id": max_nics_per_node}
 
-    def form_request(self, target_nodename: str, port_id: int) -> RequestFormat:
+    def form_request(self, target_nodename: str, port_id: int) -> List[str]:
         """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
         if target_nodename is None or port_id is None:
             return ["do_nothing"]
@@ -1071,247 +997,6 @@ class NodeNetworkServiceReconAction(AbstractAction):
         ]
 
 
-class ConfigureC2BeaconAction(AbstractAction):
-    """Action which configures a C2 Beacon based on the parameters given."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this action."""
-
-        c2_server_ip_address: str
-        keep_alive_frequency: int = Field(default=5, ge=1)
-        masquerade_protocol: str = Field(default="TCP")
-        masquerade_port: str = Field(default="HTTP")
-
-        @field_validator(
-            "c2_server_ip_address",
-            "keep_alive_frequency",
-            "masquerade_protocol",
-            "masquerade_port",
-            mode="before",
-        )
-        @classmethod
-        def not_none(cls, v: str, info: ValidationInfo) -> int:
-            """If None is passed, use the default value instead."""
-            if v is None:
-                return cls.model_fields[info.field_name].default
-            return v
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, config: Dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        config = ConfigureC2BeaconAction._Opts(
-            c2_server_ip_address=config["c2_server_ip_address"],
-            keep_alive_frequency=config["keep_alive_frequency"],
-            masquerade_port=config["masquerade_port"],
-            masquerade_protocol=config["masquerade_protocol"],
-        )
-
-        ConfigureC2BeaconAction._Opts.model_validate(config)  # check that options adhere to schema
-
-        return ["network", "node", node_name, "application", "C2Beacon", "configure", config.__dict__]
-
-
-class NodeAccountsChangePasswordAction(AbstractAction):
-    """Action which changes the password for a user."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: str, username: str, current_password: str, new_password: str) -> RequestFormat:
-        """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        return [
-            "network",
-            "node",
-            node_name,
-            "service",
-            "UserManager",
-            "change_password",
-            username,
-            current_password,
-            new_password,
-        ]
-
-
-class NodeSessionsRemoteLoginAction(AbstractAction):
-    """Action which performs a remote session login."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: str, username: str, password: str, remote_ip: str) -> RequestFormat:
-        """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        return [
-            "network",
-            "node",
-            node_name,
-            "service",
-            "Terminal",
-            "ssh_to_remote",
-            username,
-            password,
-            remote_ip,
-        ]
-
-
-class NodeSessionsRemoteLogoutAction(AbstractAction):
-    """Action which performs a remote session logout."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: str, remote_ip: str) -> RequestFormat:
-        """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        return ["network", "node", node_name, "service", "Terminal", "remote_logoff", remote_ip]
-
-
-class RansomwareConfigureC2ServerAction(AbstractAction):
-    """Action which sends a command from the C2 Server to the C2 Beacon which configures a local RansomwareScript."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, config: Dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        # Using the ransomware scripts model to validate.
-        ConfigureRansomwareScriptAction._Opts.model_validate(config)  # check that options adhere to schema
-        return ["network", "node", node_name, "application", "C2Server", "ransomware_configure", config]
-
-
-class RansomwareLaunchC2ServerAction(AbstractAction):
-    """Action which causes the C2 Server to send a command to the C2 Beacon to launch the RansomwareScript."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        # This action currently doesn't require any further configuration options.
-        return ["network", "node", node_name, "application", "C2Server", "ransomware_launch"]
-
-
-class ExfiltrationC2ServerAction(AbstractAction):
-    """Action which exfiltrates a target file from a certain node onto the C2 beacon and then the C2 Server."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this action."""
-
-        username: Optional[str]
-        password: Optional[str]
-        target_ip_address: str
-        target_file_name: str
-        target_folder_name: str
-        exfiltration_folder_name: Optional[str]
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(
-        self,
-        node_id: int,
-        account: dict,
-        target_ip_address: str,
-        target_file_name: str,
-        target_folder_name: str,
-        exfiltration_folder_name: Optional[str],
-    ) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-
-        command_model = {
-            "target_file_name": target_file_name,
-            "target_folder_name": target_folder_name,
-            "exfiltration_folder_name": exfiltration_folder_name,
-            "target_ip_address": target_ip_address,
-            "username": account["username"],
-            "password": account["password"],
-        }
-        ExfiltrationC2ServerAction._Opts.model_validate(command_model)
-        return ["network", "node", node_name, "application", "C2Server", "exfiltrate", command_model]
-
-
-class NodeSendRemoteCommandAction(AbstractAction):
-    """Action which sends a terminal command to a remote node via SSH."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, remote_ip: str, command: RequestFormat) -> RequestFormat:
-        """Return the action formatted as a request which can be ingested by the PrimAITE simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        return [
-            "network",
-            "node",
-            node_name,
-            "service",
-            "Terminal",
-            "send_remote_command",
-            remote_ip,
-            {"command": command},
-        ]
-
-
-class TerminalC2ServerAction(AbstractAction):
-    """Action which causes the C2 Server to send a command to the C2 Beacon to execute the terminal command passed."""
-
-    class _Opts(BaseModel):
-        """Schema for options that can be passed to this action."""
-
-        commands: Union[List[RequestFormat], RequestFormat]
-        ip_address: Optional[str]
-        username: Optional[str]
-        password: Optional[str]
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int, commands: List, ip_address: Optional[str], account: dict) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-
-        command_model = {
-            "commands": commands,
-            "ip_address": ip_address,
-            "username": account["username"],
-            "password": account["password"],
-        }
-
-        TerminalC2ServerAction._Opts.model_validate(command_model)
-        return ["network", "node", node_name, "application", "C2Server", "terminal_command", command_model]
-
-
-class RansomwareLaunchC2ServerAction(AbstractAction):
-    """Action which causes the C2 Server to send a command to the C2 Beacon to launch the RansomwareScript."""
-
-    def __init__(self, manager: "ActionManager", **kwargs) -> None:
-        super().__init__(manager=manager)
-
-    def form_request(self, node_id: int) -> RequestFormat:
-        """Return the action formatted as a request that can be ingested by the simulation."""
-        node_name = self.manager.get_node_name_by_idx(node_id)
-        if node_name is None:
-            return ["do_nothing"]
-        # This action currently doesn't require any further configuration options.
-        return ["network", "node", node_name, "application", "C2Server", "ransomware_launch"]
-
-
 class ActionManager:
     """Class which manages the action space for an agent."""
 
@@ -1360,18 +1045,6 @@ class ActionManager:
         "NODE_NMAP_PING_SCAN": NodeNMAPPingScanAction,
         "NODE_NMAP_PORT_SCAN": NodeNMAPPortScanAction,
         "NODE_NMAP_NETWORK_SERVICE_RECON": NodeNetworkServiceReconAction,
-        "CONFIGURE_DATABASE_CLIENT": ConfigureDatabaseClientAction,
-        "CONFIGURE_RANSOMWARE_SCRIPT": ConfigureRansomwareScriptAction,
-        "CONFIGURE_DOSBOT": ConfigureDoSBotAction,
-        "CONFIGURE_C2_BEACON": ConfigureC2BeaconAction,
-        "C2_SERVER_RANSOMWARE_LAUNCH": RansomwareLaunchC2ServerAction,
-        "C2_SERVER_RANSOMWARE_CONFIGURE": RansomwareConfigureC2ServerAction,
-        "C2_SERVER_TERMINAL_COMMAND": TerminalC2ServerAction,
-        "C2_SERVER_DATA_EXFILTRATE": ExfiltrationC2ServerAction,
-        "NODE_ACCOUNTS_CHANGE_PASSWORD": NodeAccountsChangePasswordAction,
-        "SSH_TO_REMOTE": NodeSessionsRemoteLoginAction,
-        "SESSIONS_REMOTE_LOGOFF": NodeSessionsRemoteLogoutAction,
-        "NODE_SEND_REMOTE_COMMAND": NodeSendRemoteCommandAction,
     }
     """Dictionary which maps action type strings to the corresponding action class."""
 
@@ -1567,7 +1240,7 @@ class ActionManager:
         act_identifier, act_options = self.action_map[action]
         return act_identifier, act_options
 
-    def form_request(self, action_identifier: str, action_options: Dict) -> RequestFormat:
+    def form_request(self, action_identifier: str, action_options: Dict) -> List[str]:
         """Take action in CAOS format and use the execution definition to change it into PrimAITE request format."""
         act_obj = self.actions[action_identifier]
         return act_obj.form_request(**action_options)

src/primaite/game/agent/agent_log.py

@@ -1,188 +0,0 @@
-# © Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
-import logging
-from pathlib import Path
-
-from prettytable import MARKDOWN, PrettyTable
-
-from primaite.simulator import LogLevel, SIM_OUTPUT
-
-
-class _NotJSONFilter(logging.Filter):
-    def filter(self, record: logging.LogRecord) -> bool:
-        """
-        Determines if a log message does not start and end with '{' and '}' (i.e., it is not a JSON-like message).
-
-        :param record: LogRecord object containing all the information pertinent to the event being logged.
-        :return: True if log message is not JSON-like, False otherwise.
-        """
-        return not record.getMessage().startswith("{") and not record.getMessage().endswith("}")
-
-
-class AgentLog:
-    """
-    A Agent Log class is a simple logger dedicated to managing and writing logging updates and information for an agent.
-
-    Each log message is written to a file located at: <simulation output directory>/agent_name/agent_name.log
-    """
-
-    def __init__(self, agent_name: str):
-        """
-        Constructs a Agent Log instance for a given hostname.
-
-        :param hostname: The hostname associated with the system logs being recorded.
-        """
-        self.agent_name = agent_name
-        self.current_episode: int = 1
-        self.current_timestep: int = 0
-        self.setup_logger()
-
-    @property
-    def timestep(self) -> int:
-        """Returns the current timestep. Used for log indexing.
-
-        :return: The current timestep as an Int.
-        """
-        return self.current_timestep
-
-    def update_timestep(self, new_timestep: int):
-        """
-        Updates the self.current_timestep attribute with the given parameter.
-
-        This method is called within .step() to ensure that all instances of Agent Logs
-        are in sync with one another.
-
-        :param new_timestep: The new timestep.
-        """
-        self.current_timestep = new_timestep
-
-    def setup_logger(self):
-        """
-        Configures the logger for this Agent Log instance.
-
-        The logger is set to the DEBUG level, and is equipped with a handler that writes to a file and filters out
-        JSON-like messages.
-        """
-        if not SIM_OUTPUT.save_agent_logs:
-            return
-
-        log_path = self._get_log_path()
-        file_handler = logging.FileHandler(filename=log_path)
-        file_handler.setLevel(logging.DEBUG)
-
-        log_format = "%(timestep)s::%(levelname)s::%(message)s"
-        file_handler.setFormatter(logging.Formatter(log_format))
-
-        self.logger = logging.getLogger(f"{self.agent_name}_log")
-        for handler in self.logger.handlers:
-            self.logger.removeHandler(handler)
-        self.logger.setLevel(logging.DEBUG)
-        self.logger.addHandler(file_handler)
-
-    def _get_log_path(self) -> Path:
-        """
-        Constructs the path for the log file based on the agent name.
-
-        :return: Path object representing the location of the log file.
-        """
-        root = SIM_OUTPUT.agent_behaviour_path / f"episode_{self.current_episode}" / self.agent_name
-        root.mkdir(exist_ok=True, parents=True)
-        return root / f"{self.agent_name}.log"
-
-    def _write_to_terminal(self, msg: str, level: str, to_terminal: bool = False):
-        if to_terminal or SIM_OUTPUT.write_agent_log_to_terminal:
-            print(f"{self.agent_name}: ({ self.timestep}) ({level}) {msg}")
-
-    def debug(self, msg: str, to_terminal: bool = False):
-        """
-        Logs a message with the DEBUG level.
-
-        :param msg: The message to be logged.
-        :param to_terminal: If True, prints to the terminal too.
-        """
-        if SIM_OUTPUT.agent_log_level > LogLevel.DEBUG:
-            return
-
-        if SIM_OUTPUT.save_agent_logs:
-            self.logger.debug(msg, extra={"timestep": self.timestep})
-        self._write_to_terminal(msg, "DEBUG", to_terminal)
-
-    def info(self, msg: str, to_terminal: bool = False):
-        """
-        Logs a message with the INFO level.
-
-        :param msg: The message to be logged.
-        :param timestep: The current timestep.
-        :param to_terminal: If True, prints to the terminal too.
-        """
-        if SIM_OUTPUT.agent_log_level > LogLevel.INFO:
-            return
-
-        if SIM_OUTPUT.save_agent_logs:
-            self.logger.info(msg, extra={"timestep": self.timestep})
-        self._write_to_terminal(msg, "INFO", to_terminal)
-
-    def warning(self, msg: str, to_terminal: bool = False):
-        """
-        Logs a message with the WARNING level.
-
-        :param msg: The message to be logged.
-        :param timestep: The current timestep.
-        :param to_terminal: If True, prints to the terminal too.
-        """
-        if SIM_OUTPUT.agent_log_level > LogLevel.WARNING:
-            return
-
-        if SIM_OUTPUT.save_agent_logs:
-            self.logger.warning(msg, extra={"timestep": self.timestep})
-        self._write_to_terminal(msg, "WARNING", to_terminal)
-
-    def error(self, msg: str, to_terminal: bool = False):
-        """
-        Logs a message with the ERROR level.
-
-        :param msg: The message to be logged.
-        :param timestep: The current timestep.
-        :param to_terminal: If True, prints to the terminal too.
-        """
-        if SIM_OUTPUT.agent_log_level > LogLevel.ERROR:
-            return
-
-        if SIM_OUTPUT.save_agent_logs:
-            self.logger.error(msg, extra={"timestep": self.timestep})
-        self._write_to_terminal(msg, "ERROR", to_terminal)
-
-    def critical(self, msg: str, to_terminal: bool = False):
-        """
-        Logs a message with the CRITICAL level.
-
-        :param msg: The message to be logged.
-        :param timestep: The current timestep.
-        :param to_terminal: If True, prints to the terminal too.
-        """
-        if LogLevel.CRITICAL < SIM_OUTPUT.agent_log_level:
-            return
-
-        if SIM_OUTPUT.save_agent_logs:
-            self.logger.critical(msg, extra={"timestep": self.timestep})
-        self._write_to_terminal(msg, "CRITICAL", to_terminal)
-
-    def show(self, last_n: int = 10, markdown: bool = False):
-        """
-        Print an Agents Log as a table.
-
-        Generate and print PrettyTable instance that shows the agents behaviour log, with columns Time step,
-        Level and Message.
-
-        :param markdown: Use Markdown style in table output. Defaults to False.
-        """
-        table = PrettyTable(["Time Step", "Level", "Message"])
-        if markdown:
-            table.set_style(MARKDOWN)
-        table.align = "l"
-        table.title = f"{self.agent_name} Behaviour Log"
-        if self._get_log_path().exists():
-            with open(self._get_log_path()) as file:
-                lines = file.readlines()
-            for line in lines[-last_n:]:
-                table.add_row(line.strip().split("::"))
-        print(table)

src/primaite/game/agent/interface.py

@@ -7,7 +7,6 @@ from gymnasium.core import ActType, ObsType
 from pydantic import BaseModel, model_validator
 
 from primaite.game.agent.actions import ActionManager
-from primaite.game.agent.agent_log import AgentLog
 from primaite.game.agent.observations.observation_manager import ObservationManager
 from primaite.game.agent.rewards import RewardFunction
 from primaite.interface.request import RequestFormat, RequestResponse
@@ -36,8 +35,6 @@ class AgentHistoryItem(BaseModel):
 
     reward: Optional[float] = None
 
-    reward_info: Dict[str, Any] = {}
-
 
 class AgentStartSettings(BaseModel):
     """Configuration values for when an agent starts performing actions."""
@@ -72,8 +69,6 @@ class AgentSettings(BaseModel):
     "Configuration for when an agent begins performing it's actions"
     flatten_obs: bool = True
     "Whether to flatten the observation space before passing it to the agent. True by default."
-    action_masking: bool = False
-    "Whether to return action masks at each step."
 
     @classmethod
     def from_config(cls, config: Optional[Dict]) -> "AgentSettings":
@@ -121,7 +116,6 @@ class AbstractAgent(ABC):
         self.reward_function: Optional[RewardFunction] = reward_function
         self.agent_settings = agent_settings or AgentSettings()
         self.history: List[AgentHistoryItem] = []
-        self.logger = AgentLog(agent_name)
 
     def update_observation(self, state: Dict) -> ObsType:
         """
@@ -211,7 +205,6 @@ class ProxyAgent(AbstractAgent):
         )
         self.most_recent_action: ActType
         self.flatten_obs: bool = agent_settings.flatten_obs if agent_settings else False
-        self.action_masking: bool = agent_settings.action_masking if agent_settings else False
 
     def get_action(self, obs: ObsType, timestep: int = 0) -> Tuple[str, Dict]:
         """

src/primaite/game/agent/observations/file_system_observations.py

@@ -23,10 +23,8 @@ class FileObservation(AbstractObservation, identifier="FILE"):
         """Name of the file, used for querying simulation state dictionary."""
         include_num_access: Optional[bool] = None
         """Whether to include the number of accesses to the file in the observation."""
-        file_system_requires_scan: Optional[bool] = None
-        """If True, the file must be scanned to update the health state. Tf False, the true state is always shown."""
 
-    def __init__(self, where: WhereType, include_num_access: bool, file_system_requires_scan: bool) -> None:
+    def __init__(self, where: WhereType, include_num_access: bool) -> None:
         """
         Initialise a file observation instance.
 
@@ -36,13 +34,9 @@ class FileObservation(AbstractObservation, identifier="FILE"):
         :type where: WhereType
         :param include_num_access: Whether to include the number of accesses to the file in the observation.
         :type include_num_access: bool
-        :param file_system_requires_scan: If True, the file must be scanned to update the health state. Tf False,
-            the true state is always shown.
-        :type file_system_requires_scan: bool
         """
         self.where: WhereType = where
         self.include_num_access: bool = include_num_access
-        self.file_system_requires_scan: bool = file_system_requires_scan
 
         self.default_observation: ObsType = {"health_status": 0}
         if self.include_num_access:
@@ -80,11 +74,7 @@ class FileObservation(AbstractObservation, identifier="FILE"):
         file_state = access_from_nested_dict(state, self.where)
         if file_state is NOT_PRESENT_IN_STATE:
             return self.default_observation
-        if self.file_system_requires_scan:
-            health_status = file_state["visible_status"]
-        else:
-            health_status = file_state["health_status"]
-        obs = {"health_status": health_status}
+        obs = {"health_status": file_state["visible_status"]}
         if self.include_num_access:
             obs["num_access"] = self._categorise_num_access(file_state["num_access"])
         return obs
@@ -114,15 +104,8 @@ class FileObservation(AbstractObservation, identifier="FILE"):
         :type parent_where: WhereType, optional
         :return: Constructed file observation instance.
         :rtype: FileObservation
-        :param file_system_requires_scan: If True, the folder must be scanned to update the health state. Tf False,
-            the true state is always shown.
-        :type file_system_requires_scan: bool
         """
-        return cls(
-            where=parent_where + ["files", config.file_name],
-            include_num_access=config.include_num_access,
-            file_system_requires_scan=config.file_system_requires_scan,
-        )
+        return cls(where=parent_where + ["files", config.file_name], include_num_access=config.include_num_access)
 
 
 class FolderObservation(AbstractObservation, identifier="FOLDER"):
@@ -139,16 +122,9 @@ class FolderObservation(AbstractObservation, identifier="FOLDER"):
         """Number of spaces for file observations in this folder."""
         include_num_access: Optional[bool] = None
         """Whether files in this folder should include the number of accesses in their observation."""
-        file_system_requires_scan: Optional[bool] = None
-        """If True, the folder must be scanned to update the health state. Tf False, the true state is always shown."""
 
     def __init__(
-        self,
-        where: WhereType,
-        files: Iterable[FileObservation],
-        num_files: int,
-        include_num_access: bool,
-        file_system_requires_scan: bool,
+        self, where: WhereType, files: Iterable[FileObservation], num_files: int, include_num_access: bool
     ) -> None:
         """
         Initialise a folder observation instance.
@@ -162,23 +138,12 @@ class FolderObservation(AbstractObservation, identifier="FOLDER"):
         :type num_files: int
         :param include_num_access: Whether to include the number of accesses to files in the observation.
         :type include_num_access: bool
-        :param file_system_requires_scan: If True, the folder must be scanned to update the health state. Tf False,
-            the true state is always shown.
-        :type file_system_requires_scan: bool
         """
         self.where: WhereType = where
 
-        self.file_system_requires_scan: bool = file_system_requires_scan
-
         self.files: List[FileObservation] = files
         while len(self.files) < num_files:
-            self.files.append(
-                FileObservation(
-                    where=None,
-                    include_num_access=include_num_access,
-                    file_system_requires_scan=self.file_system_requires_scan,
-                )
-            )
+            self.files.append(FileObservation(where=None, include_num_access=include_num_access))
         while len(self.files) > num_files:
             truncated_file = self.files.pop()
             msg = f"Too many files in folder observation. Truncating file {truncated_file}"
@@ -203,10 +168,7 @@ class FolderObservation(AbstractObservation, identifier="FOLDER"):
         if folder_state is NOT_PRESENT_IN_STATE:
             return self.default_observation
 
-        if self.file_system_requires_scan:
-            health_status = folder_state["visible_status"]
-        else:
-            health_status = folder_state["health_status"]
+        health_status = folder_state["health_status"]
 
         obs = {}
 
@@ -247,13 +209,6 @@ class FolderObservation(AbstractObservation, identifier="FOLDER"):
         # pass down shared/common config items
         for file_config in config.files:
             file_config.include_num_access = config.include_num_access
-            file_config.file_system_requires_scan = config.file_system_requires_scan
 
         files = [FileObservation.from_config(config=f, parent_where=where) for f in config.files]
-        return cls(
-            where=where,
-            files=files,
-            num_files=config.num_files,
-            include_num_access=config.include_num_access,
-            file_system_requires_scan=config.file_system_requires_scan,
-        )
+        return cls(where=where, files=files, num_files=config.num_files, include_num_access=config.include_num_access)

src/primaite/game/agent/observations/firewall_observation.py

@@ -10,7 +10,6 @@ from primaite import getLogger
 from primaite.game.agent.observations.acl_observation import ACLObservation
 from primaite.game.agent.observations.nic_observations import PortObservation
 from primaite.game.agent.observations.observations import AbstractObservation, WhereType
-from primaite.game.agent.utils import access_from_nested_dict, NOT_PRESENT_IN_STATE
 
 _LOGGER = getLogger(__name__)
 
@@ -33,8 +32,6 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
         """List of protocols for encoding ACLs."""
         num_rules: Optional[int] = None
         """Number of rules ACL rules to show."""
-        include_users: Optional[bool] = True
-        """If True, report user session information."""
 
     def __init__(
         self,
@@ -44,7 +41,6 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
         port_list: List[int],
         protocol_list: List[str],
         num_rules: int,
-        include_users: bool,
     ) -> None:
         """
         Initialise a firewall observation instance.
@@ -62,13 +58,9 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
         :type protocol_list: List[str]
         :param num_rules: Number of rules configured in the firewall.
         :type num_rules: int
-        :param include_users: If True, report user session information.
-        :type include_users: bool
         """
         self.where: WhereType = where
-        self.include_users: bool = include_users
-        self.max_users: int = 3
-        """Maximum number of remote sessions observable, excess sessions are truncated."""
+
         self.ports: List[PortObservation] = [
             PortObservation(where=self.where + ["NICs", port_num]) for port_num in (1, 2, 3)
         ]
@@ -150,9 +142,6 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
         :return: Observation containing the status of ports and ACLs for internal, DMZ, and external traffic.
         :rtype: ObsType
         """
-        firewall_state = access_from_nested_dict(state, self.where)
-        if firewall_state is NOT_PRESENT_IN_STATE:
-            return self.default_observation
         obs = {
             "PORTS": {i + 1: p.observe(state) for i, p in enumerate(self.ports)},
             "ACL": {
@@ -170,12 +159,6 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
                 },
             },
         }
-        if self.include_users:
-            sess = firewall_state["services"]["UserSessionManager"]
-            obs["users"] = {
-                "local_login": 1 if sess["current_local_user"] else 0,
-                "remote_sessions": min(self.max_users, len(sess["active_remote_sessions"])),
-            }
         return obs
 
     @property
@@ -235,5 +218,4 @@ class FirewallObservation(AbstractObservation, identifier="FIREWALL"):
             port_list=config.port_list,
             protocol_list=config.protocol_list,
             num_rules=config.num_rules,
-            include_users=config.include_users,
         )

src/primaite/game/agent/observations/host_observations.py

@@ -48,12 +48,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         """A dict containing which traffic types are to be included in the observation."""
         include_num_access: Optional[bool] = None
         """Whether to include the number of accesses to files observations on this host."""
-        file_system_requires_scan: Optional[bool] = None
-        """
-        If True, files and folders must be scanned to update the health state. If False, true state is always shown.
-        """
-        include_users: Optional[bool] = True
-        """If True, report user session information."""
 
     def __init__(
         self,
@@ -70,8 +64,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         include_nmne: bool,
         monitored_traffic: Optional[Dict],
         include_num_access: bool,
-        file_system_requires_scan: bool,
-        include_users: bool,
     ) -> None:
         """
         Initialise a host observation instance.
@@ -103,18 +95,10 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         :type monitored_traffic: Dict
         :param include_num_access: Flag to include the number of accesses to files.
         :type include_num_access: bool
-        :param file_system_requires_scan: If True, the files and folders must be scanned to update the health state.
-            If False, the true state is always shown.
-        :type file_system_requires_scan: bool
-        :param include_users: If True, report user session information.
-        :type include_users: bool
         """
         self.where: WhereType = where
 
         self.include_num_access = include_num_access
-        self.include_users = include_users
-        self.max_users: int = 3
-        """Maximum number of remote sessions observable, excess sessions are truncated."""
 
         # Ensure lists have lengths equal to specified counts by truncating or padding
         self.services: List[ServiceObservation] = services
@@ -136,13 +120,7 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         self.folders: List[FolderObservation] = folders
         while len(self.folders) < num_folders:
             self.folders.append(
-                FolderObservation(
-                    where=None,
-                    files=[],
-                    num_files=num_files,
-                    include_num_access=include_num_access,
-                    file_system_requires_scan=file_system_requires_scan,
-                )
+                FolderObservation(where=None, files=[], num_files=num_files, include_num_access=include_num_access)
             )
         while len(self.folders) > num_folders:
             truncated_folder = self.folders.pop()
@@ -173,8 +151,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         if self.include_num_access:
             self.default_observation["num_file_creations"] = 0
             self.default_observation["num_file_deletions"] = 0
-        if self.include_users:
-            self.default_observation["users"] = {"local_login": 0, "remote_sessions": 0}
 
     def observe(self, state: Dict) -> ObsType:
         """
@@ -202,12 +178,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         if self.include_num_access:
             obs["num_file_creations"] = node_state["file_system"]["num_file_creations"]
             obs["num_file_deletions"] = node_state["file_system"]["num_file_deletions"]
-        if self.include_users:
-            sess = node_state["services"]["UserSessionManager"]
-            obs["users"] = {
-                "local_login": 1 if sess["current_local_user"] else 0,
-                "remote_sessions": min(self.max_users, len(sess["active_remote_sessions"])),
-            }
         return obs
 
     @property
@@ -232,10 +202,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         if self.include_num_access:
             shape["num_file_creations"] = spaces.Discrete(4)
             shape["num_file_deletions"] = spaces.Discrete(4)
-        if self.include_users:
-            shape["users"] = spaces.Dict(
-                {"local_login": spaces.Discrete(2), "remote_sessions": spaces.Discrete(self.max_users + 1)}
-            )
         return spaces.Dict(shape)
 
     @classmethod
@@ -260,7 +226,6 @@ class HostObservation(AbstractObservation, identifier="HOST"):
         for folder_config in config.folders:
             folder_config.include_num_access = config.include_num_access
             folder_config.num_files = config.num_files
-            folder_config.file_system_requires_scan = config.file_system_requires_scan
         for nic_config in config.network_interfaces:
             nic_config.include_nmne = config.include_nmne
 
@@ -292,6 +257,4 @@ class HostObservation(AbstractObservation, identifier="HOST"):
             include_nmne=config.include_nmne,
             monitored_traffic=config.monitored_traffic,
             include_num_access=config.include_num_access,
-            file_system_requires_scan=config.file_system_requires_scan,
-            include_users=config.include_users,
         )

src/primaite/game/agent/observations/node_observations.py

@@ -44,10 +44,6 @@ class NodesObservation(AbstractObservation, identifier="NODES"):
         """A dict containing which traffic types are to be included in the observation."""
         include_num_access: Optional[bool] = None
         """Flag to include the number of accesses."""
-        file_system_requires_scan: bool = True
-        """If True, the folder must be scanned to update the health state. Tf False, the true state is always shown."""
-        include_users: Optional[bool] = True
-        """If True, report user session information."""
         num_ports: Optional[int] = None
         """Number of ports."""
         ip_list: Optional[List[str]] = None
@@ -191,10 +187,6 @@ class NodesObservation(AbstractObservation, identifier="NODES"):
                 host_config.monitored_traffic = config.monitored_traffic
             if host_config.include_num_access is None:
                 host_config.include_num_access = config.include_num_access
-            if host_config.file_system_requires_scan is None:
-                host_config.file_system_requires_scan = config.file_system_requires_scan
-            if host_config.include_users is None:
-                host_config.include_users = config.include_users
 
         for router_config in config.routers:
             if router_config.num_ports is None:
@@ -209,8 +201,6 @@ class NodesObservation(AbstractObservation, identifier="NODES"):
                 router_config.protocol_list = config.protocol_list
             if router_config.num_rules is None:
                 router_config.num_rules = config.num_rules
-            if router_config.include_users is None:
-                router_config.include_users = config.include_users
 
         for firewall_config in config.firewalls:
             if firewall_config.ip_list is None:
@@ -223,8 +213,6 @@ class NodesObservation(AbstractObservation, identifier="NODES"):
                 firewall_config.protocol_list = config.protocol_list
             if firewall_config.num_rules is None:
                 firewall_config.num_rules = config.num_rules
-            if firewall_config.include_users is None:
-                firewall_config.include_users = config.include_users
 
         hosts = [HostObservation.from_config(config=c, parent_where=where) for c in config.hosts]
         routers = [RouterObservation.from_config(config=c, parent_where=where) for c in config.routers]

src/primaite/game/agent/observations/router_observation.py

@@ -39,8 +39,6 @@ class RouterObservation(AbstractObservation, identifier="ROUTER"):
         """List of protocols for encoding ACLs."""
         num_rules: Optional[int] = None
         """Number of rules ACL rules to show."""
-        include_users: Optional[bool] = True
-        """If True, report user session information."""
 
     def __init__(
         self,
@@ -48,7 +46,6 @@ class RouterObservation(AbstractObservation, identifier="ROUTER"):
         ports: List[PortObservation],
         num_ports: int,
         acl: ACLObservation,
-        include_users: bool,
     ) -> None:
         """
         Initialise a router observation instance.
@@ -62,16 +59,12 @@ class RouterObservation(AbstractObservation, identifier="ROUTER"):
         :type num_ports: int
         :param acl: ACL observation representing the access control list of the router.
         :type acl: ACLObservation
-        :param include_users: If True, report user session information.
-        :type include_users: bool
         """
         self.where: WhereType = where
         self.ports: List[PortObservation] = ports
         self.acl: ACLObservation = acl
         self.num_ports: int = num_ports
-        self.include_users: bool = include_users
-        self.max_users: int = 3
-        """Maximum number of remote sessions observable, excess sessions are truncated."""
+
         while len(self.ports) < num_ports:
             self.ports.append(PortObservation(where=None))
         while len(self.ports) > num_ports:
@@ -102,12 +95,6 @@ class RouterObservation(AbstractObservation, identifier="ROUTER"):
         obs["ACL"] = self.acl.observe(state)
         if self.ports:
             obs["PORTS"] = {i + 1: p.observe(state) for i, p in enumerate(self.ports)}
-        if self.include_users:
-            sess = router_state["services"]["UserSessionManager"]
-            obs["users"] = {
-                "local_login": 1 if sess["current_local_user"] else 0,
-                "remote_sessions": min(self.max_users, len(sess["active_remote_sessions"])),
-            }
         return obs
 
     @property
@@ -156,4 +143,4 @@ class RouterObservation(AbstractObservation, identifier="ROUTER"):
 
         ports = [PortObservation.from_config(config=c, parent_where=where) for c in config.ports]
         acl = ACLObservation.from_config(config=config.acl, parent_where=where)
-        return cls(where=where, ports=ports, num_ports=config.num_ports, acl=acl, include_users=config.include_users)
+        return cls(where=where, ports=ports, num_ports=config.num_ports, acl=acl)

src/primaite/game/agent/rewards.py

@@ -8,7 +8,6 @@ weighed sum of the components.
 
 The reward function is typically specified using a config yaml file or a config dictionary. The following example shows
 the structure:
-
 ```yaml
     reward_function:
         reward_components:
@@ -47,15 +46,7 @@ class AbstractReward:
 
     @abstractmethod
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
-        """Calculate the reward for the current state.
-
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
-        """
+        """Calculate the reward for the current state."""
         return 0.0
 
     @classmethod
@@ -75,15 +66,7 @@ class DummyReward(AbstractReward):
     """Dummy reward function component which always returns 0."""
 
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
-        """Calculate the reward for the current state.
-
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
-        """
+        """Calculate the reward for the current state."""
         return 0.0
 
     @classmethod
@@ -125,12 +108,8 @@ class DatabaseFileIntegrity(AbstractReward):
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
         """Calculate the reward for the current state.
 
-        :param state: Current simulation state
+        :param state: The current state of the simulation.
         :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
         """
         database_file_state = access_from_nested_dict(state, self.location_in_state)
         if database_file_state is NOT_PRESENT_IN_STATE:
@@ -171,52 +150,33 @@ class DatabaseFileIntegrity(AbstractReward):
 class WebServer404Penalty(AbstractReward):
     """Reward function component which penalises the agent when the web server returns a 404 error."""
 
-    def __init__(self, node_hostname: str, service_name: str, sticky: bool = True) -> None:
+    def __init__(self, node_hostname: str, service_name: str) -> None:
         """Initialise the reward component.
 
         :param node_hostname: Hostname of the node which contains the web server service.
         :type node_hostname: str
         :param service_name: Name of the web server service.
         :type service_name: str
-        :param sticky: If True, calculate the reward based on the most recent response status. If False, only calculate
-            the reward if there were any responses this timestep.
-        :type sticky: bool
         """
-        self.sticky: bool = sticky
-        self.reward: float = 0.0
-        """Reward value calculated last time any responses were seen. Used for persisting sticky rewards."""
         self.location_in_state = ["network", "nodes", node_hostname, "services", service_name]
 
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
         """Calculate the reward for the current state.
 
-        :param state: Current simulation state
+        :param state: The current state of the simulation.
         :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
         """
         web_service_state = access_from_nested_dict(state, self.location_in_state)
-
-        # if webserver is no longer installed on the node, return 0
         if web_service_state is NOT_PRESENT_IN_STATE:
             return 0.0
-
-        codes = web_service_state.get("response_codes_this_timestep")
-        if codes:
-
-            def status2rew(status: int) -> int:
-                """Map status codes to reward values."""
-                return 1.0 if status == 200 else -1.0 if status == 404 else 0.0
-
-            self.reward = sum(map(status2rew, codes)) / len(codes)  # convert form HTTP codes to rewards and average
-        elif not self.sticky:  # there are no codes, but reward is not sticky, set reward to 0
-            self.reward = 0.0
-        else:  # skip calculating if sticky and no new codes. instead, reuse last step's value
-            pass
-
-        return self.reward
+        most_recent_return_code = web_service_state["last_response_status_code"]
+        # TODO: reward needs to use the current web state. Observation should return web state at the time of last scan.
+        if most_recent_return_code == 200:
+            return 1.0
+        elif most_recent_return_code == 404:
+            return -1.0
+        else:
+            return 0.0
 
     @classmethod
     def from_config(cls, config: Dict) -> "WebServer404Penalty":
@@ -236,29 +196,23 @@ class WebServer404Penalty(AbstractReward):
             )
             _LOGGER.warning(msg)
             raise ValueError(msg)
-        sticky = config.get("sticky", True)
 
-        return cls(node_hostname=node_hostname, service_name=service_name, sticky=sticky)
+        return cls(node_hostname=node_hostname, service_name=service_name)
 
 
 class WebpageUnavailablePenalty(AbstractReward):
     """Penalises the agent when the web browser fails to fetch a webpage."""
 
-    def __init__(self, node_hostname: str, sticky: bool = True) -> None:
+    def __init__(self, node_hostname: str) -> None:
         """
         Initialise the reward component.
 
         :param node_hostname: Hostname of the node which has the web browser.
         :type node_hostname: str
-        :param sticky: If True, calculate the reward based on the most recent response status. If False, only calculate
-            the reward if there were any responses this timestep.
-        :type sticky: bool
         """
         self._node: str = node_hostname
         self.location_in_state: List[str] = ["network", "nodes", node_hostname, "applications", "WebBrowser"]
-        self.sticky: bool = sticky
-        self.reward: float = 0.0
-        """Reward value calculated last time any responses were seen. Used for persisting sticky rewards."""
+        self._last_request_failed: bool = False
 
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
         """
@@ -267,50 +221,32 @@ class WebpageUnavailablePenalty(AbstractReward):
         When the green agent requests to execute the browser application, and that request fails, this reward
         component will keep track of that information. In that case, it doesn't matter whether the last webpage
         had a 200 status code, because there has been an unsuccessful request since.
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
         """
+        if last_action_response.request == ["network", "node", self._node, "application", "WebBrowser", "execute"]:
+            self._last_request_failed = last_action_response.response.status != "success"
+
+        # if agent couldn't even get as far as sending the request (because for example the node was off), then
+        # apply a penalty
+        if self._last_request_failed:
+            return -1.0
+
+        # If the last request did actually go through, then check if the webpage also loaded
         web_browser_state = access_from_nested_dict(state, self.location_in_state)
-
-        if web_browser_state is NOT_PRESENT_IN_STATE:
-            self.reward = 0.0
-
-        # check if the most recent action was to request the webpage
-        request_attempted = last_action_response.request == [
-            "network",
-            "node",
-            self._node,
-            "application",
-            "WebBrowser",
-            "execute",
-        ]
-
-        # skip calculating if sticky and no new codes, reusing last step value
-        if not request_attempted and self.sticky:
-            return self.reward
-
-        if last_action_response.response.status != "success":
-            self.reward = -1.0
-        elif web_browser_state is NOT_PRESENT_IN_STATE or not web_browser_state["history"]:
+        if web_browser_state is NOT_PRESENT_IN_STATE or "history" not in web_browser_state:
             _LOGGER.debug(
                 "Web browser reward could not be calculated because the web browser history on node",
                 f"{self._node} was not reported in the simulation state. Returning 0.0",
             )
-            self.reward = 0.0
-        else:
-            outcome = web_browser_state["history"][-1]["outcome"]
-            if outcome == "PENDING":
-                self.reward = 0.0  # 0 if a request was attempted but not yet resolved
-            elif outcome == 200:
-                self.reward = 1.0  # 1 for successful request
-            else:  # includes failure codes and SERVER_UNREACHABLE
-                self.reward = -1.0  # -1 for failure
-
-        return self.reward
+            return 0.0  # 0 if the web browser cannot be found
+        if not web_browser_state["history"]:
+            return 0.0  # 0 if no requests have been attempted yet
+        outcome = web_browser_state["history"][-1]["outcome"]
+        if outcome == "PENDING":
+            return 0.0  # 0 if a request was attempted but not yet resolved
+        elif outcome == 200:
+            return 1.0  # 1 for successful request
+        else:  # includes failure codes and SERVER_UNREACHABLE
+            return -1.0  # -1 for failure
 
     @classmethod
     def from_config(cls, config: dict) -> AbstractReward:
@@ -321,28 +257,22 @@ class WebpageUnavailablePenalty(AbstractReward):
         :type config: Dict
         """
         node_hostname = config.get("node_hostname")
-        sticky = config.get("sticky", True)
-        return cls(node_hostname=node_hostname, sticky=sticky)
+        return cls(node_hostname=node_hostname)
 
 
 class GreenAdminDatabaseUnreachablePenalty(AbstractReward):
     """Penalises the agent when the green db clients fail to connect to the database."""
 
-    def __init__(self, node_hostname: str, sticky: bool = True) -> None:
+    def __init__(self, node_hostname: str) -> None:
         """
         Initialise the reward component.
 
         :param node_hostname: Hostname of the node where the database client sits.
         :type node_hostname: str
-        :param sticky: If True, calculate the reward based on the most recent response status. If False, only calculate
-            the reward if there were any responses this timestep.
-        :type sticky: bool
         """
         self._node: str = node_hostname
         self.location_in_state: List[str] = ["network", "nodes", node_hostname, "applications", "DatabaseClient"]
-        self.sticky: bool = sticky
-        self.reward: float = 0.0
-        """Reward value calculated last time any responses were seen. Used for persisting sticky rewards."""
+        self._last_request_failed: bool = False
 
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
         """
@@ -352,33 +282,26 @@ class GreenAdminDatabaseUnreachablePenalty(AbstractReward):
         component will keep track of that information. In that case, it doesn't matter whether the last successful
         request returned was able to connect to the database server, because there has been an unsuccessful request
         since.
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
         """
-        request_attempted = last_action_response.request == [
-            "network",
-            "node",
-            self._node,
-            "application",
-            "DatabaseClient",
-            "execute",
-        ]
+        if last_action_response.request == ["network", "node", self._node, "application", "DatabaseClient", "execute"]:
+            self._last_request_failed = last_action_response.response.status != "success"
 
-        if request_attempted:  # if agent makes request, always recalculate fresh value
-            last_action_response.reward_info = {"connection_attempt_status": last_action_response.response.status}
-            self.reward = 1.0 if last_action_response.response.status == "success" else -1.0
-        elif not self.sticky:  # if no new request and not sticky, set reward to 0
-            last_action_response.reward_info = {"connection_attempt_status": "n/a"}
-            self.reward = 0.0
-        else:  # if no new request and sticky, reuse reward value from last step
-            last_action_response.reward_info = {"connection_attempt_status": "n/a"}
-            pass
+        # if agent couldn't even get as far as sending the request (because for example the node was off), then
+        # apply a penalty
+        if self._last_request_failed:
+            return -1.0
 
-        return self.reward
+        # If the last request was actually sent, then check if the connection was established.
+        db_state = access_from_nested_dict(state, self.location_in_state)
+        if db_state is NOT_PRESENT_IN_STATE or "last_connection_successful" not in db_state:
+            _LOGGER.debug(f"Can't calculate reward for {self.__class__.__name__}")
+            return 0.0
+        last_connection_successful = db_state["last_connection_successful"]
+        if last_connection_successful is False:
+            return -1.0
+        elif last_connection_successful is True:
+            return 1.0
+        return 0.0
 
     @classmethod
     def from_config(cls, config: Dict) -> AbstractReward:
@@ -389,8 +312,7 @@ class GreenAdminDatabaseUnreachablePenalty(AbstractReward):
         :type config: Dict
         """
         node_hostname = config.get("node_hostname")
-        sticky = config.get("sticky", True)
-        return cls(node_hostname=node_hostname, sticky=sticky)
+        return cls(node_hostname=node_hostname)
 
 
 class SharedReward(AbstractReward):
@@ -423,15 +345,7 @@ class SharedReward(AbstractReward):
         """Method that retrieves an agent's current reward given the agent's name."""
 
     def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
-        """Simply access the other agent's reward and return it.
-
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
-        """
+        """Simply access the other agent's reward and return it."""
         return self.callback(self.agent_name)
 
     @classmethod
@@ -446,46 +360,6 @@ class SharedReward(AbstractReward):
         return cls(agent_name=agent_name)
 
 
-class ActionPenalty(AbstractReward):
-    """Apply a negative reward when taking any action except DONOTHING."""
-
-    def __init__(self, action_penalty: float, do_nothing_penalty: float) -> None:
-        """
-        Initialise the reward.
-
-        Reward or penalise agents for doing nothing or taking actions.
-
-        :param action_penalty: Reward to give agents for taking any action except DONOTHING
-        :type action_penalty: float
-        :param do_nothing_penalty: Reward to give agent for taking the DONOTHING action
-        :type do_nothing_penalty: float
-        """
-        self.action_penalty = action_penalty
-        self.do_nothing_penalty = do_nothing_penalty
-
-    def calculate(self, state: Dict, last_action_response: "AgentHistoryItem") -> float:
-        """Calculate the penalty to be applied.
-
-        :param state: Current simulation state
-        :type state: Dict
-        :param last_action_response: Current agent history state
-        :type last_action_response: AgentHistoryItem state
-        :return: Reward value
-        :rtype: float
-        """
-        if last_action_response.action == "DONOTHING":
-            return self.do_nothing_penalty
-        else:
-            return self.action_penalty
-
-    @classmethod
-    def from_config(cls, config: Dict) -> "ActionPenalty":
-        """Build the ActionPenalty object from config."""
-        action_penalty = config.get("action_penalty", -1.0)
-        do_nothing_penalty = config.get("do_nothing_penalty", 0.0)
-        return cls(action_penalty=action_penalty, do_nothing_penalty=do_nothing_penalty)
-
-
 class RewardFunction:
     """Manages the reward function for the agent."""
 
@@ -496,7 +370,6 @@ class RewardFunction:
         "WEBPAGE_UNAVAILABLE_PENALTY": WebpageUnavailablePenalty,
         "GREEN_ADMIN_DATABASE_UNREACHABLE_PENALTY": GreenAdminDatabaseUnreachablePenalty,
         "SHARED_REWARD": SharedReward,
-        "ACTION_PENALTY": ActionPenalty,
     }
     """List of reward class identifiers."""
 
@@ -529,7 +402,6 @@ class RewardFunction:
             weight = comp_and_weight[1]
             total += weight * comp.calculate(state=state, last_action_response=last_action_response)
         self.current_reward = total
-
         return self.current_reward
 
     @classmethod

src/primaite/game/agent/scripted_agents/data_manipulation_bot.py

@@ -38,11 +38,10 @@ class DataManipulationAgent(AbstractScriptedAgent):
         :rtype: Tuple[str, Dict]
         """
         if timestep < self.next_execution_timestep:
-            self.logger.debug(msg="Performing do NOTHING")
             return "DONOTHING", {}
 
         self._set_next_execution_timestep(timestep + self.agent_settings.start_settings.frequency)
-        self.logger.info(msg="Performing a data manipulation attack!")
+
         return "NODE_APPLICATION_EXECUTE", {"node_id": self.starting_node_idx, "application_id": 0}
 
     def setup_agent(self) -> None:
@@ -55,4 +54,3 @@ class DataManipulationAgent(AbstractScriptedAgent):
         # we are assuming that every node in the node manager has a data manipulation application at idx 0
         num_nodes = len(self.action_manager.node_names)
         self.starting_node_idx = random.randint(0, num_nodes - 1)
-        self.logger.debug(msg=f"Select Start Node ID: {self.starting_node_idx}")