diff --git a/.azure/azure-ci-build-pipeline.yaml b/.azure/azure-ci-build-pipeline.yaml index b6f24777..4353b082 100644 --- a/.azure/azure-ci-build-pipeline.yaml +++ b/.azure/azure-ci-build-pipeline.yaml @@ -15,37 +15,37 @@ parameters: type: object default: - job_name: 'UbuntuPython39' - py: '3.9' + py: 'v3.9' img: 'ubuntu-latest' every_time: false publish_coverage: false - job_name: 'UbuntuPython310' - py: '3.10' + py: 'v3.10' img: 'ubuntu-latest' every_time: true publish_coverage: true - job_name: 'UbuntuPython311' - py: '3.11' + py: 'v3.11' img: 'ubuntu-latest' every_time: false publish_coverage: false - job_name: 'WindowsPython39' - py: '3.9' + py: 'v3.9' img: 'windows-latest' every_time: false publish_coverage: false - job_name: 'WindowsPython311' - py: '3.11' + py: 'v3.11' img: 'windows-latest' every_time: false publish_coverage: false - job_name: 'MacOSPython39' - py: '3.9' + py: 'v3.9' img: 'macOS-latest' every_time: false publish_coverage: false - job_name: 'MacOSPython311' - py: '3.11' + py: 'v3.11' img: 'macOS-latest' every_time: false publish_coverage: false diff --git a/.github/workflows/build-sphinx.yml b/.github/workflows/build-sphinx.yml index 4bfa4f4e..da20fbd3 100644 --- a/.github/workflows/build-sphinx.yml +++ b/.github/workflows/build-sphinx.yml @@ -1,8 +1,8 @@ name: build-sphinx-to-github-pages env: - GITHUB_ACTOR: {todo:fill in URL} - GITHUB_REPOSITORY: {todo:fill in URL}/PrimAITE + GITHUB_ACTOR: Autonomous-Resilient-Cyber-Defence + GITHUB_REPOSITORY: Autonomous-Resilient-Cyber-Defence/PrimAITE GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN}} on: diff --git a/CHANGELOG.md b/CHANGELOG.md index fbcfa707..7c64365e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,7 +39,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Updated tests that don't use YAMLs to still use the new action and agent schemas - Nodes now use a config schema and are extensible, allowing for plugin support. - Node tests have been updated to use the new node config schemas when not using YAML files. -- Documentation has been updated to include details of extensability with PrimAITE. +- Documentation has been updated to include details of extensibility with PrimAITE. - Software is created in the GOOD health state instead of UNUSED. - Standardised naming convention for YAML config files using kebab-case. This naming convention is used for configuring software, observations, actions and node types. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dc10edbb..815400e2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,17 +4,17 @@ ### **Did you find a bug?** -* **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/{todo:fill in URL}/PrimAITE/issues). -* If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/{todo:fill in URL}/PrimAITE/issues/new?assignees=&labels=bug&projects=&template=bug_report.md&title=%5BBUG%5D+-+%3Cbug+title+goes+here%3E). Be sure to follow our bug report template with the headers **Describe the bug**, **To Reproduce**, **Expected behaviour**, **Screenshots/Outputs**, **Environment**, and **Additional context** +* **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues). +* If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=bug&projects=&template=bug_report.md&title=%5BBUG%5D+-+%3Cbug+title+goes+here%3E). Be sure to follow our bug report template with the headers **Describe the bug**, **To Reproduce**, **Expected behaviour**, **Screenshots/Outputs**, **Environment**, and **Additional context** ### **Do you have a solution to fix the bug?** -* [Fork the repository](https://github.com/{todo:fill in URL}/PrimAITE/fork). +* [Fork the repository](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/fork). * Install the pre-commit hook with `pre-commit install`. * Implement the bug fix. -* Commit the bug fix to the dev branch on your fork. If the bug has an open issue under [Issues](https://github.com/{todo:fill in URL}/PrimAITE/issues), reference the issue in the commit message (e.g. #1 references issue 1). -* Submit a pull request from your dev branch to the {todo:fill in URL}/PrimAITE dev branch. Again, if the bug has an open issue under [Issues](https://github.com/{todo:fill in URL}/PrimAITE/issues), reference the issue in the pull request description. +* Commit the bug fix to the dev branch on your fork. If the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the commit message (e.g. #1 references issue 1). +* Submit a pull request from your dev branch to the Autonomous-Resilient-Cyber-Defence/PrimAITE dev branch. Again, if the bug has an open issue under [Issues](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues), reference the issue in the pull request description. ### **Did you fix whitespace, format code, or make a purely cosmetic patch?** @@ -22,7 +22,7 @@ Changes that are cosmetic in nature and do not add anything substantial to the s ### **Do you intend to add a new feature or change an existing one?** -* Submit a [feature request issue](https://github.com/{todo:fill in URL}/PrimAITE/issues/new?assignees=&labels=feature_request&projects=&template=feature_request.md&title=%5BREQUEST%5D+-+%3Crequest+title+goes+here%3E). +* Submit a [feature request issue](https://github.com/Autonomous-Resilient-Cyber-Defence/PrimAITE/issues/new?assignees=&labels=feature_request&projects=&template=feature_request.md&title=%5BREQUEST%5D+-+%3Crequest+title+goes+here%3E). * Know how to implement the new feature or change? Follow the same steps in the bug fix section above to fork, build, document, test, commit, and submit a pull request. ### **Do you have questions about the source code?** diff --git a/LICENSE b/LICENSE index 9575f430..8038f3d0 100644 --- a/LICENSE +++ b/LICENSE @@ -4,24 +4,21 @@ MIT License Conditions These MIT License conditions confirm the provision of the following artefacts as MIT License by Defence Science and Technology of this software and associated documentation files (the "Software"), to deal -request to the QQ or FNC mailbox): +in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is -- Use Case Release Packs furnished to do so, subject to the following conditions: -Suppliers are required to read and confirm acceptance of the {todo:fill in URL} Foundry SyOPs (https://github.com/{todo:fill in URL}/foundry-syops) before being admitted access to material hosted on the {todo:fill in URL} Foundry GitHub site. +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -The material is supplied in confidence to QQ / FNC and their subcontractors under SERAPIS, and is issued to inform only those +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - - -of DSTL. The material must be stored and protected appropriately. All material must be destroyed at the end of the task. - diff --git a/README.md b/README.md index c8f644be..a5375d66 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ To make your own changes to PrimAITE, perform the install from source (developer #### 1. Clone the PrimAITE repository ``` unix -git clone git@github.com:{todo:fill in URL}/PrimAITE.git +git clone git@github.com:Autonomous-Resilient-Cyber-Defence/PrimAITE.git ``` #### 2. CD into the repo directory diff --git a/docs/Makefile b/docs/Makefile index d0f9af01..bc101a07 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -8,16 +8,18 @@ SOURCEDIR = . BUILDDIR = _build AUTOSUMMARY="source/_autosummary" +NOTEBOOKS="source/notebooks/notebooks" # Remove command is different depending on OS ifdef OS - RM = IF exist $(AUTOSUMMARY) ( RMDIR $(AUTOSUMMARY) /s /q ) + RM = IF exist $(AUTOSUMMARY) (RMDIR $(AUTOSUMMARY) /s /q) & IF exist $(NOTEBOOKS) (RMDIR $(NOTEBOOKS) /s /q) else ifeq ($(shell uname), Linux) - RM = rm -rf $(AUTOSUMMARY) + RM = rm -rf $(AUTOSUMMARY) $(NOTEBOOKS) endif endif + # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/_templates/custom-class-template.rst b/docs/_templates/custom-class-template.rst index 71e992bc..c9ad7420 100644 --- a/docs/_templates/custom-class-template.rst +++ b/docs/_templates/custom-class-template.rst @@ -12,7 +12,8 @@ .. autoclass:: {{ objname }} :members: :show-inheritance: - :inherited-members: + :inherited-members: BaseModel + :exclude-members: model_computed_fields, model_config, model_fields :special-members: __init__, __call__, __add__, __mul__ {% block methods %} @@ -22,7 +23,14 @@ .. autosummary:: :nosignatures: {% for item in methods %} - {%- if not item.startswith('_') %} + {%- if not item.startswith('_') and item not in [ + 'construct', 'copy', 'dict', 'from_orm', 'json', 'model_construct', + 'model_copy', 'model_dump', 'model_dump_json', 'model_json_schema', + 'model_parametrized_name', 'model_post_init', 'model_rebuild', '', + 'model_validate', 'model_validate_json', 'model_validate_strings', + 'parse_file', 'parse_obj', 'parse_raw', 'schema', 'schema_json', + 'update_forward_refs', 'validate', + ] %} ~{{ name }}.{{ item }} {%- endif -%} {%- endfor %} @@ -35,7 +43,12 @@ .. autosummary:: {% for item in attributes %} + {%- if not item.startswith('_') and item not in [ + 'model_computed_fields', 'model_config', 'model_extra', 'model_fields', + 'model_fields_set', + ] %} ~{{ name }}.{{ item }} + {%- endif -%} {%- endfor %} {% endif %} {% endblock %} diff --git a/docs/build-sphinx-docs-to-github-pages.sh b/docs/build-sphinx-docs-to-github-pages.sh index a180f168..dc6394cb 100644 --- a/docs/build-sphinx-docs-to-github-pages.sh +++ b/docs/build-sphinx-docs-to-github-pages.sh @@ -43,7 +43,7 @@ touch .nojekyll # Add README cat > README.md <`_ is used as the basis for AI blue agent interaction with the PrimAITE environment -* `Networkx `_ is used as the underlying data structure used for the PrimAITE environment -* `Stable Baselines 3 `_ is used as a default source of RL algorithms (although PrimAITE is not limited to SB3 agents) -* `Ray RLlib `_ is used as an additional source of RL algorithms -* `Typer `_ is used for building CLIs (Command Line Interface applications) +* `Pydantic `_ is used for data validation +* `Platformdirs `_ is used for storing user data and configuration correctly between platforms +* `Typer `_ is used for the Command Line Interface * `Jupyterlab `_ is used as an extensible environment for interactive and reproducible computing, based on the Jupyter Notebook Architecture -* `Platformdirs `_ is used for finding the right location to store user data and configuration but varies per platform * `Plotly `_ is used for building high level charts +* `Stable Baselines 3 `_ is used for ensuring compatibility with RL libraries +* `Ray RLlib `_ is also used for ensuring compatibility with RL libraries Getting Started with PrimAITE ----------------------------- Head over to the :ref:`getting-started` page to install and setup PrimAITE! - -.. - Architecture - Nodes and Links - ****************************** - **Nodes** - An inheritance model has been adopted in order to model nodes. All nodes have the following base attributes (Class: Node): - * ID - * Name - * Type (e.g. computer, switch, RTU - enumeration) - * Priority (P1, P2, P3, P4 or P5 - enumeration) - * Hardware State (ON, OFF, RESETTING, SHUTTING_DOWN, BOOTING - enumeration) - Active Nodes also have the following attributes (Class: Active Node): - * IP Address - * Software State (GOOD, FIXING, COMPROMISED - enumeration) - * File System State (GOOD, CORRUPT, DESTROYED, REPAIRING, RESTORING - enumeration) - Service Nodes also have the following attributes (Class: Service Node): - * List of Services (where service is composed of service name and port). There is no theoretical limit on the number of services that can be modelled. Services and protocols are currently intrinsically linked (i.e. a service is an application on a node transmitting traffic of this protocol type) - * Service state (GOOD, FIXING, COMPROMISED, OVERWHELMED - enumeration) - Passive Nodes are currently not used (but may be employed for non IP-based components such as machinery actuators in future releases). - **Links** - Links are modelled both as network edges (networkx) and as Python classes, in order to extend their functionality. Links include the following attributes: - * ID - * Name - * Bandwidth (bits/s) - * Source node ID - * Destination node ID - * Protocol list (containing the loading of protocols currently running on the link) - When the simulation runs, IERs are applied to the links in order to model traffic loading, individually assigned to each protocol. This allows green (background) and red agent behaviour to be modelled, and defensive agents to identify suspicious traffic patterns at a protocol / traffic loading level of fidelity. - Information Exchange Requirements (IERs) - **************************************** - PrimAITE adopts the concept of Information Exchange Requirements (IERs) to model both green agent (background) and red agent (adversary) behaviour. IERs are used to initiate modelling of traffic loading on the network, and have the following attributes: - * ID - * Start step (i.e. which step in the training episode should the IER start) - * End step (i.e. which step in the training episode should the IER end) - * Source node ID - * Destination node ID - * Load (bits/s) - * Protocol - * Port - * Running status (i.e. on / off) - The application of green agent IERs between a source and destination follows a number of rules. Specifically: - 1. Does the current simulation time step fall between IER start and end step - 2. Is the source node operational (both physically and at an O/S level), and is the service (protocol / port) associated with the IER (a) present on this node, and (b) in an operational state (i.e. not FIXING) - 3. Is the destination node operational (both physically and at an O/S level), and is the service (protocol / port) associated with the IER (a) present on this node, and (b) in an operational state (i.e. not FIXING) - 4. Are there any Access Control List rules in place that prevent the application of this IER - 5. Are all switches in the (OSPF) path between source and destination operational (both physically and at an O/S level) - For red agent IERs, the application of IERs between a source and destination follows a number of subtly different rules. Specifically: - 1. Does the current simulation time step fall between IER start and end step - 2. Is the source node operational, and is the service (protocol / port) associated with the IER (a) present on that node and (b) already in a compromised state - 3. Is the destination node operational, and is the service (protocol / port) associated with the IER present on that node - 4. Are there any Access Control List rules in place that prevent the application of this IER - 5. Are all switches in the (OSPF) path between source and destination operational (both physically and at an O/S level) - Assuming the rules pass, the IER is applied to all relevant links (based on use of OSPF) between source and destination. - Node Pattern-of-Life - ******************** - Every node can be impacted (i.e. have a status change applied to it) by either green agent pattern-of-life or red agent pattern-of-life. This is distinct from IERs, and allows for attacks (and defence) to be modelled purely within the confines of a node. - The status changes that can be made to a node are as follows: - * All Nodes: - * Hardware State: - * ON - * OFF - * RESETTING - when a status of resetting is entered, the node will automatically exit this state after a number of steps (as defined by the nodeResetDuration configuration item) after which it returns to an ON state - * BOOTING - * SHUTTING_DOWN - * Active Nodes and Service Nodes: - * Software State: - * GOOD - * FIXING - when a status of FIXING is entered, the node will automatically exit this state after a number of steps (as defined by the osFIXINGDuration configuration item) after which it returns to a GOOD state - * COMPROMISED - * File System State: - * GOOD - * CORRUPT (can be resolved by repair or restore) - * DESTROYED (can be resolved by restore only) - * REPAIRING - when a status of repairing is entered, the node will automatically exit this state after a number of steps (as defined by the fileSystemRepairingLimit configuration item) after which it returns to a GOOD state - * RESTORING - when a status of repairing is entered, the node will automatically exit this state after a number of steps (as defined by the fileSystemRestoringLimit configuration item) after which it returns to a GOOD state - * Service Nodes only: - * Service State (for any associated service): - * GOOD - * FIXING - when a status of FIXING is entered, the service will automatically exit this state after a number of steps (as defined by the serviceFIXINGDuration configuration item) after which it returns to a GOOD state - * COMPROMISED - * OVERWHELMED - Red agent pattern-of-life has an additional feature not found in the green pattern-of-life. This is the ability to influence the state of the attributes of a node via a number of different conditions: - * DIRECT: - The pattern-of-life described by the configuration file item will be applied regardless of any other conditions in the network. This is particularly useful for direct red agent entry into the network. - * IER: - The pattern-of-life described by the configuration file item will be applied to the service on the node, only if there is an IER of the same protocol / service type incoming at the specified timestep. - * SERVICE: - The pattern-of-life described by the configuration file item will be applied to the node based on the state of a service. The service can either be on the same node, or a different node within the network. - Access Control List modelling - ***************************** - An Access Control List (ACL) is modelled to provide the means to manage traffic flows in the system. This will allow defensive agents the means to turn on / off rules, or potentially create new rules, to counter an attack. - The ACL follows a standard network firewall format. For example: - .. list-table:: ACL example - :widths: 25 25 25 25 25 - :header-rows: 1 - * - Permission - - Source IP - - Dest IP - - Protocol - - Port - * - DENY - - 192.168.1.2 - - 192.168.1.3 - - HTTPS - - 443 - * - ALLOW - - 192.168.1.4 - - ANY - - SMTP - - 25 - * - DENY - - ANY - - 192.168.1.5 - - ANY - - ANY - All ACL rules are considered when applying an IER. Logic follows the order of rules, so a DENY or PERMIT for the same parameters will override an earlier entry. - Observation Spaces - ****************** - The observation space provides the blue agent with information about the current status of nodes and links. - PrimAITE builds on top of Gymnasium Spaces to create an observation space that is easily configurable for users. It's made up of components which are managed by the :py:class:`primaite.environment.observations.ObservationsHandler`. Each training scenario can define its own observation space, and the user can choose which information to inlude, and how it should be formatted. - NodeLinkTable component - ----------------------- - For example, the :py:class:`primaite.environment.observations.NodeLinkTable` component represents the status of nodes and links as a ``gym.spaces.Box`` with an example format shown below: - An example observation space is provided below: - .. list-table:: Observation Space example - :widths: 25 25 25 25 25 25 25 - :header-rows: 1 - * - - - ID - - Hardware State - - Software State - - File System State - - Service / Protocol A - - Service / Protocol B - * - Node A - - 1 - - 1 - - 1 - - 1 - - 1 - - 1 - * - Node B - - 2 - - 1 - - 3 - - 1 - - 1 - - 1 - * - Node C - - 3 - - 2 - - 1 - - 1 - - 3 - - 2 - * - Link 1 - - 5 - - 0 - - 0 - - 0 - - 0 - - 10000 - * - Link 2 - - 6 - - 0 - - 0 - - 0 - - 0 - - 10000 - * - Link 3 - - 7 - - 0 - - 0 - - 0 - - 5000 - - 0 - For the nodes, the following values are represented: - .. code-block:: - [ - ID - Hardware State (1=ON, 2=OFF, 3=RESETTING, 4=SHUTTING_DOWN, 5=BOOTING) - Operating System State (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED) - File System State (0=none, 1=GOOD, 2=CORRUPT, 3=DESTROYED, 4=REPAIRING, 5=RESTORING) - Service1/Protocol1 state (0=none, 1=GOOD, 2=FIXING, 3=COMPROMISED) - Service2/Protocol2 state (0=none, 1=GOOD, 2=FIXING, 3=COMPROMISED) - ] - (Note that each service available in the network is provided as a column, although not all nodes may utilise all services) - For the links, the following statuses are represented: - .. code-block:: - [ - ID - Hardware State (0=not applicable) - Operating System State (0=not applicable) - File System State (0=not applicable) - Service1/Protocol1 state (Traffic load from this protocol on this link) - Service2/Protocol2 state (Traffic load from this protocol on this link) - ] - NodeStatus component - ---------------------- - This is a MultiDiscrete observation space that can be though of as a one-dimensional vector of discrete states. - The example above would have the following structure: - .. code-block:: - [ - node1_info - node2_info - node3_info - ] - Each ``node_info`` contains the following: - .. code-block:: - [ - hardware_state (0=none, 1=ON, 2=OFF, 3=RESETTING, 4=SHUTTING_DOWN, 5=BOOTING) - software_state (0=none, 1=GOOD, 2=PATCHING, 3=COMPROMISED) - file_system_state (0=none, 1=GOOD, 2=CORRUPT, 3=DESTROYED, 4=REPAIRING, 5=RESTORING) - service1_state (0=none, 1=GOOD, 2=FIXING, 3=COMPROMISED) - service2_state (0=none, 1=GOOD, 2=FIXING, 3=COMPROMISED) - ] - In a network with three nodes and two services, the full observation space would have 15 elements. It can be written with ``gym`` notation to indicate the number of discrete options for each of the elements of the observation space. For example: - .. code-block:: - gym.spaces.MultiDiscrete([4,5,6,4,4,4,5,6,4,4,4,5,6,4,4]) - .. note:: - NodeStatus observation component provides information only about nodes. Links are not considered. - LinkTrafficLevels - ----------------- - This component is a MultiDiscrete space showing the traffic flow levels on the links in the network, after applying a threshold to convert it from a continuous to a discrete value. - There are two configurable parameters: - * ``quantisation_levels`` determines how many discrete bins to use for converting the continuous traffic value to discrete (default is 5). - * ``combine_service_traffic`` determines whether to separately output traffic use for each network protocol or whether to combine them into an overall value for the link. (default is ``True``) - For example, with default parameters and a network with three links, the structure of this component would be: - .. code-block:: - [ - link1_status - link2_status - link3_status - ] - Each ``link_status`` is a number from 0-4 representing the network load in relation to bandwidth. - .. code-block:: - 0 = No traffic (0%) - 1 = low traffic (1%-33%) - 2 = medium traffic (33%-66%) - 3 = high traffic (66%-99%) - 4 = max traffic/ overwhelmed (100%) - Using ``gym`` notation, the shape of the obs space is: ``gym.spaces.MultiDiscrete([5,5,5])``. - Action Spaces - ************** - The action space available to the blue agent comes in two types: - 1. Node-based - 2. Access Control List - 3. Any (Agent can take both node-based and ACL-based actions) - The choice of action space used during a training session is determined in the config_[name].yaml file. - **Node-Based** - The agent is able to influence the status of nodes by switching them off, resetting, or FIXING operating systems and services. In this instance, the action space is a Gymnasium spaces.Discrete type, as follows: - * Dictionary item {... ,1: [x1, x2, x3,x4] ...} - The placeholders inside the list under the key '1' mean the following: - * [0, num nodes] - Node ID (0 = nothing, node ID) - * [0, 4] - What property it's acting on (0 = nothing, 1 = state, 2 = SoftwareState, 3 = service state, 4 = file system state) - * [0, 3] - Action on property (0 = nothing, 1 = on / scan, 2 = off / repair, 3 = reset / patch / restore) - * [0, num services] - Resolves to service ID (0 = nothing, resolves to service) - **Access Control List** - The blue agent is able to influence the configuration of the Access Control List rule set (which implements a system-wide firewall). In this instance, the action space is an Gymnasium spaces.Discrete type, as follows: - * Dictionary item {... ,1: [x1, x2, x3, x4, x5, x6] ...} - The placeholders inside the list under the key '1' mean the following: - * [0, 2] - Action (0 = do nothing, 1 = create rule, 2 = delete rule) - * [0, 1] - Permission (0 = DENY, 1 = PERMIT) - * [0, num nodes] - Source IP (0 = any, then 1 -> x resolving to IP addresses) - * [0, num nodes] - Dest IP (0 = any, then 1 -> x resolving to IP addresses) - * [0, num services] - Protocol (0 = any, then 1 -> x resolving to protocol) - * [0, num ports] - Port (0 = any, then 1 -> x resolving to port) - **ANY** - The agent is able to carry out both **Node-Based** and **Access Control List** operations. - This means the dictionary will contain key-value pairs in the format of BOTH Node-Based and Access Control List as seen above. - Rewards - ******* - A reward value is presented back to the blue agent on the conclusion of every step. The reward value is calculated via two methods which combine to give the total value: - 1. Node and service status - 2. IER status - **Node and service status** - On every step, the status of each node is compared against both a reference environment (simulating the situation if the red and blue agents had not impacted the environment) - and the before and after state of the environment. If the comparison against the reference environment shows no difference, then the score provided is "AllOK". If there is a - difference with respect to the reference environment, the before and after states are compared, and a score determined. See :ref:`config` for details of reward values. - **IER status** - On every step, the full IER set is examined to determine whether green and red agent IERs are being permitted to run. Any red agent IERs running incur a penalty; any green agent - IERs not permitted to run also incur a penalty. See :ref:`config` for details of reward values. - Future Enhancements - ******************* - The PrimAITE project has an ambition to include the following enhancements in future releases: - * Integration with a suitable standardised framework to allow multi-agent integration - * Integration with external threat emulation tools, either using off-line data, or integrating at runtime diff --git a/docs/source/action_masking.rst b/docs/source/action_masking.rst index bee4674b..c779ecb5 100644 --- a/docs/source/action_masking.rst +++ b/docs/source/action_masking.rst @@ -2,11 +2,13 @@ © Crown-owned copyright 2025, Defence Science and Technology Laboratory UK +.. _action_masking: + Action Masking ************** The PrimAITE simulation is able to provide action masks in the environment output. These action masks let the agents know about which actions are invalid based on the current environment state. For instance, it's not possible to install -software on a node that is turned off. Therefore, if an agent has a NODE_SOFTWARE_INSTALL in it's action map for that node, +software on a node that is turned off. Therefore, if an agent has a ``node-software-install`` in it's action map for that node, the action mask will show `0` in the corresponding entry. *Note: just because an action is available in the action mask does not mean it will be successful when executed. It just means it's possible to try to execute the action at this time.* @@ -20,132 +22,127 @@ Masking Logic ============= The following logic is applied: -+------------------------------------------+---------------------------------------------------------------------+ -| Action | Action Mask Logic | -+==========================================+=====================================================================+ -| **do-nothing** | Always Possible. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-scan** | Node is on. Service is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-stop** | Node is on. Service is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-start** | Node is on. Service is stopped. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-pause** | Node is on. Service is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-resume** | Node is on. Service is paused. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-restart** | Node is on. Service is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-disable** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-enable** | Node is on. Service is disabled. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-service-fix** | Node is on. Service is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-execute** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-scan** | Node is on. Application is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-close** | Node is on. Application is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-fix** | Node is on. Application is running. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-install** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-application-remove** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-scan** | Node is on. File exists. File not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-create** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-checkhash** | Node is on. File exists. File not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-delete** | Node is on. File exists. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-repair** | Node is on. File exists. File not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-restore** | Node is on. File exists. File is deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-corrupt** | Node is on. File exists. File not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-file-access** | Node is on. File exists. File not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-folder-create** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-folder-scan** | Node is on. Folder exists. Folder not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-folder-checkhash** | Node is on. Folder exists. Folder not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-folder-repair** | Node is on. Folder exists. Folder not deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-folder-restore** | Node is on. Folder exists. Folder is deleted. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-os-scan** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **host-nic-enable** | NIC is disabled. Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **host-nic-disable** | NIC is enabled. Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-shutdown** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-startup** | Node is off. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-reset** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-nmap-ping-scan** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-nmap-port-scan** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-network-service-recon** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **network-port-enable** | Node is on. Router is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **network-port-disable** | Router is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **router-acl-add-rule** | Router is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **router-acl-remove-rule** | Router is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **firewall-acl-add-rule** | Firewall is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **firewall-acl-remove-rule** | Firewall is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **configure-database-client** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **configure-ransomware-script** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **c2-server-ransomware-configure** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **configure-dos-bot** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **configure-c2-beacon** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **c2-server-ransomware-launch** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **c2-server-terminal-command** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **c2-server-data-exfiltrate** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-account-change-password** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-session-remote-login** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-session-remote-logoff** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ -| **node-send-remote-command** | Node is on. | -+------------------------------------------+---------------------------------------------------------------------+ ++------------------------------------------+------------------------------------------------+ +| Action | Action Mask Logic | ++==========================================+================================================+ +| **do-nothing** | Always Possible. | ++------------------------------------------+------------------------------------------------+ +| **node-service-scan** | Node is on. Service is running. | ++------------------------------------------+------------------------------------------------+ +| **node-service-stop** | Node is on. Service is running. | ++------------------------------------------+------------------------------------------------+ +| **node-service-start** | Node is on. Service is stopped. | ++------------------------------------------+------------------------------------------------+ +| **node-service-pause** | Node is on. Service is running. | ++------------------------------------------+------------------------------------------------+ +| **node-service-resume** | Node is on. Service is paused. | ++------------------------------------------+------------------------------------------------+ +| **node-service-restart** | Node is on. Service is running. | ++------------------------------------------+------------------------------------------------+ +| **node-service-disable** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-service-enable** | Node is on. Service is disabled. | ++------------------------------------------+------------------------------------------------+ +| **node-service-fix** | Node is on. Service is running. | ++------------------------------------------+------------------------------------------------+ +| **node-application-execute** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-application-scan** | Node is on. Application is running. | ++------------------------------------------+------------------------------------------------+ +| **node-application-close** | Node is on. Application is running. | ++------------------------------------------+------------------------------------------------+ +| **node-application-fix** | Node is on. Application is running. | ++------------------------------------------+------------------------------------------------+ +| **node-application-install** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-application-remove** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-file-scan** | Node is on. File exists. File not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-file-create** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-file-checkhash** | Node is on. File exists. File not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-file-delete** | Node is on. File exists. | ++------------------------------------------+------------------------------------------------+ +| **node-file-repair** | Node is on. File exists. File not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-file-restore** | Node is on. File exists. File is deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-file-corrupt** | Node is on. File exists. File not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-file-access** | Node is on. File exists. File not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-folder-create** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-folder-scan** | Node is on. Folder exists. Folder not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-folder-checkhash** | Node is on. Folder exists. Folder not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-folder-repair** | Node is on. Folder exists. Folder not deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-folder-restore** | Node is on. Folder exists. Folder is deleted. | ++------------------------------------------+------------------------------------------------+ +| **node-os-scan** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **host-nic-enable** | NIC is disabled. Node is on. | ++------------------------------------------+------------------------------------------------+ +| **host-nic-disable** | NIC is enabled. Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-shutdown** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-startup** | Node is off. | ++------------------------------------------+------------------------------------------------+ +| **node-reset** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-nmap-ping-scan** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-nmap-port-scan** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-network-service-recon** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **network-port-enable** | Node is on. Router is on. | ++------------------------------------------+------------------------------------------------+ +| **network-port-disable** | Router is on. | ++------------------------------------------+------------------------------------------------+ +| **router-acl-add-rule** | Router is on. | ++------------------------------------------+------------------------------------------------+ +| **router-acl-remove-rule** | Router is on. | ++------------------------------------------+------------------------------------------------+ +| **firewall-acl-add-rule** | Firewall is on. | ++------------------------------------------+------------------------------------------------+ +| **firewall-acl-remove-rule** | Firewall is on. | ++------------------------------------------+------------------------------------------------+ +| **configure-database-client** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **configure-ransomware-script** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **c2-server-ransomware-configure** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **configure-dos-bot** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **configure-c2-beacon** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **c2-server-ransomware-launch** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **c2-server-terminal-command** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **c2-server-data-exfiltrate** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-account-change-password** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-session-remote-login** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-session-remote-logoff** | Node is on. | ++------------------------------------------+------------------------------------------------+ +| **node-send-remote-command** | Node is on. | ++------------------------------------------+------------------------------------------------+ Mechanism ========= -The environment iterates over the RL agent's ``action_map`` and generates the corresponding simulator request string. -It uses the ``RequestManager.check_valid()`` method to invoke the relevant ``RequestPermissionValidator`` without -actually running the request on the simulation. +The environment iterates over the RL agent's ``action_map`` and generates the corresponding simulator :ref:`request ` string. It uses the :py:meth:`RequestManager.check_valid()` method to invoke the relevant :py:class:`RequestPermissionValidator ` without actually running the request on the simulation. Current Limitations =================== -Currently, action masking only considers whether the action as a whole is possible, it doesn't verify that the exact -parameter combination passed to the action make sense in the current context. For instance, if ACL rule 3 on router_1 is -already populated, the action for adding another rule at position 3 will be available regardless, as long as that router -is turned on. This will never block valid actions. It will just occasionally allow invalid actions. +Currently, action masking only considers whether the action as a whole is possible, it doesn't verify that the exact parameter combination passed to the action make sense in the current context. or instance, if ACL rule 3 on router_1 is already populated, the action for adding another rule at position 3 will be available regardless, as long as that router is turned on. This will never block valid actions. It will just occasionally allow invalid actions. diff --git a/docs/source/config.rst b/docs/source/config.rst index 0fa4a4d5..14219c15 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -2,6 +2,8 @@ © Crown-owned copyright 2025, Defence Science and Technology Laboratory UK +.. _Configurable_Items: + PrimAITE |VERSION| Configuration ******************************** diff --git a/docs/source/configuration/agents.rst b/docs/source/configuration/agents.rst index c2674e31..96a09448 100644 --- a/docs/source/configuration/agents.rst +++ b/docs/source/configuration/agents.rst @@ -13,20 +13,19 @@ Agents can be scripted (deterministic and stochastic), or controlled by a reinfo .. code-block:: yaml agents: - - ref: red_agent_example - ... - - ref: blue_agent_example - ... - - ref: green_agent_example - team: GREEN - type: probabilistic-agent + - ref: red_agent_example + ... + - ref: blue_agent_example + ... + - ref: green_agent_example + team: GREEN + type: probabilistic-agent - agent_settings: - start_settings: - start_step: 5 - frequency: 4 - variance: 3 - flatten_obs: False + agent_settings: + start_step: 5 + frequency: 4 + variance: 3 + flatten_obs: False ``ref`` ------- @@ -97,8 +96,7 @@ Similar to action space, this is defined as a list of components from the :py:mo ``reward_components`` ^^^^^^^^^^^^^^^^^^^^^ -TODO: update description -A list of reward types from :py:mod:`primaite.game.agent.rewards.RewardFunction.rew_class_identifiers` +A list of available reward types from :py:mod:`primaite.game.agent.rewards.RewardFunction.rew_class_identifiers` e.g. @@ -106,7 +104,13 @@ e.g. reward_components: - type: dummy + weight: 1.0 - type: database-file-integrity + weight: 0.40 + options: + node_hostname: database_server + folder_name: database + file_name: database.db ``agent_settings`` @@ -119,10 +123,9 @@ e.g. .. code-block:: yaml agent_settings: - start_settings: - start_step: 25 - frequency: 20 - variance: 5 + start_step: 25 + frequency: 20 + variance: 5 ``start_step`` ^^^^^^^^^^^^^^ @@ -154,4 +157,4 @@ If ``True``, gymnasium flattening will be performed on the observation space bef ----------------- Agents will record their action log for each step. This is a summary of what the agent did, along with response information from requests within the simulation. -A summary of the actions taken by the agent can be viewed using the `show_history()` function. By default, this will display all actions taken apart from ``DONOTHING``. +A summary of the actions taken by the agent can be viewed using the `show_history()` function. By default, this will display all actions taken apart from ``do-nothing``. diff --git a/docs/source/configuration/game.rst b/docs/source/configuration/game.rst index b3c139b2..02c2a441 100644 --- a/docs/source/configuration/game.rst +++ b/docs/source/configuration/game.rst @@ -42,14 +42,14 @@ The maximum number of episodes a Reinforcement Learning agent(s) can be trained A list of ports that the Reinforcement Learning agent(s) are able to see in the observation space. -See :ref:`List of Ports ` for a list of ports. +See :py:const:`primaite.utils.validation.port.PORT_LOOKUP` for a list of ports. ``protocols`` ------------- A list of protocols that the Reinforcement Learning agent(s) are able to see in the observation space. -See :ref:`List of IPProtocols ` for a list of protocols. +See :py:const:`primaite.utils.validation.ip_protocol.PROTOCOL_LOOKUP` for a list of protocols. ``thresholds`` -------------- @@ -59,4 +59,4 @@ These are used to determine the thresholds of high, medium and low categories fo ``seed`` -------- -Used to configure the random seeds used within PrimAITE, ensuring determinism within episode/session runs. If empty or set to -1, no seed is set. +Used to configure the random seeds used within PrimAITE, ensuring determinism within episode/session runs. If empty or set to -1, no seed is set. The given seed value is logged (by default) in ``primaite//sessions//