oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

2623 Add docs for action masking

Browse Source
This commit is contained in:
Marek Wolan
2024-07-11 10:01:33 +01:00
parent 0c58c3969a
commit bdf5ff2167
2 changed files with 81 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/index.rst
View File

@@ -123,6 +123,7 @@ Head over to the :ref:`getting-started` page to install and setup PrimAITE!
source/environment
source/customising_scenarios
source/varying_config_files
source/action_masking
.. toctree::
:caption: Notebooks:

80
docs/source/action_masking.rst Normal file
View File

@@ -0,0 +1,80 @@
.. only:: comment
© Crown-owned copyright 2024, Defence Science and Technology Laboratory UK
Action Masking
**************
The PrimAITE simulation is able to provide action masks in the environment output. These action masks let the agents know
about which actions are invalid based on the current environment state. For instance, it's not possible to install
software on a node that is turned off. Therefore, if an agent has a NODE_SOFTWARE_INSTALL in it's action map for that node,
the action mask will show `0` in the corresponding entry.
Configuration
=============
Action masking is supported for agents that use the `ProxyAgent` class (the class used for connecting to RL algorithms).
In order to use action masking, set the agent_settings.action_masking parameter to True in the config file.
Masking Logic
=============
The following logic is applied:
* **DONOTHING** : Always possible
* **NODE_HOST_SERVICE_SCAN** : Node is on. Service is running.
* **NODE_HOST_SERVICE_STOP** : Node is on. Service is running.
* **NODE_HOST_SERVICE_START** : Node is on. Service is stopped.
* **NODE_HOST_SERVICE_PAUSE** : Node is on. Service is running.
* **NODE_HOST_SERVICE_RESUME** : Node is on. Service is paused.
* **NODE_HOST_SERVICE_RESTART** : Node is on. Service is running.
* **NODE_HOST_SERVICE_DISABLE** : Node is on.
* **NODE_HOST_SERVICE_ENABLE** : Node is on. Service is disabled.
* **NODE_HOST_SERVICE_FIX** : Node is on. Service is running.
* **NODE_HOST_APPLICATION_EXECUTE** : Node is on.
* **NODE_HOST_APPLICATION_SCAN** : Node is on. Application is running.
* **NODE_HOST_APPLICATION_CLOSE** : Node is on. Application is running.
* **NODE_HOST_APPLICATION_FIX** : Node is on. Application is running.
* **NODE_HOST_APPLICATION_INSTALL** : Node is on.
* **NODE_HOST_APPLICATION_REMOVE** : Node is on.
* **NODE_HOST_FILE_SCAN** : Node is on. File exists. File not deleted.
* **NODE_HOST_FILE_CREATE** : Node is on.
* **NODE_HOST_FILE_CHECKHASH** : Node is on. File exists. File not deleted.
* **NODE_HOST_FILE_DELETE** : Node is on. File exists.
* **NODE_HOST_FILE_REPAIR** : Node is on. File exists. File not deleted.
* **NODE_HOST_FILE_RESTORE** : Node is on. File exists. File is deleted.
* **NODE_HOST_FILE_CORRUPT** : Node is on. File exists. File not deleted.
* **NODE_HOST_FILE_ACCESS** : Node is on. File exists. File not deleted.
* **NODE_HOST_FOLDER_CREATE** : Node is on.
* **NODE_HOST_FOLDER_SCAN** : Node is on. Folder exists. Folder not deleted.
* **NODE_HOST_FOLDER_CHECKHASH** : Node is on. Folder exists. Folder not deleted.
* **NODE_HOST_FOLDER_REPAIR** : Node is on. Folder exists. Folder not deleted.
* **NODE_HOST_FOLDER_RESTORE** : Node is on. Folder exists. Folder is deleted.
* **NODE_HOST_OS_SCAN** : Node is on.
* **NODE_HOST_NIC_ENABLE** : NIC is disabled. Node is on.
* **NODE_HOST_NIC_DISABLE** : NIC is enabled. Node is on.
* **NODE_HOST_SHUTDOWN** : Node is on.
* **NODE_HOST_STARTUP** : Node is off.
* **NODE_HOST_RESET** : Node is on.
* **NODE_HOST_NMAP_PING_SCAN** : Node is on.
* **NODE_HOST_NMAP_PORT_SCAN** : Node is on.
* **NODE_HOST_NMAP_NETWORK_SERVICE_RECON** : Node is on.
* **NODE_ROUTER_PORT_ENABLE** : Router is on.
* **NODE_ROUTER_PORT_DISABLE** : Router is on.
* **NODE_ROUTER_ACL_ADDRULE** : Router is on.
* **NODE_ROUTER_ACL_REMOVERULE** : Router is on.
* **NODE_FIREWALL_PORT_ENABLE** : Firewall is on.
* **NODE_FIREWALL_PORT_DISABLE** : Firewall is on.
* **NODE_FIREWALL_ACL_ADDRULE** : Firewall is on.
* **NODE_FIREWALL_ACL_REMOVERULE** : Firewall is on.
Mechanism
=========
The environment iterates over the RL agent's ``action_map`` and generates the corresponding simulator request string.
It uses the ``RequestManager.check_valid()`` method to invoke the relevant ``RequestPermissionValidator`` without
actually running the request on the simulation.
Current Limitations
===================
Currently, action masking only considers whether the action as a whole is possible, it doesn't verify that the exact
parameter combination passed to the action make sense in the current context. For instance, if ACL rule 3 on router_1 is
already populated, the action for adding another rule at position 3 will be available regardless, as long as that router
is turned on. This will never block valid actions. It will just occasionally allow invalid actions.