oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

#2840 Documentation and minor bug fixes found in terminal and session manager.

Browse Source
This commit is contained in:
Archer Bowen
2024-09-02 16:55:43 +01:00
parent 987546f77f
commit fd3d3812f6
6 changed files with 385 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@@ -6,6 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] ## [Unreleased]
- New ``NODE_SEND_LOCAL_COMMAND`` action implemented which grants agents the ability to execute commands locally. (Previously limited to remote only)
### Added ### Added
- Random Number Generator Seeding by specifying a random number seed in the config file. - Random Number Generator Seeding by specifying a random number seed in the config file.
- Implemented Terminal service class, providing a generic terminal simulation. - Implemented Terminal service class, providing a generic terminal simulation.

106
docs/source/simulation_components/system/services/terminal.rst
View File

@@ -30,6 +30,7 @@ Usage
- Terminal Clients connect, execute commands and disconnect from remote nodes. - Terminal Clients connect, execute commands and disconnect from remote nodes.
- Ensures that users are logged in to the component before executing any commands. - Ensures that users are logged in to the component before executing any commands.
- Service runs on SSH port 22 by default. - Service runs on SSH port 22 by default.
- Enables Agents to send commands both remotely and locally.
Implementation Implementation
"""""""""""""" """"""""""""""
@@ -39,9 +40,110 @@ Implementation
- Extends Service class. - Extends Service class.
- A detailed guide on the implementation and functionality of the Terminal class can be found in the "Terminal-Processing" jupyter notebook. - A detailed guide on the implementation and functionality of the Terminal class can be found in the "Terminal-Processing" jupyter notebook.
Command Format
^^^^^^^^^^^^^^
``Terminals`` implement their commands through leveraging the pre-existing :doc:`../../request_system`.
Due to this ``Terminals`` will only accept commands passed within the ``RequestFormat``.
:py:class:`primaite.game.interface.RequestFormat`
For example, ``terminal`` command actions when used in ``yaml`` format are formatted as follows:
.. code-block:: yaml
command:
- "file_system"
- "create"
- "file"
- "downloads"
- "cat.png"
- "False"
**This command creates file called ``cat.png`` within the ``downloads`` folder.**
This is then loaded from ``yaml`` into a dictionary containing the terminal command:
.. code-block:: python
{"command":["file_system", "create", "file", "downloads", "cat.png", "False"]}
Which is then parsed to the ``Terminals`` Request Manager to be executed.
Game Layer Usage (Agents)
========================
The below code examples demonstrate how to use terminal related actions in yaml files.
yaml
""""
``NODE_SEND_LOCAL_COMMAND``
"""""""""""""""""""""""""""
Agents can execute local commands without needing to perform a separate remote login action (``SSH_TO_REMOTE``).
.. code-block:: yaml
...
...
action: NODE_SEND_LOCAL_COMMAND
options:
node_id: 0
username: admin
password: admin
command: # Example command - Creates a file called 'cat.png' in the downloads folder.
- "file_system"
- "create"
- "file"
- "downloads"
- "cat.png"
- "False"
``SSH_TO_REMOTE``
"""""""""""""""""
Agents are able to use the terminal to login into remote nodes via ``SSH`` which allows for agents to execute commands on remote hosts.
.. code-block:: yaml
...
...
action: SSH_TO_REMOTE
options:
node_id: 0
username: admin
password: admin
remote_ip: 192.168.0.10 # Example Ip Address. (The remote host's IP that will be used by ssh)
``NODE_SEND_REMOTE_COMMAND``
""""""""""""""""""""""""""""
After remotely login into another host, a agent can use the ``NODE_SEND_REMOTE_COMMAND`` to execute commands across the network remotely.
.. code-block:: yaml
...
...
action: NODE_SEND_REMOTE_COMMAND
options:
node_id: 0
remote_ip: 192.168.0.10
command:
- "file_system"
- "create"
- "file"
- "downloads"
- "cat.png"
- "False"
Simulation Layer Usage
======================
Usage
=====
The below code examples demonstrate how to create a terminal, a remote terminal, and how to send a basic application install command to a remote node. The below code examples demonstrate how to create a terminal, a remote terminal, and how to send a basic application install command to a remote node.

274
src/primaite/notebooks/Terminal-Processing.ipynb
View File

@@ -9,6 +9,13 @@
"© Crown-owned copyright 2024, Defence Science and Technology Laboratory UK" "© Crown-owned copyright 2024, Defence Science and Technology Laboratory UK"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simulation Layer Implementation."
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
@@ -198,6 +205,271 @@
"source": [ "source": [
"computer_b.user_session_manager.show(include_historic=True, include_session_id=True)" "computer_b.user_session_manager.show(include_historic=True, include_session_id=True)"
] ]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Game Layer Implementation\n",
"\n",
"This notebook section will detail the implementation of how the game layer utilises the terminal to support different agent actions.\n",
"\n",
"The ``Terminal`` is used in a variety of different ways in the game layer. Specifically, the terminal is leveraged to implement the following actions:\n",
"\n",
"\n",
"| Game Layer Action | Simulation Layer |\n",
"|-----------------------------------|--------------------------|\n",
"| ``NODE_SEND_LOCAL_COMMAND`` | Uses the given user credentials, creates a ``LocalTerminalSession`` and executes the given command and returns the ``RequestResponse``.\n",
"| ``SSH_TO_REMOTE`` | Uses the given user credentials and remote IP to create a ``RemoteTerminalSession``.\n",
"| ``NODE_SEND_REMOTE_COMMAND`` | Uses the given remote IP to locate the correct ``RemoteTerminalSession``, executes the given command and returns the ``RequestsResponse``."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Game Layer Setup\n",
"\n",
"Similar to other notebooks, the next code cells create a custom proxy agent to demonstrate how these commands can be leveraged by agents in the ``UC2`` network environment.\n",
"\n",
"If you're unfamiliar with ``UC2`` then please refer to the [UC2-E2E-Demo notebook for further reference](./Data-Manipulation-E2E-Demonstration.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import yaml\n",
"from primaite.config.load import data_manipulation_config_path\n",
"from primaite.session.environment import PrimaiteGymEnv"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"custom_terminal_agent = \"\"\"\n",
" - ref: CustomC2Agent\n",
" team: RED\n",
" type: ProxyAgent\n",
" observation_space: null\n",
" action_space:\n",
" action_list:\n",
" - type: DONOTHING\n",
" - type: NODE_SEND_LOCAL_COMMAND\n",
" - type: SSH_TO_REMOTE\n",
" - type: NODE_SEND_REMOTE_COMMAND\n",
" options:\n",
" nodes:\n",
" - node_name: client_1\n",
" max_folders_per_node: 1\n",
" max_files_per_folder: 1\n",
" max_services_per_node: 2\n",
" max_nics_per_node: 8\n",
" max_acl_rules: 10\n",
" ip_list:\n",
" - 192.168.1.21\n",
" - 192.168.1.14\n",
" wildcard_list:\n",
" - 0.0.0.1\n",
" action_map:\n",
" 0:\n",
" action: DONOTHING\n",
" options: {}\n",
" 1:\n",
" action: NODE_SEND_LOCAL_COMMAND\n",
" options:\n",
" node_id: 0\n",
" username: admin\n",
" password: admin\n",
" command:\n",
" - file_system\n",
" - create\n",
" - file\n",
" - downloads\n",
" - dog.png\n",
" - False\n",
" 2:\n",
" action: SSH_TO_REMOTE\n",
" options:\n",
" node_id: 0\n",
" username: admin\n",
" password: admin\n",
" remote_ip: 192.168.10.22\n",
" 3:\n",
" action: NODE_SEND_REMOTE_COMMAND\n",
" options:\n",
" node_id: 0\n",
" remote_ip: 192.168.10.22\n",
" command:\n",
" - file_system\n",
" - create\n",
" - file\n",
" - downloads\n",
" - cat.png\n",
" - False\n",
" reward_function:\n",
" reward_components:\n",
" - type: DUMMY\n",
"\"\"\"\n",
"custom_terminal_agent_yaml = yaml.safe_load(custom_terminal_agent)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"with open(data_manipulation_config_path()) as f:\n",
" cfg = yaml.safe_load(f)\n",
" # removing all agents & adding the custom agent.\n",
" cfg['agents'] = {}\n",
" cfg['agents'] = custom_terminal_agent_yaml\n",
" \n",
"env = PrimaiteGymEnv(env_config=cfg)\n",
"\n",
"client_1: Computer = env.game.simulation.network.get_node_by_hostname(\"client_1\")\n",
"client_2: Computer = env.game.simulation.network.get_node_by_hostname(\"client_2\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Terminal Action | ``NODE_SEND_LOCAL_COMMAND`` \n",
"\n",
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: NODE_SEND_LOCAL_COMMAND\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 1:\n",
" action: NODE_SEND_LOCAL_COMMAND\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" username: admin\n",
" password: admin\n",
" command:\n",
" - file_system\n",
" - create\n",
" - file\n",
" - downloads\n",
" - dog.png\n",
" - False\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"env.step(1)\n",
"client_1.file_system.show(full=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Terminal Action | ``SSH_TO_REMOTE`` \n",
"\n",
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: SSH_TO_REMOTE\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 2:\n",
" action: SSH_TO_REMOTE\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" username: admin\n",
" password: admin\n",
" remote_ip: 192.168.10.22 # client_2's ip address.\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"env.step(2)\n",
"client_2.session_manager.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Terminal Action | ``NODE_SEND_REMOTE_COMMAND``\n",
"\n",
"The yaml snippet below shows all the relevant agent options for this action:\n",
"\n",
"```yaml\n",
"\n",
" action_space:\n",
" action_list:\n",
" ...\n",
" - type: NODE_SEND_REMOTE_COMMAND\n",
" ...\n",
" options:\n",
" nodes: # Node List\n",
" - node_name: client_1\n",
" ...\n",
" ...\n",
" action_map:\n",
" 1:\n",
" action: NODE_SEND_REMOTE_COMMAND\n",
" options:\n",
" node_id: 0 # Index 0 at the node list.\n",
" remote_ip: 192.168.10.22\n",
" commands:\n",
" - file_system\n",
" - create\n",
" - file\n",
" - downloads\n",
" - cat.png\n",
" - False\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"env.step(3)\n",
"client_2.file_system.show(full=True)"
]
} }
], ],
"metadata": { "metadata": {
@@ -216,7 +488,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.10.11" "version": "3.10.12"
} }
}, },
"nbformat": 4, "nbformat": 4,

2
src/primaite/simulator/system/core/session_manager.py
View File

@@ -413,5 +413,5 @@ class SessionManager:
table.align = "l" table.align = "l"
table.title = f"{self.sys_log.hostname} Session Manager" table.title = f"{self.sys_log.hostname} Session Manager"
for session in self.sessions_by_key.values(): for session in self.sessions_by_key.values():
table.add_row([session.dst_ip_address, session.dst_port.value, session.protocol.name]) table.add_row([session.with_ip_address, session.dst_port.value, session.protocol.name])
print(table) print(table)

1
tests/integration_tests/game_layer/actions/test_terminal_actions.py
View File

@@ -215,3 +215,4 @@ def test_local_terminal(game_and_agent_fixture: Tuple[PrimaiteGame, ProxyAgent])
game.step() game.step()
assert client_1.file_system.get_file("folder123", "cat.pdf") is None assert client_1.file_system.get_file("folder123", "cat.pdf") is None
client_1.session_manager.show()