Merged PR 307: #2369: commiting work done so far

## Summary
Added a page to explain how to use the Jupyter notebooks via jupyter command and VSCode

Also added Nick's suggestion to fix #2226

Going to be honest - not my finest pull request

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2369: commiting work done so far

Related work items: #2369

docs/conf.py

@@ -10,6 +10,7 @@ import datetime
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 import os
 import sys
+from typing import Any
 
 import furo  # noqa
 
@@ -63,3 +64,21 @@ html_theme = "furo"
 html_static_path = ["_static"]
 html_theme_options = {"globaltoc_collapse": True, "globaltoc_maxdepth": 2}
 html_copy_source = False
+
+
+def replace_token(app: Any, docname: Any, source: Any):
+    """Replaces a token from the list of tokens."""
+    result = source[0]
+    for key in app.config.tokens:
+        result = result.replace(key, app.config.tokens[key])
+    source[0] = result
+
+
+tokens = {"{VERSION}": release}  # Token VERSION is replaced by the value of the PrimAITE version in the version file
+"""Dict containing the tokens that need to be replaced in documentation."""
+
+
+def setup(app: Any):
+    """Custom setup for sphinx."""
+    app.add_config_value("tokens", {}, True)
+    app.connect("source-read", replace_token)

docs/index.rst

@@ -105,6 +105,7 @@ Head over to the :ref:`getting-started` page to install and setup PrimAITE!
 
    source/getting_started
    source/primaite_session
+   source/example_notebooks
    source/simulation
    source/game_layer
    source/config

docs/source/example_notebooks.rst

@@ -0,0 +1,77 @@
+.. only:: comment
+
+    © Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
+
+Example Jupyter Notebooks
+=========================
+
+There are a few example notebooks included which help with the understanding of PrimAITE's capabilities.
+
+The Jupyter Notebooks can be run via the 2 examples below. These assume that the instructions to install PrimAITE from the :ref:`Getting Started <getting-started>` page is completed as a prerequisite.
+
+Running Jupyter Notebooks
+-------------------------
+
+1. Navigate to the PrimAITE directory
+
+.. code-block:: bash
+    :caption: Unix
+
+    cd ~/primaite/{VERSION}
+
+.. code-block:: powershell
+    :caption: Windows (Powershell)
+
+    cd ~\primaite\{VERSION}
+
+2. Run jupyter notebook (the python environment to which you installed PrimAITE must be active)
+
+.. code-block:: bash
+    :caption: Unix
+
+    jupyter notebook
+
+.. code-block:: powershell
+    :caption: Windows (Powershell)
+
+    jupyter notebook
+
+3. Opening the jupyter webpage (optional)
+
+The default web browser may automatically open the webpage. However, if that is not the case, click the link shown in your command prompt output. It should look like this: ``http://localhost:8888/?token=0123456798abc0123456789abc``
+
+
+4. Navigate to the list of notebooks
+
+The example notebooks are located in ``notebooks/example_notebooks/``. The file system shown in the jupyter webpage is relative to the location in which the ``jupyter notebook`` command was used.
+
+
+Running Jupyter Notebooks via VSCode
+------------------------------------
+
+It is also possible to view the Jupyter notebooks within VSCode.
+
+The best place to start is by opening a notebook file (.ipynb) in VSCode. If using VSCode to view a notebook for the first time, follow the steps below.
+
+Installing extensions
+"""""""""""""""""""""
+
+VSCode may need some extensions to be installed if not already done.
+To do this, press the "Select Kernel" button on the top right.
+
+This should open a dialog which has the option to install python and jupyter extensions.
+
+.. image:: ../../_static/notebooks/install_extensions.png
+    :width: 700
+    :align: center
+    :alt: ::    The top dialog option that appears will automatically install the extensions
+
+The following extensions should now be installed
+
+.. image:: ../../_static/notebooks/extensions.png
+    :width: 300
+    :align: center
+
+VSCode will then ask for a Python environment version to use. PrimAITE is compatible with Python versions 3.8 - 3.10
+
+You should now be able to interact with the notebook.

docs/source/getting_started.rst

@@ -11,7 +11,7 @@ Getting Started
 
 Pre-Requisites
 
-In order to get **PrimAITE** installed, you will need to have a python version between 3.8 and 3.11 installed. If you don't already have it, this is how to install it:
+In order to get **PrimAITE** installed, you will need Python, venv, and pip. If you don't already have them, this is how to install it:
 
 
 .. code-block:: bash
@@ -30,6 +30,8 @@ In order to get **PrimAITE** installed, you will need to have a python version b
 
 **PrimAITE** is designed to be OS-agnostic, and thus should work on most variations/distros of Linux, Windows, and MacOS.
 
+Installing PrimAITE has been tested with all supported python versions, venv 20.24.1, and pip 23.
+
 Install PrimAITE
 ****************
 
@@ -38,12 +40,12 @@ Install PrimAITE
 .. code-block:: bash
     :caption: Unix
 
-    mkdir ~/primaite/3.0.0
+    mkdir -p ~/primaite/{VERSION}
 
 .. code-block:: powershell
     :caption: Windows (Powershell)
 
-    mkdir ~\primaite\3.0.0
+    mkdir ~\primaite\{VERSION}
 
 
 2. Navigate to the primaite directory and create a new python virtual environment (venv)
@@ -51,13 +53,13 @@ Install PrimAITE
 .. code-block:: bash
     :caption: Unix
 
-    cd ~/primaite/3.0.0
+    cd ~/primaite/{VERSION}
     python3 -m venv .venv
 
 .. code-block:: powershell
     :caption: Windows (Powershell)
 
-        cd ~\primaite\3.0.0
+        cd ~\primaite\{VERSION}
         python3 -m venv .venv
         attrib +h .venv /s /d # Hides the .venv directory
 

docs/source/primaite_session.rst

@@ -35,7 +35,7 @@ Outputs
 -------
 
 Running a session creates a session output directory in your user data folder. The filepath looks like this:
-``~/primaite/3.0.0/sessions/YYYY-MM-DD/HH-MM-SS/``. This folder contains the simulation sys logs generated by each node,
+``~/primaite/{VERSION}/sessions/YYYY-MM-DD/HH-MM-SS/``. This folder contains the simulation sys logs generated by each node,
 the saved agent checkpoints, and final model. The folder also contains a .json file for each episode step that
 contains the action, reward, and simulation state. These can be found in
-``~/primaite/3.0.0/sessions/YYYY-MM-DD/HH-MM-SS/simulation_output/episode_<n>/step_metadata/step_<n>.json``
+``~/primaite/{VERSION}/sessions/YYYY-MM-DD/HH-MM-SS/simulation_output/episode_<n>/step_metadata/step_<n>.json``

src/primaite/config/_package_data/data_manipulation_marl.yaml

@@ -13,8 +13,6 @@ training_config:
 
 
 io_settings:
-  save_checkpoints: true
-  checkpoint_interval: 5
   save_agent_actions: true
   save_step_metadata: false
   save_pcap_logs: false

src/primaite/game/agent/rewards.py

@@ -111,7 +111,7 @@ class DatabaseFileIntegrity(AbstractReward):
         """
         database_file_state = access_from_nested_dict(state, self.location_in_state)
         if database_file_state is NOT_PRESENT_IN_STATE:
-            _LOGGER.info(
+            _LOGGER.debug(
                 f"Could not calculate {self.__class__} reward because "
                 "simulation state did not contain enough information."
             )
@@ -231,7 +231,7 @@ class WebpageUnavailablePenalty(AbstractReward):
         # If the last request did actually go through, then check if the webpage also loaded
         web_browser_state = access_from_nested_dict(state, self.location_in_state)
         if web_browser_state is NOT_PRESENT_IN_STATE or "history" not in web_browser_state:
-            _LOGGER.info(
+            _LOGGER.debug(
                 "Web browser reward could not be calculated because the web browser history on node",
                 f"{self._node} was not reported in the simulation state. Returning 0.0",
             )

src/primaite/notebooks/Data-Manipulation-Customising-Red-Agent.ipynb

@@ -22,6 +22,7 @@
     "# Imports\n",
     "\n",
     "from primaite.config.load import data_manipulation_config_path\n",
+    "from primaite.game.agent.interface import AgentActionHistoryItem\n",
     "from primaite.session.environment import PrimaiteGymEnv\n",
     "import yaml\n",
     "from pprint import pprint"
@@ -62,12 +63,12 @@
    "source": [
     "def friendly_output_red_action(info):\n",
     "    # parse the info dict form step output and write out what the red agent is doing\n",
-    "    red_info = info['agent_actions']['data_manipulation_attacker']\n",
+    "    red_info : AgentActionHistoryItem = info['agent_actions']['data_manipulation_attacker']\n",
-    "    red_action = red_info[0]\n",
+    "    red_action = red_info.action\n",
     "    if red_action == 'DONOTHING':\n",
     "        red_str = 'DO NOTHING'\n",
     "    elif red_action == 'NODE_APPLICATION_EXECUTE':\n",
-    "        client = \"client 1\" if red_info[1]['node_id'] == 0 else \"client 2\"\n",
+    "        client = \"client 1\" if red_info.parameters['node_id'] == 0 else \"client 2\"\n",
     "        red_str = f\"ATTACK from {client}\"\n",
     "    return red_str"
    ]

src/primaite/notebooks/Data-Manipulation-E2E-Demonstration.ipynb

@@ -576,7 +576,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now, even though the red agent executes an attack, the reward stays at 0.8."
+    "Now, even though the red agent executes an attack, the reward will stay at 0.8."
    ]
   },
   {
@@ -616,12 +616,12 @@
     "    tries += 1\n",
     "    obs, reward, terminated, truncated, info = env.step(0)\n",
     "\n",
-    "    if obs['NODES'][6]['NETWORK_INTERFACES'][1]['nmne']['outbound'] == 1:\n",
+    "    if obs['NODES'][6]['NICS'][1]['NMNE']['outbound'] == 1:\n",
     "        # client 1 has NMNEs, let's block it\n",
     "        obs, reward, terminated, truncated, info = env.step(50) # block client 1\n",
     "        print(\"blocking client 1\")\n",
     "        break\n",
-    "    elif obs['NODES'][7]['NETWORK_INTERFACES'][1]['nmne']['outbound'] == 1:\n",
+    "    elif obs['NODES'][7]['NICS'][1]['NMNE']['outbound'] == 1:\n",
     "        # client 2 has NMNEs, so let's block it\n",
     "        obs, reward, terminated, truncated, info = env.step(51) # block client 2\n",
     "        print(\"blocking client 2\")\n",

src/primaite/simulator/network/airspace.py

@@ -157,7 +157,7 @@ class WirelessNetworkInterface(NetworkInterface, ABC):
             return
 
         if not self._connected_node:
-            _LOGGER.error(f"Interface {self} cannot be enabled as it is not connected to a Node")
+            _LOGGER.warning(f"Interface {self} cannot be enabled as it is not connected to a Node")
             return
 
         if self._connected_node.operating_state != NodeOperatingState.ON:

src/primaite/simulator/network/hardware/base.py

@@ -297,7 +297,7 @@ class WiredNetworkInterface(NetworkInterface, ABC):
             return True
 
         if not self._connected_node:
-            _LOGGER.error(f"Interface {self} cannot be enabled as it is not connected to a Node")
+            _LOGGER.warning(f"Interface {self} cannot be enabled as it is not connected to a Node")
             return False
 
         if self._connected_node.operating_state != NodeOperatingState.ON:
@@ -343,11 +343,11 @@ class WiredNetworkInterface(NetworkInterface, ABC):
         :param link: The Link instance to connect to this network interface.
         """
         if self._connected_link:
-            _LOGGER.error(f"Cannot connect Link to network interface {self} as it already has a connection")
+            _LOGGER.warning(f"Cannot connect Link to network interface {self} as it already has a connection")
             return
 
         if self._connected_link == link:
-            _LOGGER.error(f"Cannot connect Link to network interface {self} as it is already connected")
+            _LOGGER.warning(f"Cannot connect Link to network interface {self} as it is already connected")
             return
 
         self._connected_link = link

src/primaite/simulator/system/applications/application.py

@@ -83,7 +83,7 @@ class Application(IOSoftware):
 
         if self.operating_state is not self.operating_state.RUNNING:
             # service is not running
-            _LOGGER.error(f"Cannot perform action: {self.name} is {self.operating_state.name}")
+            _LOGGER.debug(f"Cannot perform action: {self.name} is {self.operating_state.name}")
             return False
 
         return True

src/primaite/simulator/system/services/service.py

@@ -59,7 +59,7 @@ class Service(IOSoftware):
 
         if self.operating_state is not ServiceOperatingState.RUNNING:
             # service is not running
-            _LOGGER.error(f"Cannot perform action: {self.name} is {self.operating_state.name}")
+            _LOGGER.debug(f"Cannot perform action: {self.name} is {self.operating_state.name}")
             return False
 
         return True