Merged PR 359: Improved Dev CLI tools

## Summary
Improved dev tools:
- can configure what the dev mode can override
  - sys log level
  - outputting sys log
  - outputting pcap logs
  - outputting to terminal
  - the location where to output session (by default will print to the primaite repo root)

## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/359?_a=files&path=/tests/integration_tests/cli/test_dev_cli.py

and a lot of manually checking if the thing outputs to the correct directories

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2447, #2533

.gitignore

@@ -82,6 +82,7 @@ target/
 
 # Jupyter Notebook
 .ipynb_checkpoints
+PPO_UC2/
 
 # IPython
 profile_default/
@@ -150,6 +151,7 @@ docs/source/primaite-dependencies.rst
 # outputs
 src/primaite/outputs/
 simulation_output/
+sessions/
 
 # benchmark session outputs
 benchmark/output

CHANGELOG.md

@@ -14,13 +14,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added ability to define scenarios that change depending on the episode number.
 - Standardised Environment API by renaming the config parameter of `PrimaiteGymEnv` from `game_config` to `env_config`
 - Database Connection ID's are now created/issued by DatabaseService and not DatabaseClient
-- added ability to set PrimAITE between development and production modes via PrimAITE CLI ``mode`` command
 - Updated DatabaseClient so that it can now have a single native DatabaseClientConnection along with a collection of DatabaseClientConnection's.
 - Implemented the uninstall functionality for DatabaseClient so that all connections are terminated at the DatabaseService.
 - Added the ability for a DatabaseService to terminate a connection.
 - Added active_connection to DatabaseClientConnection so that if the connection is terminated active_connection is set to False and the object can no longer be used.
 - Added additional show functions to enable connection inspection.
 - Updates to agent logging, to include the reward both per step and per episode.
+- Introduced Developer CLI tools to assist with developing/debugging PrimAITE
+  - Can be enabled via `primaite dev-mode enable`
+  - Activating dev-mode will change the location where the sessions will be output - by default will output where the PrimAITE repository is located
 - Refactored all air-space usage to that a new instance of AirSpace is created for each instance of Network. This 1:1 relationship between network and airspace will allow parallelization.
 
 

docs/index.rst

@@ -116,6 +116,7 @@ Head over to the :ref:`getting-started` page to install and setup PrimAITE!
    :caption: Developer information:
    :hidden:
 
+   source/developer_tools
    source/state_system
    source/request_system
    PrimAITE API <source/_autosummary/primaite>

docs/source/developer_tools.rst

@@ -0,0 +1,210 @@
+.. only:: comment
+
+    © Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
+
+.. _Developer Tools:
+
+Developer Tools
+***************
+
+PrimAITE includes developer CLI tools that are intended to be used by developers.
+
+dev-mode
+========
+
+The dev-mode contains configuration which override any of the config files during runtime.
+
+This is intended to make debugging easier by removing the need to find the relevant configuration file/settings.
+
+Enabling dev-mode
+-----------------
+
+The PrimAITE dev-mode can be enabled via the use of
+
+.. code-block::
+
+    primaite dev-mode enable
+
+Disabling dev-mode
+------------------
+
+The PrimAITE dev-mode can be disabled via the use of
+
+.. code-block::
+
+    primaite dev-mode disable
+
+Show current mode
+-----------------
+
+To show if the dev-mode is enabled or not, use
+The PrimAITE dev-mode can be disabled via the use of
+
+.. code-block::
+
+    primaite dev-mode show
+
+dev-mode configuration
+======================
+
+The following configures some specific items that the dev-mode overrides, if enabled.
+
+`--sys-log-level` or `-level`
+----------------------------
+
+The level of system logs can be overridden by dev-mode.
+
+By default, this is set to DEBUG
+
+The available options are [DEBUG|INFO|WARNING|ERROR|CRITICAL]
+
+.. code-block::
+
+    primaite dev-mode config -level INFO
+
+or
+
+.. code-block::
+
+    primaite dev-mode config --sys-log-level INFO
+
+`--output-sys-logs` or `-sys`
+-----------------------------
+
+The outputting of system logs can be overridden by dev-mode.
+
+By default, this is set to False
+
+Enabling system logs
+""""""""""""""""""""
+
+To enable outputting of system logs
+
+.. code-block::
+
+    primaite dev-mode config --output-sys-logs
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -sys
+
+Disabling system logs
+"""""""""""""""""""""
+
+To disable outputting of system logs
+
+.. code-block::
+
+    primaite dev-mode config --no-sys-logs
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -nsys
+
+`--output-pcap-logs` or `-pcap`
+-------------------------------
+
+The outputting of packet capture logs can be overridden by dev-mode.
+
+By default, this is set to False
+
+Enabling PCAP logs
+""""""""""""""""""
+
+To enable outputting of packet capture logs
+
+.. code-block::
+
+    primaite dev-mode config --output-pcap-logs
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -pcap
+
+Disabling PCAP logs
+"""""""""""""""""""
+
+To disable outputting of packet capture logs
+
+.. code-block::
+
+    primaite dev-mode config --no-pcap-logs
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -npcap
+
+`--output-to-terminal` or `-t`
+------------------------------
+
+The outputting of system logs to the terminal can be overridden by dev-mode.
+
+By default, this is set to False
+
+Enabling system log output to terminal
+""""""""""""""""""""""""""""""""""""""
+
+To enable outputting of system logs to terminal
+
+.. code-block::
+
+    primaite dev-mode config --output-to-terminal
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -t
+
+Disabling system log output to terminal
+"""""""""""""""""""""""""""""""""""""""
+
+To disable outputting of system logs to terminal
+
+.. code-block::
+
+    primaite dev-mode config --no-terminal
+
+or
+
+.. code-block::
+
+    primaite dev-mode config -nt
+
+path
+----
+
+PrimAITE dev-mode can override where sessions are output.
+
+By default, PrimAITE will output the sessions in USER_HOME/primaite/sessions
+
+With dev-mode enabled, by default, this will be changed to PRIMAITE_REPOSITORY_ROOT/sessions
+
+However, providing a path will let dev-mode output sessions to the given path e.g.
+
+.. code-block:: bash
+    :caption: Unix
+
+    primaite dev-mode config path ~/output/path
+
+.. code-block:: powershell
+    :caption: Windows (Powershell)
+
+    primaite dev-mode config path ~\output\path
+
+default path
+""""""""""""
+
+To reset the path to use the PRIMAITE_REPOSITORY_ROOT/sessions, run the command
+
+.. code-block::
+
+    primaite dev-mode config path --default

docs/source/getting_started.rst

@@ -161,9 +161,11 @@ To set PrimAITE to run in development mode:
 .. code-block:: bash
     :caption: Unix
 
-    primaite mode --dev
+    primaite dev-mode enable
 
 .. code-block:: powershell
     :caption: Windows (Powershell)
 
-    primaite mode --dev
+    primaite dev-mode enable
+
+More information about :ref:`Developer Tools`

src/primaite/__init__.py

@@ -122,35 +122,20 @@ class _PrimaitePaths:
 PRIMAITE_PATHS: Final[_PrimaitePaths] = _PrimaitePaths()
 
 
-def _host_primaite_config() -> None:
-    if not PRIMAITE_PATHS.app_config_file_path.exists():
-        pkg_config_path = Path(pkg_resources.resource_filename("primaite", "setup/_package_data/primaite_config.yaml"))
-        shutil.copy2(pkg_config_path, PRIMAITE_PATHS.app_config_file_path)
-
-
-_host_primaite_config()
-
-
 def _get_primaite_config() -> Dict:
     config_path = PRIMAITE_PATHS.app_config_file_path
     if not config_path.exists():
+        # load from package if config does not exist
         config_path = Path(pkg_resources.resource_filename("primaite", "setup/_package_data/primaite_config.yaml"))
+        # generate app config
+        shutil.copy2(config_path, PRIMAITE_PATHS.app_config_file_path)
     with open(config_path, "r") as file:
+        # load from config
         primaite_config = yaml.safe_load(file)
-    log_level_map = {
+        return primaite_config
-        "NOTSET": logging.NOTSET,
-        "DEBUG": logging.DEBUG,
-        "INFO": logging.INFO,
-        "WARN": logging.WARN,
-        "WARNING": logging.WARN,
-        "ERROR": logging.ERROR,
-        "CRITICAL": logging.CRITICAL,
-    }
-    primaite_config["log_level"] = log_level_map[primaite_config["logging"]["log_level"]]
-    return primaite_config
 
 
-_PRIMAITE_CONFIG = _get_primaite_config()
+PRIMAITE_CONFIG = _get_primaite_config()
 
 
 class _LevelFormatter(Formatter):
@@ -177,11 +162,11 @@ class _LevelFormatter(Formatter):
 
 _LEVEL_FORMATTER: Final[_LevelFormatter] = _LevelFormatter(
     {
-        logging.DEBUG: _PRIMAITE_CONFIG["logging"]["logger_format"]["DEBUG"],
+        logging.DEBUG: PRIMAITE_CONFIG["logging"]["logger_format"]["DEBUG"],
-        logging.INFO: _PRIMAITE_CONFIG["logging"]["logger_format"]["INFO"],
+        logging.INFO: PRIMAITE_CONFIG["logging"]["logger_format"]["INFO"],
-        logging.WARNING: _PRIMAITE_CONFIG["logging"]["logger_format"]["WARNING"],
+        logging.WARNING: PRIMAITE_CONFIG["logging"]["logger_format"]["WARNING"],
-        logging.ERROR: _PRIMAITE_CONFIG["logging"]["logger_format"]["ERROR"],
+        logging.ERROR: PRIMAITE_CONFIG["logging"]["logger_format"]["ERROR"],
-        logging.CRITICAL: _PRIMAITE_CONFIG["logging"]["logger_format"]["CRITICAL"],
+        logging.CRITICAL: PRIMAITE_CONFIG["logging"]["logger_format"]["CRITICAL"],
     }
 )
 
@@ -193,10 +178,10 @@ _FILE_HANDLER: Final[RotatingFileHandler] = RotatingFileHandler(
     backupCount=9,  # Max 100MB of logs
     encoding="utf8",
 )
-_STREAM_HANDLER.setLevel(_PRIMAITE_CONFIG["logging"]["log_level"])
+_STREAM_HANDLER.setLevel(PRIMAITE_CONFIG["logging"]["log_level"])
-_FILE_HANDLER.setLevel(_PRIMAITE_CONFIG["logging"]["log_level"])
+_FILE_HANDLER.setLevel(PRIMAITE_CONFIG["logging"]["log_level"])
 
-_LOG_FORMAT_STR: Final[str] = _PRIMAITE_CONFIG["logging"]["logger_format"]
+_LOG_FORMAT_STR: Final[str] = PRIMAITE_CONFIG["logging"]["logger_format"]
 _STREAM_HANDLER.setFormatter(_LEVEL_FORMATTER)
 _FILE_HANDLER.setFormatter(_LEVEL_FORMATTER)
 
@@ -215,6 +200,6 @@ def getLogger(name: str) -> Logger:  # noqa
         logging config.
     """
     logger = logging.getLogger(name)
-    logger.setLevel(_PRIMAITE_CONFIG["log_level"])
+    logger.setLevel(PRIMAITE_CONFIG["logging"]["log_level"])
 
     return logger

src/primaite/cli.py

@@ -2,16 +2,21 @@
 """Provides a CLI using Typer as an entry point."""
 import logging
 import os
+import shutil
 from enum import Enum
+from pathlib import Path
 from typing import Optional
 
+import pkg_resources
 import typer
 import yaml
 from typing_extensions import Annotated
 
 from primaite import PRIMAITE_PATHS
+from primaite.utils.cli import dev_cli
 
 app = typer.Typer(no_args_is_help=True)
+app.add_typer(dev_cli.dev, name="dev-mode")
 
 
 @app.command()
@@ -89,7 +94,7 @@ def version() -> None:
 
 
 @app.command()
-def setup(overwrite_existing: bool = True) -> None:
+def setup(overwrite_existing: bool = False) -> None:
     """
     Perform the PrimAITE first-time setup.
 
@@ -102,11 +107,14 @@ def setup(overwrite_existing: bool = True) -> None:
 
     _LOGGER.info("Performing the PrimAITE first-time setup...")
 
-    _LOGGER.info("Building primaite_config.yaml...")
-
     _LOGGER.info("Building the PrimAITE app directories...")
     PRIMAITE_PATHS.mkdirs()
 
+    _LOGGER.info("Building primaite_config.yaml...")
+    if overwrite_existing:
+        pkg_config_path = Path(pkg_resources.resource_filename("primaite", "setup/_package_data/primaite_config.yaml"))
+        shutil.copy(pkg_config_path, PRIMAITE_PATHS.app_config_file_path)
+
     _LOGGER.info("Rebuilding the demo notebooks...")
     reset_demo_notebooks.run(overwrite_existing=True)
 
@@ -114,47 +122,3 @@ def setup(overwrite_existing: bool = True) -> None:
     reset_example_configs.run(overwrite_existing=True)
 
     _LOGGER.info("PrimAITE setup complete!")
-
-
-@app.command()
-def mode(
-    dev: Annotated[bool, typer.Option("--dev", help="Activates PrimAITE developer mode")] = None,
-    prod: Annotated[bool, typer.Option("--prod", help="Activates PrimAITE production mode")] = None,
-) -> None:
-    """
-    Switch PrimAITE between developer mode and production mode.
-
-    By default, PrimAITE will be in production mode.
-
-    To view the current mode, use: primaite mode
-
-    To set to development mode, use: primaite mode --dev
-
-    To return to production mode, use: primaite mode --prod
-    """
-    if PRIMAITE_PATHS.app_config_file_path.exists():
-        with open(PRIMAITE_PATHS.app_config_file_path, "r") as file:
-            primaite_config = yaml.safe_load(file)
-            if dev and prod:
-                print("Unable to activate developer and production modes concurrently.")
-                return
-
-            if (dev is None) and (prod is None):
-                is_dev_mode = primaite_config["developer_mode"]
-
-                if is_dev_mode:
-                    print("PrimAITE is running in developer mode.")
-                else:
-                    print("PrimAITE is running in production mode.")
-            if dev:
-                # activate dev mode
-                primaite_config["developer_mode"] = True
-                with open(PRIMAITE_PATHS.app_config_file_path, "w") as file:
-                    yaml.dump(primaite_config, file)
-                print("PrimAITE is running in developer mode.")
-            if prod:
-                # activate prod mode
-                primaite_config["developer_mode"] = False
-                with open(PRIMAITE_PATHS.app_config_file_path, "w") as file:
-                    yaml.dump(primaite_config, file)
-                print("PrimAITE is running in production mode.")

src/primaite/session/io.py

@@ -5,9 +5,9 @@ from typing import Dict, List, Optional
 
 from pydantic import BaseModel, ConfigDict
 
-from primaite import getLogger, PRIMAITE_PATHS
+from primaite import _PRIMAITE_ROOT, getLogger, PRIMAITE_CONFIG, PRIMAITE_PATHS
 from primaite.simulator import LogLevel, SIM_OUTPUT
-from primaite.utils.primaite_config_utils import is_dev_mode
+from primaite.utils.cli.primaite_config_utils import is_dev_mode
 
 _LOGGER = getLogger(__name__)
 
@@ -62,12 +62,15 @@ class PrimaiteIO:
         date_str = timestamp.strftime("%Y-%m-%d")
         time_str = timestamp.strftime("%H-%M-%S")
 
+        session_path = PRIMAITE_PATHS.user_sessions_path / date_str / time_str
+
         # check if running in dev mode
         if is_dev_mode():
-            # if dev mode, simulation output will be the current working directory
+            session_path = _PRIMAITE_ROOT.parent.parent / "sessions" / date_str / time_str
-            session_path = Path.cwd() / "simulation_output" / date_str / time_str
+
-        else:
+            # check if there is an output directory set in config
-            session_path = PRIMAITE_PATHS.user_sessions_path / date_str / time_str
+            if PRIMAITE_CONFIG["developer_mode"]["output_dir"]:
+                session_path = Path(PRIMAITE_CONFIG["developer_mode"]["output_dir"]) / "sessions" / date_str / time_str
 
         session_path.mkdir(exist_ok=True, parents=True)
         return session_path

src/primaite/setup/_package_data/primaite_config.yaml

@@ -1,6 +1,12 @@
 # The main PrimAITE application config file
 
-developer_mode: False  # false by default
+developer_mode:
+  enabled: False  # not enabled by default
+  sys_log_level: DEBUG # level of output for system logs, DEBUG by default
+  output_sys_logs: False  # system logs not output by default
+  output_pcap_logs: False  # pcap logs not output by default
+  output_to_terminal: False  # do not output to terminal by default
+  output_dir: null  # none by default - none will print to repository root
 
 # Logging
 logging:

src/primaite/simulator/__init__.py

@@ -3,10 +3,12 @@ from datetime import datetime
 from enum import IntEnum
 from pathlib import Path
 
-from primaite import _PRIMAITE_ROOT
+from primaite import _PRIMAITE_ROOT, PRIMAITE_CONFIG, PRIMAITE_PATHS
 
 __all__ = ["SIM_OUTPUT"]
 
+from primaite.utils.cli.primaite_config_utils import is_dev_mode
+
 
 class LogLevel(IntEnum):
     """Enum containing all the available log levels for PrimAITE simulation output."""
@@ -25,16 +27,34 @@ class LogLevel(IntEnum):
 
 class _SimOutput:
     def __init__(self):
-        self._path: Path = (
+        date_str = datetime.now().strftime("%Y-%m-%d")
-            _PRIMAITE_ROOT.parent.parent / "simulation_output" / datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+        time_str = datetime.now().strftime("%H-%M-%S")
-        )
+
-        self.save_pcap_logs: bool = False
+        path = PRIMAITE_PATHS.user_sessions_path / date_str / time_str
-        self.save_sys_logs: bool = False
+
-        self.write_sys_log_to_terminal: bool = False
+        self._path = path
-        self.sys_log_level: LogLevel = LogLevel.WARNING  # default log level is at WARNING
+        self._save_pcap_logs: bool = False
+        self._save_sys_logs: bool = False
+        self._write_sys_log_to_terminal: bool = False
+        self._sys_log_level: LogLevel = LogLevel.WARNING  # default log level is at WARNING
 
     @property
     def path(self) -> Path:
+        if is_dev_mode():
+            date_str = datetime.now().strftime("%Y-%m-%d")
+            time_str = datetime.now().strftime("%H-%M-%S")
+            # if dev mode is enabled, if output dir is not set, print to primaite repo root
+            path: Path = _PRIMAITE_ROOT.parent.parent / "sessions" / date_str / time_str / "simulation_output"
+            # otherwise print to output dir
+            if PRIMAITE_CONFIG["developer_mode"]["output_dir"]:
+                path: Path = (
+                    Path(PRIMAITE_CONFIG["developer_mode"]["output_dir"])
+                    / "sessions"
+                    / date_str
+                    / time_str
+                    / "simulation_output"
+                )
+            self._path = path
         return self._path
 
     @path.setter
@@ -42,5 +62,45 @@ class _SimOutput:
         self._path = new_path
         self._path.mkdir(exist_ok=True, parents=True)
 
+    @property
+    def save_pcap_logs(self) -> bool:
+        if is_dev_mode():
+            return PRIMAITE_CONFIG.get("developer_mode").get("output_pcap_logs")
+        return self._save_pcap_logs
+
+    @save_pcap_logs.setter
+    def save_pcap_logs(self, save_pcap_logs: bool) -> None:
+        self._save_pcap_logs = save_pcap_logs
+
+    @property
+    def save_sys_logs(self) -> bool:
+        if is_dev_mode():
+            return PRIMAITE_CONFIG.get("developer_mode").get("output_sys_logs")
+        return self._save_sys_logs
+
+    @save_sys_logs.setter
+    def save_sys_logs(self, save_sys_logs: bool) -> None:
+        self._save_sys_logs = save_sys_logs
+
+    @property
+    def write_sys_log_to_terminal(self) -> bool:
+        if is_dev_mode():
+            return PRIMAITE_CONFIG.get("developer_mode").get("output_to_terminal")
+        return self._write_sys_log_to_terminal
+
+    @write_sys_log_to_terminal.setter
+    def write_sys_log_to_terminal(self, write_sys_log_to_terminal: bool) -> None:
+        self._write_sys_log_to_terminal = write_sys_log_to_terminal
+
+    @property
+    def sys_log_level(self) -> LogLevel:
+        if is_dev_mode():
+            return LogLevel[PRIMAITE_CONFIG.get("developer_mode").get("sys_log_level")]
+        return self._sys_log_level
+
+    @sys_log_level.setter
+    def sys_log_level(self, sys_log_level: LogLevel) -> None:
+        self._sys_log_level = sys_log_level
+
 
 SIM_OUTPUT = _SimOutput()

src/primaite/utils/cli/dev_cli.py

@@ -0,0 +1,171 @@
+import click
+import typer
+from rich import print
+from rich.table import Table
+from typing_extensions import Annotated
+
+from primaite import _PRIMAITE_ROOT, PRIMAITE_CONFIG
+from primaite.simulator import LogLevel
+from primaite.utils.cli.primaite_config_utils import is_dev_mode, update_primaite_application_config
+
+dev = typer.Typer()
+
+PRODUCTION_MODE_MESSAGE = (
+    "\n[green]:rocket::rocket::rocket: "
+    " PrimAITE is running in Production mode "
+    " :rocket::rocket::rocket: [/green]\n"
+)
+
+DEVELOPER_MODE_MESSAGE = (
+    "\n[yellow] :construction::construction::construction: "
+    " PrimAITE is running in Development mode "
+    " :construction::construction::construction: [/yellow]\n"
+)
+
+
+def dev_mode():
+    """
+    CLI commands relevant to the dev-mode for PrimAITE.
+
+    The dev-mode contains tools that help with the ease of developing or debugging PrimAITE.
+
+    By default, PrimAITE will be in production mode.
+
+    To enable development mode, use `primaite dev-mode enable`
+    """
+
+
+@dev.command()
+def show():
+    """Show if PrimAITE is in development mode or production mode."""
+    # print if dev mode is enabled
+    print(DEVELOPER_MODE_MESSAGE if is_dev_mode() else PRODUCTION_MODE_MESSAGE)
+
+    table = Table(title="Current Dev-Mode Settings")
+    table.add_column("Setting", style="cyan")
+    table.add_column("Value", style="default")
+    for setting, value in PRIMAITE_CONFIG["developer_mode"].items():
+        table.add_row(setting, str(value))
+
+    print(table)
+    print("\nTo see available options, use [cyan]`primaite dev-mode --help`[/cyan]\n")
+
+
+@dev.command()
+def enable():
+    """Enable the development mode for PrimAITE."""
+    # enable dev mode
+    PRIMAITE_CONFIG["developer_mode"]["enabled"] = True
+    update_primaite_application_config()
+    print(DEVELOPER_MODE_MESSAGE)
+
+
+@dev.command()
+def disable():
+    """Disable the development mode for PrimAITE."""
+    # disable dev mode
+    PRIMAITE_CONFIG["developer_mode"]["enabled"] = False
+    update_primaite_application_config()
+    print(PRODUCTION_MODE_MESSAGE)
+
+
+def config_callback(
+    ctx: typer.Context,
+    sys_log_level: Annotated[
+        LogLevel,
+        typer.Option(
+            "--sys-log-level",
+            "-level",
+            click_type=click.Choice(LogLevel._member_names_, case_sensitive=False),
+            help="The level of system logs to output.",
+            show_default=False,
+        ),
+    ] = None,
+    output_sys_logs: Annotated[
+        bool,
+        typer.Option(
+            "--output-sys-logs/--no-sys-logs", "-sys/-nsys", help="Output system logs to file.", show_default=False
+        ),
+    ] = None,
+    output_pcap_logs: Annotated[
+        bool,
+        typer.Option(
+            "--output-pcap-logs/--no-pcap-logs",
+            "-pcap/-npcap",
+            help="Output network packet capture logs to file.",
+            show_default=False,
+        ),
+    ] = None,
+    output_to_terminal: Annotated[
+        bool,
+        typer.Option(
+            "--output-to-terminal/--no-terminal", "-t/-nt", help="Output system logs to terminal.", show_default=False
+        ),
+    ] = None,
+):
+    """Configure the development tools and environment."""
+    if ctx.params.get("sys_log_level") is not None:
+        PRIMAITE_CONFIG["developer_mode"]["sys_log_level"] = ctx.params.get("sys_log_level")
+        print(f"PrimAITE dev-mode config updated sys_log_level={ctx.params.get('sys_log_level')}")
+
+    if output_sys_logs is not None:
+        PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"] = output_sys_logs
+        print(f"PrimAITE dev-mode config updated {output_sys_logs=}")
+
+    if output_pcap_logs is not None:
+        PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"] = output_pcap_logs
+        print(f"PrimAITE dev-mode config updated {output_pcap_logs=}")
+
+    if output_to_terminal is not None:
+        PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"] = output_to_terminal
+        print(f"PrimAITE dev-mode config updated {output_to_terminal=}")
+
+    # update application config
+    update_primaite_application_config()
+
+
+config_typer = typer.Typer(
+    callback=config_callback,
+    name="config",
+    no_args_is_help=True,
+    invoke_without_command=True,
+)
+dev.add_typer(config_typer)
+
+
+@config_typer.command()
+def path(
+    directory: Annotated[
+        str,
+        typer.Argument(
+            help="Directory where the system logs and PCAP logs will be output. By default, this will be where the"
+            "root of the PrimAITE repository is located.",
+            show_default=False,
+        ),
+    ] = None,
+    default: Annotated[
+        bool,
+        typer.Option(
+            "--default",
+            "-root",
+            help="Set PrimAITE to output system logs and pcap logs to the PrimAITE repository root.",
+        ),
+    ] = None,
+):
+    """Set the output directory for the PrimAITE system and PCAP logs."""
+    if default:
+        PRIMAITE_CONFIG["developer_mode"]["output_dir"] = None
+        # update application config
+        update_primaite_application_config()
+        print(
+            f"PrimAITE dev-mode output_dir [cyan]"
+            f"{str(_PRIMAITE_ROOT.parent.parent / 'simulation_output')}"
+            f"[/cyan]"
+        )
+        return
+
+    if directory:
+        PRIMAITE_CONFIG["developer_mode"]["output_dir"] = directory
+        # update application config
+        update_primaite_application_config()
+        print(f"PrimAITE dev-mode output_dir [cyan]{directory}[/cyan]")

src/primaite/utils/cli/primaite_config_utils.py

@@ -0,0 +1,22 @@
+from typing import Dict, Optional
+
+import yaml
+
+from primaite import PRIMAITE_CONFIG, PRIMAITE_PATHS
+
+
+def is_dev_mode() -> bool:
+    """Returns True if PrimAITE is currently running in developer mode."""
+    return PRIMAITE_CONFIG.get("developer_mode", {}).get("enabled", False)
+
+
+def update_primaite_application_config(config: Optional[Dict] = None) -> None:
+    """
+    Update the PrimAITE application config file.
+
+    :params: config: Leave empty so that PRIMAITE_CONFIG is used - otherwise provide the Dict
+    """
+    with open(PRIMAITE_PATHS.app_config_file_path, "w") as file:
+        if not config:
+            config = PRIMAITE_CONFIG
+        yaml.dump(config, file)

src/primaite/utils/primaite_config_utils.py

@@ -1,11 +0,0 @@
-import yaml
-
-from primaite import PRIMAITE_PATHS
-
-
-def is_dev_mode() -> bool:
-    """Returns True if PrimAITE is currently running in developer mode."""
-    if PRIMAITE_PATHS.app_config_file_path.exists():
-        with open(PRIMAITE_PATHS.app_config_file_path, "r") as file:
-            primaite_config = yaml.safe_load(file)
-            return primaite_config["developer_mode"]

tests/conftest.py

@@ -1,19 +1,14 @@
 # © Crown-owned copyright 2023, Defence Science and Technology Laboratory UK
-from datetime import datetime
+from typing import Any, Dict, Tuple
-from pathlib import Path
-from typing import Any, Dict, Optional, Tuple, Union
 
 import pytest
-import yaml
-from _pytest.monkeypatch import MonkeyPatch
 
-from primaite import getLogger, PRIMAITE_PATHS
+from primaite import getLogger
 from primaite.game.agent.actions import ActionManager
 from primaite.game.agent.interface import AbstractAgent
 from primaite.game.agent.observations.observation_manager import NestedObservation, ObservationManager
 from primaite.game.agent.rewards import RewardFunction
 from primaite.game.game import PrimaiteGame
-from primaite.simulator import SIM_OUTPUT
 from primaite.simulator.file_system.file_system import FileSystem
 from primaite.simulator.network.container import Network
 from primaite.simulator.network.hardware.nodes.host.computer import Computer
@@ -32,7 +27,6 @@ from primaite.simulator.system.services.dns.dns_server import DNSServer
 from primaite.simulator.system.services.service import Service
 from primaite.simulator.system.services.web_server.web_server import WebServer
 from tests import TEST_ASSETS_ROOT
-from tests.mock_and_patch.get_session_path_mock import temp_user_sessions_path
 
 ACTION_SPACE_NODE_VALUES = 1
 ACTION_SPACE_NODE_ACTION_VALUES = 1
@@ -40,21 +34,6 @@ ACTION_SPACE_NODE_ACTION_VALUES = 1
 _LOGGER = getLogger(__name__)
 
 
-@pytest.fixture(scope="function", autouse=True)
-def set_syslog_output_to_true():
-    """Will be run before each test."""
-    monkeypatch = MonkeyPatch()
-    monkeypatch.setattr(
-        SIM_OUTPUT,
-        "path",
-        Path(TEST_ASSETS_ROOT.parent.parent / "simulation_output" / datetime.now().strftime("%Y-%m-%d_%H-%M-%S")),
-    )
-    monkeypatch.setattr(SIM_OUTPUT, "save_pcap_logs", False)
-    monkeypatch.setattr(SIM_OUTPUT, "save_sys_logs", False)
-
-    yield
-
-
 class TestService(Service):
     """Test Service class"""
 

tests/e2e_integration_tests/environments/test_rllib_single_agent_environment.py

@@ -38,3 +38,5 @@ def test_rllib_single_agent_compatibility():
     save_file = Path(tempfile.gettempdir()) / "ray/"
     algo.save(save_file)
     assert save_file.exists()
+
+    save_file.unlink()  # clean up

tests/e2e_integration_tests/environments/test_sb3_environment.py

@@ -25,3 +25,4 @@ def test_sb3_compatibility():
     model.save(save_path)
 
     assert (save_path).exists()
+    save_path.unlink()  # clean up

tests/integration_tests/cli/__init__.py

@@ -0,0 +1,11 @@
+from typing import List
+
+from typer.testing import CliRunner, Result
+
+from primaite.cli import app
+
+
+def cli(args: List[str]) -> Result:
+    """Pass in a list of arguments and it will return the result."""
+    runner = CliRunner()
+    return runner.invoke(app, args)

tests/integration_tests/cli/test_dev_cli.py

@@ -0,0 +1,171 @@
+import os
+import shutil
+import tempfile
+from pathlib import Path
+
+import pkg_resources
+import pytest
+import yaml
+
+from primaite import PRIMAITE_CONFIG
+from primaite.utils.cli.primaite_config_utils import update_primaite_application_config
+from tests.integration_tests.cli import cli
+
+
+@pytest.fixture(autouse=True)
+def test_setup():
+    """
+    Setup this test by using the default primaite app config in package
+    """
+    global PRIMAITE_CONFIG
+    current_config = PRIMAITE_CONFIG.copy()  # store the config before test
+
+    pkg_config_path = Path(pkg_resources.resource_filename("primaite", "setup/_package_data/primaite_config.yaml"))
+
+    with open(pkg_config_path, "r") as file:
+        # load from config
+        config_dict = yaml.safe_load(file)
+
+    PRIMAITE_CONFIG["developer_mode"] = config_dict["developer_mode"]
+
+    yield
+
+    PRIMAITE_CONFIG["developer_mode"] = current_config["developer_mode"]  # restore config to prevent being yelled at
+    update_primaite_application_config(config=PRIMAITE_CONFIG)
+
+
+def test_dev_mode_enable_disable():
+    """Test dev mode enable and disable."""
+    # check defaults
+    assert PRIMAITE_CONFIG["developer_mode"]["enabled"] is False  # not enabled by default
+
+    result = cli(["dev-mode", "show"])
+    assert "Production" in result.output  # should print that it is in Production mode by default
+
+    result = cli(["dev-mode", "enable"])
+
+    assert "Development" in result.output  # should print that it is in Development mode
+
+    assert PRIMAITE_CONFIG["developer_mode"]["enabled"]  # config should reflect that dev mode is enabled
+
+    result = cli(["dev-mode", "show"])
+    assert "Development" in result.output  # should print that it is in Development mode
+
+    result = cli(["dev-mode", "disable"])
+
+    assert "Production" in result.output  # should print that it is in Production mode
+
+    assert PRIMAITE_CONFIG["developer_mode"]["enabled"] is False  # config should reflect that dev mode is disabled
+
+    result = cli(["dev-mode", "show"])
+    assert "Production" in result.output  # should print that it is in Production mode
+
+
+def test_dev_mode_config_sys_log_level():
+    """Check that the system log level can be changed via CLI."""
+    # check defaults
+    assert PRIMAITE_CONFIG["developer_mode"]["sys_log_level"] == "DEBUG"  # DEBUG by default
+
+    result = cli(["dev-mode", "config", "-level", "WARNING"])
+
+    assert "sys_log_level=WARNING" in result.output  # should print correct value
+
+    # config should reflect that log level is WARNING
+    assert PRIMAITE_CONFIG["developer_mode"]["sys_log_level"] == "WARNING"
+
+    result = cli(["dev-mode", "config", "--sys-log-level", "INFO"])
+
+    assert "sys_log_level=INFO" in result.output  # should print correct value
+
+    # config should reflect that log level is WARNING
+    assert PRIMAITE_CONFIG["developer_mode"]["sys_log_level"] == "INFO"
+
+
+def test_dev_mode_config_sys_logs_enable_disable():
+    """Test that the system logs output can be enabled or disabled."""
+    # check defaults
+    assert PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"] is False  # False by default
+
+    result = cli(["dev-mode", "config", "--output-sys-logs"])
+    assert "output_sys_logs=True" in result.output  # should print correct value
+
+    # config should reflect that output_sys_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"]
+
+    result = cli(["dev-mode", "config", "--no-sys-logs"])
+    assert "output_sys_logs=False" in result.output  # should print correct value
+
+    # config should reflect that output_sys_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"] is False
+
+    result = cli(["dev-mode", "config", "-sys"])
+    assert "output_sys_logs=True" in result.output  # should print correct value
+
+    # config should reflect that output_sys_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"]
+
+    result = cli(["dev-mode", "config", "-nsys"])
+    assert "output_sys_logs=False" in result.output  # should print correct value
+
+    # config should reflect that output_sys_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_sys_logs"] is False
+
+
+def test_dev_mode_config_pcap_logs_enable_disable():
+    """Test that the pcap logs output can be enabled or disabled."""
+    # check defaults
+    assert PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"] is False  # False by default
+
+    result = cli(["dev-mode", "config", "--output-pcap-logs"])
+    assert "output_pcap_logs=True" in result.output  # should print correct value
+
+    # config should reflect that output_pcap_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"]
+
+    result = cli(["dev-mode", "config", "--no-pcap-logs"])
+    assert "output_pcap_logs=False" in result.output  # should print correct value
+
+    # config should reflect that output_pcap_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"] is False
+
+    result = cli(["dev-mode", "config", "-pcap"])
+    assert "output_pcap_logs=True" in result.output  # should print correct value
+
+    # config should reflect that output_pcap_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"]
+
+    result = cli(["dev-mode", "config", "-npcap"])
+    assert "output_pcap_logs=False" in result.output  # should print correct value
+
+    # config should reflect that output_pcap_logs is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_pcap_logs"] is False
+
+
+def test_dev_mode_config_output_to_terminal_enable_disable():
+    """Test that the output to terminal can be enabled or disabled."""
+    # check defaults
+    assert PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"] is False  # False by default
+
+    result = cli(["dev-mode", "config", "--output-to-terminal"])
+    assert "output_to_terminal=True" in result.output  # should print correct value
+
+    # config should reflect that output_to_terminal is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"]
+
+    result = cli(["dev-mode", "config", "--no-terminal"])
+    assert "output_to_terminal=False" in result.output  # should print correct value
+
+    # config should reflect that output_to_terminal is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"] is False
+
+    result = cli(["dev-mode", "config", "-t"])
+    assert "output_to_terminal=True" in result.output  # should print correct value
+
+    # config should reflect that output_to_terminal is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"]
+
+    result = cli(["dev-mode", "config", "-nt"])
+    assert "output_to_terminal=False" in result.output  # should print correct value
+
+    # config should reflect that output_to_terminal is True
+    assert PRIMAITE_CONFIG["developer_mode"]["output_to_terminal"] is False

tests/unit_tests/_primaite/_simulator/_system/core/test_sys_log.py

@@ -2,10 +2,20 @@ from uuid import uuid4
 
 import pytest
 
+from primaite import PRIMAITE_CONFIG
 from primaite.simulator import LogLevel, SIM_OUTPUT
 from primaite.simulator.system.core.sys_log import SysLog
 
 
+@pytest.fixture(autouse=True)
+def override_dev_mode_temporarily():
+    """Temporarily turn off dev mode for this test."""
+    primaite_dev_mode = PRIMAITE_CONFIG["developer_mode"]["enabled"]
+    PRIMAITE_CONFIG["developer_mode"]["enabled"] = False
+    yield  # run tests
+    PRIMAITE_CONFIG["developer_mode"]["enabled"] = primaite_dev_mode
+
+
 @pytest.fixture(scope="function")
 def syslog() -> SysLog:
     return SysLog(hostname="test")