oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

#915 - Refactored documentation and included APi docs, dependencies.

Browse Source 
- make files now re-build autosummary and deps file.
- Added typer and platformdirs to deps in pyproject.toml.
- Made root_is_pure = True in setup.py as platform/python specific wheels don't need to be built but the option is there should we need to.
-
Added an e2e test for primaite.main.run func.
This commit is contained in:
Chris McCarthy
2023-06-08 15:57:38 +01:00
parent 92334133c9
commit 5558750f13
26 changed files with 923 additions and 261 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
docs/source/custom_agent.rst Normal file
View File

@@ -0,0 +1,65 @@
Custom Agents
=============
**Integrating a user defined blue agent**
Integrating a blue agent with PrimAITE requires some modification of the code within the main.py file. The main.py file
consists of a number of functions, each of which will invoke training for a particular agent. These are:
* Generic (run_generic)
* Stable Baselines 3 PPO (:func:`~primaite.main.run_stable_baselines3_ppo)
* Stable Baselines 3 A2C (:func:`~primaite.main.run_stable_baselines3_a2c)
The selection of which agent type to use is made via the training config file. In order to train a user generated agent,
the run_generic function should be selected, and should be modified (typically) to be:
.. code:: python
agent = MyAgent(environment, num_steps)
for episode in range(0, num_episodes):
agent.learn()
env.close()
save_agent(agent)
Where:
* *MyAgent* is the user created agent
* *environment* is the :class:`~primaite.environment.primaite_env.Primaite` environment
* *num_episodes* is the number of episodes in the session, as defined in the training config file
* *num_steps* is the number of steps in an episode, as defined in the training config file
* the *.learn()* function should be defined in the user created agent
* the *env.close()* function is defined within PrimAITE
* the *save_agent()* assumes that a *save()* function has been defined in the user created agent. If not, this line can
be ommitted (although it is encouraged, since it will allow the agent to be saved and ported)
The code below provides a suggested format for the learn() function within the user created agent.
It's important to include the *self.environment.reset()* call within the episode loop in order that the
environment is reset between episodes. Note that the example below should not be considered exhaustive.
.. code:: python
def learn(self) :
# pre-reqs
# reset the environment
self.environment.reset()
done = False
for step in range(max_steps):
# calculate the action
action = ...
# execute the environment step
new_state, reward, done, info = self.environment.step(action)
# algorithm updates
...
# update to our new state
state = new_state
# if done, finish episode
if done == True:
break

28
docs/source/dependencies.rst
View File

@@ -1,24 +1,10 @@
.. _dependencies:
.. role:: raw-html(raw)
:format: html
Dependencies
============
PrimAITE Dependencies
=====================
---------------------
PrimAITE is built with the following versions of dependencies:
* Python 3.10.9
* PyYAML 6.0
* numpy 1.23.5
* networkx 2.8.8
* gym 0.21.0
* matplotlib 3.6.2
* stable_baselines_3 1.6.2
The latest release of PrimAITE has been tested against the following versions of dependencies:
* Python 3.10.9
* PyYAML 6.0
* numpy 1.23.5
* networkx 2.8.8
* gym 0.21.0
* matplotlib 3.6.2
* stable_baselines_3 1.6.2
.. include:: primaite-dependencies.rst

385
docs/source/primaite-dependencies.rst Normal file
View File

@@ -0,0 +1,385 @@
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Name | Version | License | URL |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Babel | 2.12.1 | BSD License | https://babel.pocoo.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| GPUtil | 1.4.0 | MIT | https://github.com/anderskm/gputil |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Gymnasium | 0.26.3 | MIT | https://gymnasium.farama.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Jinja2 | 3.1.2 | BSD License | https://palletsprojects.com/p/jinja/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Markdown | 3.4.3 | BSD License | https://Python-Markdown.github.io/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| MarkupSafe | 2.1.2 | BSD License | https://palletsprojects.com/p/markupsafe/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Pillow | 9.5.0 | Historical Permission Notice and Disclaimer (HPND) | https://python-pillow.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| PyWavelets | 1.4.1 | MIT License | https://github.com/PyWavelets/pywt |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| PyYAML | 6.0 | MIT License | https://pyyaml.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Pygments | 2.15.1 | BSD License | https://pygments.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Send2Trash | 1.8.2 | BSD License | https://github.com/arsenetar/send2trash |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Sphinx | 6.1.3 | BSD License | https://www.sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| Werkzeug | 2.3.4 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| absl-py | 1.4.0 | Apache Software License | https://github.com/abseil/abseil-py |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| aiofiles | 22.1.0 | Apache Software License | https://github.com/Tinche/aiofiles |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| aiosignal | 1.3.1 | Apache Software License | https://github.com/aio-libs/aiosignal |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| aiosqlite | 0.19.0 | MIT License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| alabaster | 0.7.13 | BSD License | https://alabaster.readthedocs.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| anyio | 3.7.0 | MIT License | https://anyio.readthedocs.io/en/stable/versionhistory.html |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| argon2-cffi | 21.3.0 | MIT License | https://github.com/hynek/argon2-cffi/blob/main/CHANGELOG.md |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| argon2-cffi-bindings | 21.2.0 | MIT License | https://github.com/hynek/argon2-cffi-bindings |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| arrow | 1.2.3 | Apache Software License | https://arrow.readthedocs.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| asttokens | 2.2.1 | Apache 2.0 | https://github.com/gristlabs/asttokens |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| attrs | 23.1.0 | MIT License | https://www.attrs.org/en/stable/changelog.html |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| backcall | 0.2.0 | BSD License | https://github.com/takluyver/backcall |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| beautifulsoup4 | 4.12.2 | MIT License | https://www.crummy.com/software/BeautifulSoup/bs4/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| bleach | 6.0.0 | Apache Software License | https://github.com/mozilla/bleach |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| build | 0.10.0 | MIT License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| cachetools | 5.3.0 | MIT License | https://github.com/tkem/cachetools/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| certifi | 2023.5.7 | Mozilla Public License 2.0 (MPL 2.0) | https://github.com/certifi/python-certifi |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| cffi | 1.15.1 | MIT License | http://cffi.readthedocs.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| cfgv | 3.3.1 | MIT License | https://github.com/asottile/cfgv |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| charset-normalizer | 3.1.0 | MIT License | https://github.com/Ousret/charset_normalizer |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| click | 8.1.3 | BSD License | https://palletsprojects.com/p/click/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| cloudpickle | 2.2.1 | BSD License | https://github.com/cloudpipe/cloudpickle |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| colorama | 0.4.6 | BSD License | https://github.com/tartley/colorama |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| comm | 0.1.3 | BSD License | https://github.com/ipython/comm |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| contourpy | 1.0.7 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| coverage | 7.2.6 | Apache Software License | https://github.com/nedbat/coveragepy |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| cycler | 0.11.0 | BSD License | https://github.com/matplotlib/cycler |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| debugpy | 1.6.7 | Eclipse Public License 2.0 (EPL-2.0); MIT License | https://aka.ms/debugpy |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| decorator | 5.1.1 | BSD License | https://github.com/micheles/decorator |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| defusedxml | 0.7.1 | Python Software Foundation License | https://github.com/tiran/defusedxml |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| distlib | 0.3.6 | Python Software Foundation License | https://github.com/pypa/distlib |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| dm-tree | 0.1.8 | Apache Software License | https://github.com/deepmind/tree |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| docutils | 0.19 | BSD License; GNU General Public License (GPL); Public Domain; Python Software Foundation License | https://docutils.sourceforge.io/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| exceptiongroup | 1.1.1 | MIT License | https://github.com/agronholm/exceptiongroup/blob/main/CHANGES.rst |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| executing | 1.2.0 | MIT License | https://github.com/alexmojaki/executing |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| fastjsonschema | 2.17.1 | BSD License | https://github.com/horejsek/python-fastjsonschema |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| filelock | 3.12.0 | The Unlicense (Unlicense) | https://github.com/tox-dev/py-filelock |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| flake8 | 6.0.0 | MIT License | https://github.com/pycqa/flake8 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| fonttools | 4.39.4 | MIT License | http://github.com/fonttools/fonttools |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| fqdn | 1.5.1 | Mozilla Public License 2.0 (MPL 2.0) | https://github.com/ypcrts/fqdn |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| frozenlist | 1.3.3 | Apache Software License | https://github.com/aio-libs/frozenlist |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| furo | 2023.3.27 | MIT License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| google-auth | 2.19.0 | Apache Software License | https://github.com/googleapis/google-auth-library-python |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| google-auth-oauthlib | 1.0.0 | Apache Software License | https://github.com/GoogleCloudPlatform/google-auth-library-python-oauthlib |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| grpcio | 1.51.3 | Apache Software License | https://grpc.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| gym | 0.21.0 | UNKNOWN | https://github.com/openai/gym |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| gymnasium-notices | 0.0.1 | MIT License | https://github.com/Farama-Foundation/gym-notices |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| identify | 2.5.24 | MIT License | https://github.com/pre-commit/identify |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| idna | 3.4 | BSD License | https://github.com/kjd/idna |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| imageio | 2.29.0 | BSD License | https://github.com/imageio/imageio |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| imagesize | 1.4.1 | MIT License | https://github.com/shibukawa/imagesize_py |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| importlib-metadata | 4.13.0 | Apache Software License | https://github.com/python/importlib_metadata |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| iniconfig | 2.0.0 | MIT License | https://github.com/pytest-dev/iniconfig |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| ipykernel | 6.23.1 | BSD License | https://ipython.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| ipython | 8.13.2 | BSD License | https://ipython.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| ipython-genutils | 0.2.0 | BSD License | http://ipython.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| isoduration | 20.11.0 | ISC License (ISCL) | https://github.com/bolsote/isoduration |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jedi | 0.18.2 | MIT License | https://github.com/davidhalter/jedi |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| json5 | 0.9.14 | Apache Software License | https://github.com/dpranke/pyjson5 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jsonpointer | 2.3 | BSD License | https://github.com/stefankoegl/python-json-pointer |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jsonschema | 4.17.3 | MIT License | https://github.com/python-jsonschema/jsonschema |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter-events | 0.6.3 | BSD License | http://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter-ydoc | 0.2.4 | BSD 3-Clause License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_client | 8.2.0 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_core | 5.3.0 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_server | 2.6.0 | BSD License | https://jupyter-server.readthedocs.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_server_fileid | 0.9.0 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_server_terminals | 0.4.4 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyter_server_ydoc | 0.6.1 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyterlab | 3.6.1 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyterlab-pygments | 0.2.2 | BSD | https://github.com/jupyterlab/jupyterlab_pygments |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| jupyterlab_server | 2.22.1 | BSD License | https://jupyterlab-server.readthedocs.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| kiwisolver | 1.4.4 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| lazy_loader | 0.2 | BSD License | https://github.com/scientific-python/lazy_loader |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| lz4 | 4.3.2 | BSD License | https://github.com/python-lz4/python-lz4 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| markdown-it-py | 2.2.0 | MIT License | https://github.com/executablebooks/markdown-it-py |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| matplotlib | 3.7.1 | Python Software Foundation License | https://matplotlib.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| matplotlib-inline | 0.1.6 | BSD 3-Clause | https://github.com/ipython/matplotlib-inline |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| mccabe | 0.7.0 | MIT License | https://github.com/pycqa/mccabe |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| mdurl | 0.1.2 | MIT License | https://github.com/executablebooks/mdurl |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| mistune | 2.0.5 | BSD License | https://github.com/lepture/mistune |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| mock | 5.0.2 | BSD License | http://mock.readthedocs.org/en/latest/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| mpmath | 1.3.0 | BSD License | http://mpmath.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| msgpack | 1.0.5 | Apache Software License | https://msgpack.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nbclassic | 1.0.0 | BSD License | https://github.com/jupyter/nbclassic |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nbclient | 0.8.0 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nbconvert | 7.4.0 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nbformat | 5.9.0 | BSD License | https://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nest-asyncio | 1.5.6 | BSD License | https://github.com/erdewit/nest_asyncio |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| networkx | 3.1 | BSD License | https://networkx.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| nodeenv | 1.8.0 | BSD License | https://github.com/ekalinin/nodeenv |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| notebook | 6.5.4 | BSD License | http://jupyter.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| notebook_shim | 0.2.3 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| numpy | 1.23.5 | BSD License | https://www.numpy.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| oauthlib | 3.2.2 | BSD License | https://github.com/oauthlib/oauthlib |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| overrides | 7.3.1 | Apache License, Version 2.0 | https://github.com/mkorpela/overrides |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| packaging | 23.1 | Apache Software License; BSD License | https://github.com/pypa/packaging |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pandas | 2.0.1 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pandocfilters | 1.5.0 | BSD License | http://github.com/jgm/pandocfilters |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| parso | 0.8.3 | MIT License | https://github.com/davidhalter/parso |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pickleshare | 0.7.5 | MIT License | https://github.com/pickleshare/pickleshare |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| platformdirs | 3.5.1 | MIT License | https://github.com/platformdirs/platformdirs |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pluggy | 1.0.0 | MIT License | https://github.com/pytest-dev/pluggy |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pre-commit | 2.20.0 | MIT License | https://github.com/pre-commit/pre-commit |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| primaite | 1.2.1 | MIT License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| prometheus-client | 0.17.0 | Apache Software License | https://github.com/prometheus/client_python |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| prompt-toolkit | 3.0.38 | BSD License | https://github.com/prompt-toolkit/python-prompt-toolkit |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| protobuf | 3.20.3 | BSD-3-Clause | https://developers.google.com/protocol-buffers/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| psutil | 5.9.5 | BSD License | https://github.com/giampaolo/psutil |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pure-eval | 0.2.2 | MIT License | http://github.com/alexmojaki/pure_eval |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyasn1 | 0.5.0 | BSD License | https://github.com/pyasn1/pyasn1 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyasn1-modules | 0.3.0 | BSD License | https://github.com/pyasn1/pyasn1-modules |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pycodestyle | 2.10.0 | MIT License | https://pycodestyle.pycqa.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pycparser | 2.21 | BSD License | https://github.com/eliben/pycparser |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyflakes | 3.0.1 | MIT License | https://github.com/PyCQA/pyflakes |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyparsing | 3.0.9 | MIT License | https://github.com/pyparsing/pyparsing/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyproject_hooks | 1.0.0 | MIT License | https://github.com/pypa/pyproject-hooks |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyrsistent | 0.19.3 | MIT License | https://github.com/tobgu/pyrsistent/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pytest | 7.2.0 | MIT License | https://docs.pytest.org/en/latest/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pytest-cov | 4.0.0 | MIT License | https://github.com/pytest-dev/pytest-cov |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pytest-flake8 | 1.1.1 | BSD License | https://github.com/tholo/pytest-flake8 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| python-dateutil | 2.8.2 | Apache Software License; BSD License | https://github.com/dateutil/dateutil |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| python-json-logger | 2.0.7 | BSD License | http://github.com/madzak/python-json-logger |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pytz | 2023.3 | MIT License | http://pythonhosted.org/pytz |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pywin32 | 306 | Python Software Foundation License | https://github.com/mhammond/pywin32 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pywinpty | 2.0.10 | MIT | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| pyzmq | 25.1.0 | BSD License; GNU Library or Lesser General Public License (LGPL) | https://pyzmq.readthedocs.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| ray | 2.2.0 | Apache 2.0 | https://github.com/ray-project/ray |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| requests | 2.31.0 | Apache Software License | https://requests.readthedocs.io |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| requests-oauthlib | 1.3.1 | BSD License | https://github.com/requests/requests-oauthlib |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| rfc3339-validator | 0.1.4 | MIT License | https://github.com/naimetti/rfc3339-validator |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| rfc3986-validator | 0.1.1 | MIT License | https://github.com/naimetti/rfc3986-validator |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| rich | 13.3.5 | MIT License | https://github.com/Textualize/rich |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| rsa | 4.9 | Apache Software License | https://stuvel.eu/rsa |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| scikit-image | 0.20.0 | BSD License | https://scikit-image.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| scipy | 1.10.1 | BSD License | https://scipy.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| shellingham | 1.5.0.post1 | ISC License (ISCL) | https://github.com/sarugaku/shellingham |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| six | 1.16.0 | MIT License | https://github.com/benjaminp/six |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sniffio | 1.3.0 | Apache Software License; MIT License | https://github.com/python-trio/sniffio |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| snowballstemmer | 2.2.0 | BSD License | https://github.com/snowballstem/snowball |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| soupsieve | 2.4.1 | MIT License | https://github.com/facelessuser/soupsieve |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinx-basic-ng | 1.0.0b1 | MIT License | https://github.com/pradyunsg/sphinx-basic-ng |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinx-code-tabs | 0.5.3 | The Unlicense (Unlicense) | https://github.com/coldfix/sphinx-code-tabs |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinx-copybutton | 0.5.2 | MIT License | https://github.com/executablebooks/sphinx-copybutton |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-applehelp | 1.0.4 | BSD License | https://www.sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-devhelp | 1.0.2 | BSD License | http://sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-htmlhelp | 2.0.1 | BSD License | https://www.sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-jsmath | 1.0.1 | BSD License | http://sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-qthelp | 1.0.3 | BSD License | http://sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sphinxcontrib-serializinghtml | 1.1.5 | BSD License | http://sphinx-doc.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| stable-baselines3 | 1.6.2 | MIT | https://github.com/DLR-RM/stable-baselines3 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| stack-data | 0.6.2 | MIT License | http://github.com/alexmojaki/stack_data |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| sympy | 1.12 | BSD License | https://sympy.org |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tabulate | 0.9.0 | MIT License | https://github.com/astanin/python-tabulate |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tensorboard | 2.12.3 | Apache Software License | https://github.com/tensorflow/tensorboard |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tensorboard-data-server | 0.7.0 | Apache Software License | https://github.com/tensorflow/tensorboard/tree/master/tensorboard/data/server |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tensorboard-plugin-wit | 1.8.1 | Apache 2.0 | https://whatif-tool.dev |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tensorboardX | 2.6 | MIT License | https://github.com/lanpa/tensorboardX |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| terminado | 0.17.1 | BSD License | https://github.com/jupyter/terminado |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tifffile | 2023.4.12 | BSD License | https://www.cgohlke.com |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tinycss2 | 1.2.1 | BSD License | https://www.courtbouillon.org/tinycss2 |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| toml | 0.10.2 | MIT License | https://github.com/uiri/toml |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tomli | 2.0.1 | MIT License | https://github.com/hukkin/tomli |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| torch | 2.0.1 | BSD License | https://pytorch.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tornado | 6.3.2 | Apache Software License | http://www.tornadoweb.org/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| traitlets | 5.9.0 | BSD License | https://github.com/ipython/traitlets |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| typer | 0.9.0 | MIT License | https://github.com/tiangolo/typer |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| typing_extensions | 4.6.2 | Python Software Foundation License | https://github.com/python/typing_extensions/issues |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| tzdata | 2023.3 | Apache Software License | https://github.com/python/tzdata |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| uri-template | 1.2.0 | MIT License | https://github.com/plinss/uri_template/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| urllib3 | 1.26.16 | MIT License | https://urllib3.readthedocs.io/ |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| virtualenv | 20.21.0 | MIT License | https://github.com/pypa/virtualenv |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| webcolors | 1.13 | BSD License | UNKNOWN |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| webencodings | 0.5.1 | BSD License | https://github.com/SimonSapin/python-webencodings |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| websocket-client | 1.5.2 | Apache Software License | https://github.com/websocket-client/websocket-client.git |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| y-py | 0.5.9 | MIT License | https://github.com/y-crdt/ypy |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| ypy-websocket | 0.8.2 | UNKNOWN | https://github.com/y-crdt/ypy-websocket |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+
| zipp | 3.15.0 | MIT License | https://github.com/jaraco/zipp |
+-------------------------------+-------------+--------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------+

92
docs/source/primaite_session.rst Normal file
View File

@@ -0,0 +1,92 @@
Run a PrimAITE Session
======================
Run
---
A PrimAITE session can be ran either with the ``primaite session`` command from the cli
(See :func:`primaite.cli.session`), or by calling :func:`primaite.main.run` from a Python terminal or Jupyter Notebook.
Both the ``primaite session`` and :func:`primaite.main.run` take a training config and a lay down config as parameters.
.. tabs::
.. code-tab:: bash
:caption: Unix CLI
cd ~/primaite
source ./.venv/bin/activate
primaite session ./config/my_training_config.yaml ./config/my_lay_down_config.yaml
.. code-tab:: bash
:caption: Powershell CLI
cd ~\primaite
.\.venv\Scripts\activate
primaite session .\config\my_training_config.yaml .\config\my_lay_down_config.yaml
.. code-tab:: python
:caption: Python
from primaite.main import run
training_config = <path to training config yaml file>
lay_down_config = <path to lay down config yaml file>
run(training_config, lay_down_config)
When a session is ran, a session output sub-directory is created in the users app sessions directory (``~/primaite/sessions``).
The sub-directory is formatted as such: ``~/primaite/sessions/<yyyy-mm-dd>/<yyyy-mm-dd>_<hh-mm-dd>/``
For example, when running a session at 17:30:00 on 31st January 2023, the session will output to:
``~/primaite/sessions/2023-01-31/2023-01-31_17-30-00/``.
Outputs
-------
PrimAITE produces four types of outputs:
* Session Metadata
* Results
* Diagrams
* Saved agents
**Session Metadata**
PrimAITE creates a ``session_metadata.json`` file that contains the following metadata:
* **uuid** - The UUID assigned to the session upon instantiation.
* **start_datetime** - The date & time the session started in iso format.
* **end_datetime** - The date & time the session ended in iso format.
* **total_episodes** - The total number of training episodes completed.
* **total_time_steps** - The total number of training time steps completed.
* **env**
* **training_config**
* **All training config items**
* **lay_down_config**
* **All lay down config items**
**Results**
PrimAITE automatically creates two sets of results from each session:
* Average reward per episode - a csv file listing the average reward for each episode of the session. This provides, for example, an indication of the change over a training session of the reward value
* All transactions - a csv file listing the following values for every step of every episode:
* Timestamp
* Episode number
* Step number
* Initial observation space (before red and blue agent actions have been taken). Individual elements of the observation space are presented in the format OSI_X_Y
* Resulting observation space (after the red and blue agent actions have been taken) Individual elements of the observation space are presented in the format OSN_X_Y
* Reward value
* Action space (as presented by the blue agent on this step). Individual elements of the action space are presented in the format AS_X
**Diagrams**
For each session, PrimAITE automatically creates a visualisation of the system / network lay down configuration.
**Saved agents**
For each training session, assuming the agent being trained implements the *save()* function and this function is called by the code, PrimAITE automatically saves the agent state.

42
docs/source/results.rst
View File

@@ -1,42 +0,0 @@
.. _results:
Results, Output and Logging from PrimAITE
=========================================
PrimAITE produces four types of data:
* Outputs - Results
* Outputs - Diagrams
* Outputs - Saved agents
* Logging
Outputs can be found in the *[Install Directory]\\Primaite\\Primaite\\outputs* directory
Logging can be found in the *[Install Directory]\\Primaite\\Primaite\\logs* directory
**Outputs - Results**
PrimAITE automatically creates two sets of results from each session, and stores them in the *Results* folder:
* Average reward per episode - a csv file listing the average reward for each episode of the session. This provides, for example, an indication of the change over a training session of the reward value
* All transactions - a csv file listing the following values for every step of every episode:
* Timestamp
* Episode number
* Step number
* Initial observation space (before red and blue agent actions have been taken). Individual elements of the observation space are presented in the format OSI_X_Y
* Resulting observation space (after the red and blue agent actions have been taken) Individual elements of the observation space are presented in the format OSN_X_Y
* Reward value
* Action space (as presented by the blue agent on this step). Individual elements of the action space are presented in the format AS_X
**Outputs - Diagrams**
For each session, PrimAITE automatically creates a visualisation of the system / network laydown configuration, and stores it in the *Diagrams* folder.
**Outputs - Saved agents**
For each training session, assuming the agent being trained implements the *save()* function and this function is called by the code, PrimAITE automatically saves the agent state and stores it in the *agents* folder.
**Logging**
PrimAITE also provides output logs (for diagnosis) using the Python Logging package. These can be found in the *[Install Directory]\\Primaite\\Primaite\\logs* directory

86
docs/source/session.rst
View File

@@ -1,86 +0,0 @@
.. _session:
Running a PrimAITE Training or Evaluation Session
=================================================
The application will determine whether a Training or Evaluation session is being executed via the 'sessionType' value in the config_mail.yaml file. A PrimAITE session will usually be associated with a "Use Case Profile"; this document will present:
* The Use Case name, default number of steps in an episode and default number of episodes in a session. The number of steps and episodes can be modified in the configuration files
* The system laydown being modelled
* The objectives of the session (steady-state), the red agent and the blue agent (in a defensive role)
* The green agent pattern-of-life profile
* The red agent attack profile
* The observation space definition
* The action space definition
* Agent integration guidance
* Initial Access Control List settings (if applicable)
* The reward function definition
**Integrating a user defined blue agent**
Integrating a blue agent with PrimAITE requires some modification of the code within the main.py file. The main.py file consists of a number of functions, each of which will invoke training for a particular agent. These are:
* Generic (run_generic)
* Stable Baselines 3 PPO (run_stable_baselines3_ppo)
* Stable Baselines 3 A2C (run_stable_baselines3_a2c)
The selection of which agent type to use is made via the config_main.yaml file. In order to train a user generated agent,
the run_generic function should be selected, and should be modified (typically) to be:
.. code:: python
agent = MyAgent(environment, max_steps)
for episode in range(0, num_episodes):
agent.learn()
env.close()
save_agent(agent)
Where:
* *MyAgent* is the user created agent
* *environment* is the PrimAITE environment
* *max_steps* is the number of steps in an episode, as defined in the config_[name].yaml file
* *num_episodes* is the number of episodes in the session, as defined in the config_main.yaml file
* the *.learn()* function should be defined in the user created agent
* the *env.close()* function is defined within PrimAITE
* the *save_agent()* assumes that a *save()* function has been defined in the user created agent. If not, this line can be ommitted (although it is encouraged, since it will allow the agent to be saved and ported)
The code below provides a suggested format for the learn() function within the user created agent.
It's important to include the *self.environment.reset()* call within the episode loop in order that the
environment is reset between episodes. Note that the example below should not be considered exhaustive.
.. code:: python
def learn(self) :
# pre-reqs
# reset the environment
self.environment.reset()
done = False
for step in range(max_steps):
# calculate the action
action = ...
# execute the environment step
new_state, reward, done, info = self.environment.step(action)
# algorithm updates
...
# update to our new state
state = new_state
# if done, finish episode
if done == True:
break
**Running the session**
In order to execute a session, carry out the following steps:
1. Navigate to "[Install directory]\\Primaite\\Primaite\\”
2. Start a console window (type “CMD” in path window, or start a console window first and navigate to “[Install Directory]\\Primaite\\Primaite\\”)
3. Type “python main.py”
4. The session will start with an output indicating the current episode, and average reward value for the episode