oxbrbtx/PrimAITE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

#2472: add install pandoc to docs build pipeline + fix RLlib running for a long time

Browse Source
This commit is contained in:
Czar Echavez
2024-05-16 10:55:17 +01:00
parent cb90c63c18
commit 82953b4b0a
6 changed files with 106 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.azure/azure-build-deploy-docs-pipeline.yml
View File

@@ -29,6 +29,10 @@ jobs:
pip install -e .[dev]
displayName: 'Install PrimAITE for docs autosummary'
- script: |
apt-get install pandoc
displayName: 'Install Pandoc'
- script: |
primaite setup
displayName: 'Perform PrimAITE Setup'

5
docs/conf.py
View File

@@ -135,9 +135,6 @@ def copy_notebooks_to_docs() -> Any:
This allows developers to create new notebooks without having to worry about updating documentation when
a new notebook is included within PrimAITE.
"""
# temporarily ignore these notebooks because we love RLlib and its quirks
ignored_notebooks = ["Training-an-RLLib-Agent.ipynb", "Training-an-RLLIB-MARL-System.ipynb"]
notebook_asset_types = [".ipynb", ".png"]
notebook_directories = []
@@ -152,7 +149,7 @@ def copy_notebooks_to_docs() -> Any:
shutil.copytree(
src=notebook_parent,
dst=Path("source") / "notebooks" / notebook_parent.name,
ignore=notebook_assets(ignored_files=ignored_notebooks, include_file_types=notebook_asset_types),
ignore=notebook_assets(include_file_types=notebook_asset_types),
dirs_exist_ok=True,
)

2
src/primaite/notebooks/Training-an-RLLIB-MARL-System.ipynb
View File

@@ -83,7 +83,7 @@
"tune.Tuner(\n",
" \"PPO\",\n",
" run_config=air.RunConfig(\n",
" stop={\"timesteps_total\": 512},\n",
" stop={\"timesteps_total\": 5 * 128},\n",
" ),\n",
" param_space=config\n",
").fit()"

2
src/primaite/notebooks/Training-an-RLLib-Agent.ipynb
View File

@@ -74,7 +74,7 @@
"tune.Tuner(\n",
" \"PPO\",\n",
" run_config=air.RunConfig(\n",
" stop={\"timesteps_total\": 1e3 * 128}\n",
" stop={\"timesteps_total\": 5 * 128}\n",
" ),\n",
" param_space=config\n",
").fit()\n"

2
src/primaite/notebooks/Training-an-SB3-Agent.ipynb
View File

@@ -78,7 +78,7 @@
"from stable_baselines3 import PPO\n",
"\n",
"EPISODE_LEN = 128\n",
"NUM_EPISODES = 10\n",
"NUM_EPISODES = 5\n",
"NO_STEPS = EPISODE_LEN * NUM_EPISODES\n",
"BATCH_SIZE = 32\n",
"LEARNING_RATE = 3e-4"

218
src/primaite/simulator/_package_data/network_simulator_demo.ipynb
View File

@@ -258,11 +258,9 @@
{
"cell_type": "markdown",
"id": "22",
"metadata": {
"tags": []
},
"metadata": {},
"source": [
"Calling `switch.arp.show()` displays the Switch ARP Cache."
"Calling `switch.sys_log.show()` displays the Switch system log. By default, only the last 10 log entries are displayed, this can be changed by passing `last_n=<number of log entries>`."
]
},
{
@@ -273,33 +271,13 @@
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"switch_1\").arp.show()"
]
},
{
"cell_type": "markdown",
"id": "24",
"metadata": {},
"source": [
"Calling `switch.sys_log.show()` displays the Switch system log. By default, only the last 10 log entries are displayed, this can be changed by passing `last_n=<number of log entries>`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"switch_1\").sys_log.show()"
]
},
{
"cell_type": "markdown",
"id": "26",
"id": "24",
"metadata": {},
"source": [
"### Computer/Server Nodes\n",
@@ -309,7 +287,7 @@
},
{
"cell_type": "markdown",
"id": "27",
"id": "25",
"metadata": {
"tags": []
},
@@ -317,6 +295,26 @@
"Calling `computer.show()` displays the NICs on the Computer/Server."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"security_suite\").show()"
]
},
{
"cell_type": "markdown",
"id": "27",
"metadata": {},
"source": [
"Calling `computer.arp.show()` displays the Computer/Server ARP Cache."
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -326,7 +324,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"security_suite\").show()"
"network.get_node_by_hostname(\"security_suite\").arp.show()"
]
},
{
@@ -334,7 +332,7 @@
"id": "29",
"metadata": {},
"source": [
"Calling `computer.arp.show()` displays the Computer/Server ARP Cache."
"Calling `computer.sys_log.show()` displays the Computer/Server system log. By default, only the last 10 log entries are displayed, this can be changed by passing `last_n=<number of log entries>`."
]
},
{
@@ -346,7 +344,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"security_suite\").arp.show()"
"network.get_node_by_hostname(\"security_suite\").sys_log.show()"
]
},
{
@@ -354,7 +352,9 @@
"id": "31",
"metadata": {},
"source": [
"Calling `computer.sys_log.show()` displays the Computer/Server system log. By default, only the last 10 log entries are displayed, this can be changed by passing `last_n=<number of log entries>`."
"## Basic Network Comms Check\n",
"\n",
"We can perform a good old ping to check that Nodes are able to communicate with each other."
]
},
{
@@ -366,7 +366,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"security_suite\").sys_log.show()"
"network.show(nodes=False, links=False)"
]
},
{
@@ -374,9 +374,7 @@
"id": "33",
"metadata": {},
"source": [
"## Basic Network Comms Check\n",
"\n",
"We can perform a good old ping to check that Nodes are able to communicate with each other."
"We'll first ping client_1's default gateway."
]
},
{
@@ -388,27 +386,27 @@
},
"outputs": [],
"source": [
"network.show(nodes=False, links=False)"
]
},
{
"cell_type": "markdown",
"id": "35",
"metadata": {},
"source": [
"We'll first ping client_1's default gateway."
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.10.1\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "36",
"id": "35",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.10.1\")"
"network.get_node_by_hostname(\"client_1\").sys_log.show(15)"
]
},
{
"cell_type": "markdown",
"id": "36",
"metadata": {},
"source": [
"Next, we'll ping the interface of the 192.168.1.0/24 Network on the Router (port 1)."
]
},
{
@@ -420,7 +418,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_1\").sys_log.show(15)"
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.1.1\")"
]
},
{
@@ -428,7 +426,7 @@
"id": "38",
"metadata": {},
"source": [
"Next, we'll ping the interface of the 192.168.1.0/24 Network on the Router (port 1)."
"And finally, we'll ping the web server."
]
},
{
@@ -440,7 +438,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.1.1\")"
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.1.12\")"
]
},
{
@@ -448,7 +446,7 @@
"id": "40",
"metadata": {},
"source": [
"And finally, we'll ping the web server."
"To confirm that the ping was received and processed by the web_server, we can view the sys log"
]
},
{
@@ -459,33 +457,13 @@
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.1.12\")"
]
},
{
"cell_type": "markdown",
"id": "42",
"metadata": {},
"source": [
"To confirm that the ping was received and processed by the web_server, we can view the sys log"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "43",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"web_server\").sys_log.show()"
]
},
{
"cell_type": "markdown",
"id": "44",
"id": "42",
"metadata": {},
"source": [
"## Advanced Network Usage\n",
@@ -495,12 +473,32 @@
},
{
"cell_type": "markdown",
"id": "45",
"id": "43",
"metadata": {},
"source": [
"Let's attempt to prevent client_2 from being able to ping the web server. First, we'll confirm that it can ping the server first..."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "44",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").ping(\"192.168.1.12\")"
]
},
{
"cell_type": "markdown",
"id": "45",
"metadata": {},
"source": [
"If we look at the client_2 sys log we can see that the four ICMP echo requests were sent and four ICMP each replies were received:"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -509,33 +507,13 @@
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").ping(\"192.168.1.12\")"
]
},
{
"cell_type": "markdown",
"id": "47",
"metadata": {},
"source": [
"If we look at the client_2 sys log we can see that the four ICMP echo requests were sent and four ICMP each replies were received:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "48",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").sys_log.show()"
]
},
{
"cell_type": "markdown",
"id": "49",
"id": "47",
"metadata": {},
"source": [
"Now we'll add an ACL to block ICMP from 192.168.10.22"
@@ -544,7 +522,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "50",
"id": "48",
"metadata": {
"tags": []
},
@@ -564,7 +542,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "51",
"id": "49",
"metadata": {
"tags": []
},
@@ -575,12 +553,32 @@
},
{
"cell_type": "markdown",
"id": "52",
"id": "50",
"metadata": {},
"source": [
"Now we attempt (and fail) to ping the web server"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "51",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").ping(\"192.168.1.12\")"
]
},
{
"cell_type": "markdown",
"id": "52",
"metadata": {},
"source": [
"We can check that the ping was actually sent by client_2 by viewing the sys log"
]
},
{
"cell_type": "code",
"execution_count": null,
@@ -590,7 +588,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").ping(\"192.168.1.12\")"
"network.get_node_by_hostname(\"client_2\").sys_log.show()"
]
},
{
@@ -598,7 +596,7 @@
"id": "54",
"metadata": {},
"source": [
"We can check that the ping was actually sent by client_2 by viewing the sys log"
"We can check the router sys log to see why the traffic was blocked"
]
},
{
@@ -610,7 +608,7 @@
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_2\").sys_log.show()"
"network.get_node_by_hostname(\"router_1\").sys_log.show()"
]
},
{
@@ -618,7 +616,7 @@
"id": "56",
"metadata": {},
"source": [
"We can check the router sys log to see why the traffic was blocked"
"Now a final check to ensure that client_1 can still ping the web_server."
]
},
{
@@ -629,26 +627,6 @@
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"router_1\").sys_log.show()"
]
},
{
"cell_type": "markdown",
"id": "58",
"metadata": {},
"source": [
"Now a final check to ensure that client_1 can still ping the web_server."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"network.get_node_by_hostname(\"client_1\").ping(\"192.168.1.12\")"
]
@@ -656,7 +634,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "60",
"id": "58",
"metadata": {
"tags": []
},