Add second green agent and make rewards based on webbrowser
This commit is contained in:
Marek Wolan
@@ -46,7 +46,7 @@
|
||||
"source": [
|
||||
"## Green agent\n",
|
||||
"\n",
|
||||
"The green agent is logged onto client 2. It sometimes uses the web browser on client 2 to navigate to `http://arcd.com/users`. The web server replies with a status code 200 if the data is available on the database or 404 if not available."
|
||||
"There are green agents is logged onto client 1 and client 2. They use the web browser to navigate to `http://arcd.com/users`. The web server replies with a status code 200 if the data is available on the database or 404 if not available."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -68,7 +68,7 @@
|
||||
"source": [
|
||||
"## Blue agent\n",
|
||||
"\n",
|
||||
"The blue agent can view the entire network, but the health statuses of components are not updated until a scan is performed. The blue agent should restore the database file from backup after it was compromised. It can also prevent further attacks by blocking client 1 from reaching the database server. This can be done by removing client 1's network connection or adding ACL rules on the router to stop the packets from arriving."
|
||||
"The blue agent can view the entire network, but the health statuses of components are not updated until a scan is performed. The blue agent should restore the database file from backup after it was compromised. It can also prevent further attacks by blocking client 1 from sending the malicious SQL query to the database server. This can be done by removing implementing an ACL rule on the router."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -100,9 +100,9 @@
|
||||
"The red agent does not use information about the state of the network to decide its action.\n",
|
||||
"\n",
|
||||
"### Green\n",
|
||||
"The green agent sits on client 2 and uses the web browser application to send requests to the web server. The schedule of the green agent is currently random, meaning it will request webpage with a 50% probability, and do nothing with a 50% probability.\n",
|
||||
"The green agents use the web browser application to send requests to the web server. The schedule of each green agent is currently random, meaning it will request webpage with a 50% probability, and do nothing with a 50% probability.\n",
|
||||
"\n",
|
||||
"When the green agent is blocked from accessing the data through the webpage, this incurs a negative reward to the RL defender."
|
||||
"When a green agent is blocked from accessing the data through the webpage, this incurs a negative reward to the RL defender."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -295,7 +295,7 @@
|
||||
"- `28-37`: Remove ACL rules 1-10\n",
|
||||
"- `42`: Disconnect client 1 from the network\n",
|
||||
"\n",
|
||||
"The other actions will either have no effect or will negatively impact the network, so the blue agent should avoid taking other actions, and learn about these actions."
|
||||
"The other actions will either have no effect or will negatively impact the network, so the blue agent should avoid taking them."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -306,8 +306,8 @@
|
||||
"\n",
|
||||
"The blue agent's reward is calculated using two measures:\n",
|
||||
"1. Whether the database file is in a good state (+1 for good, -1 for corrupted, 0 for any other state)\n",
|
||||
"2. Whether the green agent's most recent webpage request was successful (+1 for a `200` return code, -1 for a `404` return code and 0 otherwise).\n",
|
||||
"These two components are averaged to get the final reward.\n"
|
||||
"2. Whether each green agents' most recent webpage request was successful (+1 for a `200` return code, -1 for a `404` return code and 0 otherwise).\n",
|
||||
"The file status reward and the two green-agent-related reward are averaged to get a total step reward.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -326,7 +326,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -336,20 +336,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/home/cade/repos/PrimAITE/venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
|
||||
" from .autonotebook import tqdm as notebook_tqdm\n",
|
||||
"2024-01-25 14:43:32,056\tINFO util.py:159 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.\n",
|
||||
"2024-01-25 14:43:35,213\tINFO util.py:159 -- Missing packages: ['ipywidgets']. Run `pip install -U ipywidgets`, then restart the notebook server for rich notebook output.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Imports\n",
|
||||
"from primaite.config.load import example_config_path\n",
|
||||
@@ -370,134 +359,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Resetting environment, episode 0, avg. reward: 0.0\n",
|
||||
"env created successfully\n",
|
||||
"{'ACL': {1: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 0,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 2: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 1,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 3: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 2,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 4: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 3,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 5: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 4,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 6: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 5,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 7: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 6,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 8: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 7,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 9: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 8,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 10: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 9,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0}},\n",
|
||||
" 'ICS': 0,\n",
|
||||
" 'LINKS': {1: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 2: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 3: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 4: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 5: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 6: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 7: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 8: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 9: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 10: {'PROTOCOLS': {'ALL': 1}}},\n",
|
||||
" 'NODES': {1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 1}},\n",
|
||||
" 'health_status': 1}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# create the env\n",
|
||||
"with open(example_config_path(), 'r') as f:\n",
|
||||
@@ -523,48 +387,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 1, Red action: DONOTHING, Blue reward:0.5\n",
|
||||
"step: 2, Red action: DONOTHING, Blue reward:0.5\n",
|
||||
"step: 3, Red action: DONOTHING, Blue reward:0.5\n",
|
||||
"step: 4, Red action: DONOTHING, Blue reward:0.5\n",
|
||||
"step: 5, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 6, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 7, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 8, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 9, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 10, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 11, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 12, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 13, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 14, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 15, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 16, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 17, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 18, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 19, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 20, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 21, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 22, Red action: NODE_APPLICATION_EXECUTE, Blue reward:0.0\n",
|
||||
"step: 23, Red action: DONOTHING, Blue reward:0.0\n",
|
||||
"step: 24, Red action: DONOTHING, Blue reward:0.0\n",
|
||||
"step: 25, Red action: DONOTHING, Blue reward:0.0\n",
|
||||
"step: 26, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 27, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 28, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 29, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 30, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 31, Red action: DONOTHING, Blue reward:-1.0\n",
|
||||
"step: 32, Red action: DONOTHING, Blue reward:-1.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for step in range(32):\n",
|
||||
" obs, reward, terminated, truncated, info = env.step(0)\n",
|
||||
@@ -580,44 +405,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 1}}, 'health_status': 1}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pprint(obs['NODES'])"
|
||||
]
|
||||
@@ -631,44 +421,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 3, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 2}}, 'health_status': 1}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NICS': {1: {'nic_status': 1}, 2: {'nic_status': 0}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(9) # scan database file\n",
|
||||
"obs, reward, terminated, truncated, info = env.step(1) # scan webapp service\n",
|
||||
@@ -692,24 +447,14 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 35\n",
|
||||
"Red action: DONOTHING\n",
|
||||
"Green action: NODE_APPLICATION_EXECUTE\n",
|
||||
"Blue reward:-1.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(13) # patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}\")\n",
|
||||
"print(f\"Red action: {info['agent_actions']['client_1_data_manipulation_red_bot'][0]}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_1_green_user'][0]}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_2_green_user'][0]}\" )\n",
|
||||
"print(f\"Blue reward:{reward}\" )"
|
||||
]
|
||||
@@ -727,25 +472,15 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 36\n",
|
||||
"Red action: DONOTHING\n",
|
||||
"Green action: NODE_APPLICATION_EXECUTE\n",
|
||||
"Blue reward:0.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(0) # patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}\")\n",
|
||||
"print(f\"Red action: {info['agent_actions']['client_1_data_manipulation_red_bot'][0]}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_2_green_user'][0]}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_1_green_user'][0]}\" )\n",
|
||||
"print(f\"Blue reward:{reward}\" )"
|
||||
]
|
||||
},
|
||||
@@ -758,48 +493,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 37, Red action: DONOTHING, Blue reward:0.0\n",
|
||||
"step: 38, Red action: DONOTHING, Blue reward:0.0\n",
|
||||
"step: 39, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 40, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 41, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 42, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 43, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 44, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 45, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 46, Red action: NODE_APPLICATION_EXECUTE, Blue reward:1.0\n",
|
||||
"step: 47, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 48, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 49, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 50, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 51, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 52, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 53, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 54, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 55, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 56, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 57, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 58, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 59, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 60, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 61, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 62, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 63, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 64, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 65, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 66, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 67, Red action: DONOTHING, Blue reward:1.0\n",
|
||||
"step: 68, Red action: DONOTHING, Blue reward:1.0\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"env.step(13) # Patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['client_1_data_manipulation_red_bot'][0]}, Blue reward:{reward}\" )\n",
|
||||
@@ -826,101 +522,14 @@
|
||||
"Let's also have a look at the ACL observation to verify our new ACL rule at position 5."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{1: {'position': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 2: {'position': 1,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 3: {'position': 2,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 4: {'position': 3,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 5: {'position': 4,\n",
|
||||
" 'permission': 2,\n",
|
||||
" 'source_node_id': 7,\n",
|
||||
" 'source_port': 1,\n",
|
||||
" 'dest_node_id': 4,\n",
|
||||
" 'dest_port': 1,\n",
|
||||
" 'protocol': 3},\n",
|
||||
" 6: {'position': 5,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 7: {'position': 6,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 8: {'position': 7,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 9: {'position': 8,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 10: {'position': 9,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0}}"
|
||||
]
|
||||
},
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"obs['ACL']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
"source": [
|
||||
"obs['ACL']"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
Reference in New Issue
Block a user