Update notebook with new changes
This commit is contained in:
Marek Wolan
@@ -13,7 +13,7 @@
|
||||
"source": [
|
||||
"## Scenario\n",
|
||||
"\n",
|
||||
"The network consists of an office subnet and a server subnet. Clients in the office access a website which fetches data from a database.\n",
|
||||
"The network consists of an office subnet and a server subnet. Clients in the office access a website which fetches data from a database. Occasionally, admins need to access the database directly from the clients.\n",
|
||||
"\n",
|
||||
"[<img src=\"_package_data/uc2_network.png\" width=\"500\"/>](_package_data/uc2_network.png)\n",
|
||||
"\n",
|
||||
@@ -46,7 +46,9 @@
|
||||
"source": [
|
||||
"## Green agent\n",
|
||||
"\n",
|
||||
"There are green agents logged onto client 1 and client 2. They use the web browser to navigate to `http://arcd.com/users`. The web server replies with a status code 200 if the data is available on the database or 404 if not available."
|
||||
"There are green agents logged onto client 1 and client 2. They use the web browser to navigate to `http://arcd.com/users`. The web server replies with a status code 200 if the data is available on the database or 404 if not available.\n",
|
||||
"\n",
|
||||
"Sometimes, the green agents send a request directly to the database to check that it is reachable."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -68,7 +70,9 @@
|
||||
"source": [
|
||||
"## Blue agent\n",
|
||||
"\n",
|
||||
"The blue agent can view the entire network, but the health statuses of components are not updated until a scan is performed. The blue agent should restore the database file from backup after it was compromised. It can also prevent further attacks by blocking the red agent client from sending the malicious SQL query to the database server. This can be done by implementing an ACL rule on the router."
|
||||
"The blue agent can view the entire network, but the health statuses of components are not updated until a scan is performed. The blue agent should restore the database file from backup after it was compromised. It can also prevent further attacks by blocking the red agent client from sending the malicious SQL query to the database server. This can be done by implementing an ACL rule on the router.\n",
|
||||
"\n",
|
||||
"However, these rules will also impact greens' ability to check the database connection. The blue agent should only block the infected client, it should let the other client connect freely. Once the attack has begun, automated traffic monitoring will detect it as suspicious network traffic. The blue agent's observation space will show this as an increase in the number of malicious network events (NMNE) on one of the network interfaces. To achieve optimal reward, the agent should only block the client which has the non-zero outbound NMNE."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -101,9 +105,11 @@
|
||||
"The red agent does not use information about the state of the network to decide its action.\n",
|
||||
"\n",
|
||||
"### Green\n",
|
||||
"The green agents use the web browser application to send requests to the web server. The schedule of each green agent is currently random, meaning it will request webpage with a 50% probability, and do nothing with a 50% probability.\n",
|
||||
"The green agents use the web browser application to send requests to the web server. The schedule of each green agent is currently random, it will do nothing 30% of the time, send a web request 60% of the time, and send a db status check 10% of the time.\n",
|
||||
"\n",
|
||||
"When a green agent is blocked from accessing the data through the webpage, this incurs a negative reward to the RL defender."
|
||||
"When a green agent is blocked from accessing the data through the webpage, this incurs a negative reward to the RL defender.\n",
|
||||
"\n",
|
||||
"Also, when the green agent is blocked from checking the database status, it causes a small negative reward."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -322,9 +328,10 @@
|
||||
"source": [
|
||||
"## Reward Function\n",
|
||||
"\n",
|
||||
"The blue agent's reward is calculated using two measures:\n",
|
||||
"The blue agent's reward is calculated using these measures:\n",
|
||||
"1. Whether the database file is in a good state (+1 for good, -1 for corrupted, 0 for any other state)\n",
|
||||
"2. Whether each green agents' most recent webpage request was successful (+1 for a `200` return code, -1 for a `404` return code and 0 otherwise).\n",
|
||||
"3. Whether each green agents' most recent DB status check was successful (+1 for a successful connection, -1 for no connection).\n",
|
||||
"\n",
|
||||
"The file status reward and the two green-agent-related rewards are averaged to get a total step reward.\n"
|
||||
]
|
||||
@@ -345,7 +352,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
@@ -357,7 +364,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
@@ -382,9 +389,169 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Resetting environment, episode 0, avg. reward: 0.0\n",
|
||||
"env created successfully\n",
|
||||
"{'ACL': {1: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 0,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 2: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 1,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 3: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 2,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 4: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 3,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 5: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 4,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 6: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 5,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 7: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 6,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 8: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 7,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 9: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 8,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0},\n",
|
||||
" 10: {'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'position': 9,\n",
|
||||
" 'protocol': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0}},\n",
|
||||
" 'ICS': 0,\n",
|
||||
" 'LINKS': {1: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 2: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 3: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 4: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 5: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 6: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 7: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 8: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 9: {'PROTOCOLS': {'ALL': 1}},\n",
|
||||
" 10: {'PROTOCOLS': {'ALL': 0}}},\n",
|
||||
" 'NODES': {1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 1}},\n",
|
||||
" 'health_status': 1}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}},\n",
|
||||
" 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0,\n",
|
||||
" 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# create the env\n",
|
||||
"with open(example_config_path(), 'r') as f:\n",
|
||||
@@ -407,12 +574,12 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The red agent will start attacking at some point between step 20 and 30. When this happens, the reward will drop immediately, then drop to -1.0 when green agents try to access the webpage."
|
||||
"The red agent will start attacking at some point between step 20 and 30. When this happens, the reward will drop immediately, then drop to -0.8 when green agents try to access the webpage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -430,9 +597,51 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 1, Red action: DO NOTHING, Blue reward:0.77\n",
|
||||
"step: 2, Red action: DO NOTHING, Blue reward:0.77\n",
|
||||
"step: 3, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 4, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 5, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 6, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 7, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 8, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 9, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 10, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 11, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 12, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 13, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 14, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 15, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 16, Red action: DO NOTHING, Blue reward:1.10\n",
|
||||
"step: 17, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 18, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 19, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 20, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 21, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 22, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 23, Red action: DO NOTHING, Blue reward:1.20\n",
|
||||
"step: 24, Red action: ATTACK from client 2, Blue reward:0.52\n",
|
||||
"step: 25, Red action: DO NOTHING, Blue reward:0.52\n",
|
||||
"step: 26, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 27, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 28, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 29, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 30, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 31, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 32, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 33, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 34, Red action: DO NOTHING, Blue reward:-0.80\n",
|
||||
"step: 35, Red action: DO NOTHING, Blue reward:-0.80\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for step in range(35):\n",
|
||||
" obs, reward, terminated, truncated, info = env.step(0)\n",
|
||||
@@ -448,9 +657,65 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 1}}, 'health_status': 1}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 1, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 1}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"pprint(obs['NODES'])"
|
||||
]
|
||||
@@ -464,9 +729,65 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{1: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 2: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 3, 'operating_status': 1}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 3: {'FOLDERS': {1: {'FILES': {1: {'health_status': 2}}, 'health_status': 1}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 1, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 4: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 5: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 6: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1},\n",
|
||||
" 7: {'FOLDERS': {1: {'FILES': {1: {'health_status': 0}}, 'health_status': 0}},\n",
|
||||
" 'NETWORK_INTERFACES': {1: {'nic_status': 1,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 1}},\n",
|
||||
" 2: {'nic_status': 0,\n",
|
||||
" 'nmne': {'inbound': 0, 'outbound': 0}}},\n",
|
||||
" 'SERVICES': {1: {'health_status': 0, 'operating_status': 0}},\n",
|
||||
" 'operating_status': 1}}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(9) # scan database file\n",
|
||||
"obs, reward, terminated, truncated, info = env.step(1) # scan webapp service\n",
|
||||
@@ -481,6 +802,13 @@
|
||||
"File 1 in folder 1 on node 3 has `health_status = 2`, indicating that the database file is compromised."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Also, the NMNE outbound of either client 1 (node 6) or client 2 (node 7) has increased from 0 to 1. This tells us which client is being used by the red agent."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -490,9 +818,21 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 38\n",
|
||||
"Red action: DONOTHING\n",
|
||||
"Green action: DONOTHING\n",
|
||||
"Green action: DONOTHING\n",
|
||||
"Blue reward:-0.8\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(13) # patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}\")\n",
|
||||
@@ -515,16 +855,28 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 22,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 52\n",
|
||||
"Red action: DONOTHING\n",
|
||||
"Green action: ('NODE_APPLICATION_EXECUTE', {'node_id': 0, 'application_id': 0})\n",
|
||||
"Green action: ('NODE_APPLICATION_EXECUTE', {'node_id': 0, 'application_id': 0})\n",
|
||||
"Blue reward:-0.80\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"obs, reward, terminated, truncated, info = env.step(0) # patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}\")\n",
|
||||
"print(f\"Red action: {info['agent_actions']['data_manipulation_attacker'][0]}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_2_green_user']}\" )\n",
|
||||
"print(f\"Green action: {info['agent_actions']['client_1_green_user']}\" )\n",
|
||||
"print(f\"Blue reward:{reward}\" )"
|
||||
"print(f\"Blue reward:{reward:.2f}\" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -538,29 +890,69 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 23,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 53, Red action: DONOTHING, Blue reward:-0.80\n",
|
||||
"step: 54, Red action: DONOTHING, Blue reward:-0.80\n",
|
||||
"step: 55, Red action: DONOTHING, Blue reward:-0.80\n",
|
||||
"step: 56, Red action: DONOTHING, Blue reward:0.54\n",
|
||||
"step: 57, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 58, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 59, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 60, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 61, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 62, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 63, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 64, Red action: DONOTHING, Blue reward:1.20\n",
|
||||
"step: 65, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 66, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 67, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 68, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 69, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 70, Red action: NODE_APPLICATION_EXECUTE, Blue reward:1.00\n",
|
||||
"step: 71, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 72, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 73, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 74, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 75, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 76, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 77, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 78, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 79, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 80, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 81, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 82, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 83, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 84, Red action: DONOTHING, Blue reward:0.80\n",
|
||||
"step: 85, Red action: NODE_APPLICATION_EXECUTE, Blue reward:0.80\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"env.step(13) # Patch the database\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward}\" )\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward:.2f}\" )\n",
|
||||
"\n",
|
||||
"env.step(26) # Block client 1\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward}\" )\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward:.2f}\" )\n",
|
||||
"\n",
|
||||
"env.step(27) # Block client 2\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward}\" )\n",
|
||||
"print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward:.2f}\" )\n",
|
||||
"\n",
|
||||
"for step in range(30):\n",
|
||||
" obs, reward, terminated, truncated, info = env.step(0) # do nothing\n",
|
||||
" print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward}\" )"
|
||||
" print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward:.2f}\" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, even though the red agent executes an attack, the reward stays at 1.0"
|
||||
"Now, even though the red agent executes an attack, the reward stays at 0.8."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -572,11 +964,168 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{1: {'position': 0,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 2: {'position': 1,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 3: {'position': 2,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 4: {'position': 3,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 5: {'position': 4,\n",
|
||||
" 'permission': 2,\n",
|
||||
" 'source_node_id': 7,\n",
|
||||
" 'source_port': 1,\n",
|
||||
" 'dest_node_id': 4,\n",
|
||||
" 'dest_port': 1,\n",
|
||||
" 'protocol': 3},\n",
|
||||
" 6: {'position': 5,\n",
|
||||
" 'permission': 2,\n",
|
||||
" 'source_node_id': 8,\n",
|
||||
" 'source_port': 1,\n",
|
||||
" 'dest_node_id': 4,\n",
|
||||
" 'dest_port': 1,\n",
|
||||
" 'protocol': 3},\n",
|
||||
" 7: {'position': 6,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 8: {'position': 7,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 9: {'position': 8,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0},\n",
|
||||
" 10: {'position': 9,\n",
|
||||
" 'permission': 0,\n",
|
||||
" 'source_node_id': 0,\n",
|
||||
" 'source_port': 0,\n",
|
||||
" 'dest_node_id': 0,\n",
|
||||
" 'dest_port': 0,\n",
|
||||
" 'protocol': 0}}"
|
||||
]
|
||||
},
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"obs['ACL']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We can slightly increase the reward by unblocking the client which isn't being used by the attacker. If node 6 has outbound NMNEs, let's unblock client 2, and if node 7 has outbound NMNEs, let's unblock client 1."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"obs['ACL']"
|
||||
"if obs['NODES'][6]['NETWORK_INTERFACES'][1]['nmne']['outbound'] == 1:\n",
|
||||
" # client 1 has NMNEs, let's unblock client 2\n",
|
||||
" env.step(34) # remove ACL rule 6\n",
|
||||
"elif obs['NODES'][7]['NETWORK_INTERFACES'][1]['nmne']['outbound'] == 1:\n",
|
||||
" env.step(33) # remove ACL rule 5\n",
|
||||
"else:\n",
|
||||
" print(\"something went wrong, neither client has NMNEs\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now, the reward will eventually increase to 1.0, even after red agent attempts subsequent attacks."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 35,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"step: 117, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 118, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 119, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 120, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 121, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 122, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 123, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 124, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 125, Red action: NODE_APPLICATION_EXECUTE, Blue reward:1.00\n",
|
||||
"step: 126, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 127, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 128, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 129, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 130, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 131, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 132, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 133, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 134, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 135, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 136, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 137, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 138, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 139, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 140, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 141, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 142, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 143, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 144, Red action: NODE_APPLICATION_EXECUTE, Blue reward:1.00\n",
|
||||
"step: 145, Red action: DONOTHING, Blue reward:1.00\n",
|
||||
"step: 146, Red action: DONOTHING, Blue reward:1.00\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for step in range(30):\n",
|
||||
" obs, reward, terminated, truncated, info = env.step(0) # do nothing\n",
|
||||
" print(f\"step: {env.game.step_counter}, Red action: {info['agent_actions']['data_manipulation_attacker'][0]}, Blue reward:{reward:.2f}\" )"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -594,13 +1143,6 @@
|
||||
"source": [
|
||||
"env.reset()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
Reference in New Issue
Block a user