Commit Graph

1288 Commits

Author SHA1 Message Date
Marek Wolan
80158fd9b4 Make db manipulation bot work with db client 2024-03-03 11:18:06 +00:00
Marek Wolan
2a1d99ccce Fix problem with checking connection for db admin 2024-03-01 16:36:41 +00:00
Marek Wolan
ed01293b86 Make db admin reward persistent 2024-03-01 16:02:27 +00:00
Marek Wolan
10a4053887 Fix tests 2024-03-01 15:14:00 +00:00
Marek Wolan
9ff8adab1a Merge remote-tracking branch 'origin/dev' into feature/2319-database-admin 2024-03-01 09:23:00 +00:00
Czar Echavez
b849bfd6e2 Merged PR 280: Router Routes and Configuration documentation updates
## Summary

- Updated Sphinx 6.1.3 => 7.1.2
- Updated furo 2023.3.27 => 2024.01.29
- Added tests to check that firewall, routers and nodes are properly added via config (config parser tests)
- Added some reference points in code comments for Sphinx documentation to reference
- Created a list of System Software so it can be referenced in docs and by code
- Added default values to config within game.py (The defaults are pulled from example_config.yaml)
- Created a section for creating a session via config files:
    - set up a lot of stuff so documentation is easier to maintain
    - Template for things that are repeated in places
    - added how to create nodes via config
    - added how to install applications and services via config
- diagrams to make it easier to understand some stuff e.g. ACL rules for firewall

see https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/CHANGELOG.md

## Test process
- Firewall:
  - Created a DMZ Network example config
  - Tested the creation of Firewall
  - Tested the ACL Rules of Firewall

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_firewall_config.py

https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/280?_a=files&path=/tests/integration_tests/configuration_file_parsing/nodes/network/test_router_config.py

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2257
2024-02-29 16:22:55 +00:00
Czar Echavez
69b7ca1703 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-29 15:21:20 +00:00
Czar Echavez
49a4e1fb56 #2257: added common node attributes page + ability to set node operating state via config + tests 2024-02-29 15:20:54 +00:00
Christopher McCarthy
90d51c071c Merged PR 288: #2326 - Network Interface port name/num fixed so that it carries through to s...
## Summary
This pull request fixes the issue o network interface port name and number not being pulled through into sys log and PCAP output files.

## Test process
Manually checked:
![image (4).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%284%29.png)
![image (2).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%282%29.png)
![image (3).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/288/attachments/image%20%283%29.png)

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

#2326 - Network Interface port name/num fixed so that it carries through to sys log and PCAP outputs.

Related work items: #2326
2024-02-29 15:14:08 +00:00
Chris McCarthy
8f0de8521e #2326 - removed port_name print statement 2024-02-29 14:08:42 +00:00
Marek Wolan
bd0b2e0033 Remove redundant notebook cells 2024-02-29 13:22:41 +00:00
Marek Wolan
2f3e40fb6b Fix issue around reset 2024-02-29 13:22:05 +00:00
Marek Wolan
f7c535b557 Merge 'origin/dev' into feature/2319-database-admin 2024-02-29 13:21:44 +00:00
Chris McCarthy
cf0674ce22 #2326 - Network Interface port name/num fixed so that it carries through to sys log and PCAP outputs. 2024-02-29 13:00:27 +00:00
Czar Echavez
eefc2739c8 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-29 11:34:56 +00:00
Marek Wolan
312b5c35bd Merged PR 283: Refactor episode reset
## Summary
Instead of setting all attributes back to a snapshot, simply recreate the entire game and all agents from a cached copy of the config.

This removes the need for `set_original_state` and `reset_component_for_episode` methods on SimComponents.

## Test process
* Unit tests passing
* I've also tried adding a `__del__` method to simcomponent and agent and press env.reset() to verify that the ref count reaches 0, and nothing is secretly keeping using an old part of the simulation.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2317
2024-02-29 11:11:03 +00:00
Czar Echavez
9a4587155b #2257: specifically stating that enpoint refs are node hostnames + remove TODO 2024-02-29 11:07:21 +00:00
Marek Wolan
ce39387f92 Merge 'origin/dev' into feature/2317-refactor-reset 2024-02-29 10:16:42 +00:00
Marek Wolan
8730330f73 Apply PR suggestions 2024-02-29 10:14:31 +00:00
Czar Echavez
13db07dc11 Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-28 15:09:23 +00:00
Czar Echavez
6d43c61058 #2257: apply PR suggestions 2024-02-28 15:08:00 +00:00
Christopher McCarthy
98427b0e73 Merged PR 279: #2238 - Implement NMNE detection and capture
## Summary

This pull request introduces new features for capturing Malicious Network Events (NMNE) within the `NetworkInterface` class and extends the functionality to the `NicObservation` class .Additionally, it updates the simulation configuration to allow customizable NMNE capturing settings.

### Changes
- `NetworkInterface` Enhancements: Added NMNE capturing capabilities to detect and log specified malicious activities.
- `NicObservation` Integration: Updated to support detailed monitoring and analysis based on NMNE capturing results.
- Simulation Configuration: Introduced nmne_config options allowing users to enable NMNE capturing and define specific keywords, enhancing the adaptability of network security measures.

### New Configuration Options
Added to simulation.yml:
``` yaml
simulation:
  network:
    nmne_config:
      capture_nmne: true
      nmne_capture_keywords:
        - DELETE
```

Tests

Documentation
Updated README and related documentation to guide users on how to utilize the new NMNE capturing features and configure them in their simulations.

## Test process
- **NMNE Capture Testing**: Implemented tests using the UC2 network setup, where DELETE SQL queries are initiated by the database client residing on the web server and targeted towards the database service on the database server. Post-query, the network interface cards (NICs) on both servers are examined to verify accurate counting and logging of NMNE (Malicious Network Events) as expected per configuration.

- **NicObservation Testing**: Introduced additional tests to ensure proper integration of the `NicObservation `class, focusing on its ability to accurately observe and report NMNE occurrences.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [X] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

#2238

Related work items: #2238
2024-02-28 14:57:46 +00:00
Chris McCarthy
63ea5478ab #2238 - Updated uc2_demo.ipynb to explain the NMNE in observation space 2024-02-28 13:56:19 +00:00
Chris McCarthy
d55b6a5b48 #2238 - Fixed the observations issue causing tests to fail 2024-02-28 12:03:58 +00:00
Marek Wolan
af8ca82fcb Get the db admin green agent working 2024-02-27 13:30:16 +00:00
Marek Wolan
c54f82fb1b Start implementing green agent logic for UC2 2024-02-26 20:08:13 +00:00
Marek Wolan
922298eaf0 Make database admin action possible 2024-02-26 20:07:02 +00:00
Marek Wolan
33d2ecc26a Apply suggestions from code review. 2024-02-26 16:58:43 +00:00
Marek Wolan
f9cc5af7aa Not sure how this test was passing before 2024-02-26 16:06:58 +00:00
Czar Echavez
f2d7a2fc16 #2257: added way to ensure nodes are on at start + more test to make sure nodes are on when added via config 2024-02-26 14:34:34 +00:00
Marek Wolan
2076b011ba Put back default router rules 2024-02-26 14:26:47 +00:00
Marek Wolan
a5043a8fbe Modify tests based on refactoring 2024-02-26 12:15:53 +00:00
Czar Echavez
d738a23709 #2257: list of db payloads 2024-02-26 11:35:17 +00:00
Marek Wolan
ccb10f1160 Update docs based on reset refactor 2024-02-26 11:02:37 +00:00
Marek Wolan
e5982c4599 Change agents list in game object to dictionary 2024-02-26 10:26:28 +00:00
Czar Echavez
634f634097 #2257: fix text and make examples in node configs more specific 2024-02-26 09:47:12 +00:00
Czar Echavez
e964b8a3ea #2257: more in depth changelog 2024-02-26 08:58:03 +00:00
Czar Echavez
1d5c153752 #2257: changelog update 2024-02-26 08:49:11 +00:00
Czar Echavez
07373d941e #2257: Downgrade version of sphinx - 7.2.0 drops support for Python 3.8 2024-02-26 08:44:08 +00:00
Marek Wolan
63c9a36c30 Fix typos 2024-02-25 18:36:20 +00:00
Marek Wolan
994dbc3501 Finalise the refactor. It works well now. 2024-02-25 17:44:41 +00:00
Marek Wolan
a34cf08209 Merge remote-tracking branch 'origin/dev' into feature/2317-refactor-reset 2024-02-25 16:41:40 +00:00
Marek Wolan
c115095157 Fix router from config using wrong method 2024-02-25 16:17:12 +00:00
Czar Echavez
fb148dc4fb #2257: applications and services docs 2024-02-23 16:49:01 +00:00
Chris McCarthy
52677538a8 #2238 - Tidied up code, added more docstrings, and implemented suggestions from PR. 2024-02-23 15:12:46 +00:00
Marek Wolan
f933341df5 eod commit 2024-02-23 10:06:48 +00:00
Czar Echavez
ef61fe219c Merge remote-tracking branch 'origin/dev' into feature/2257-router-routes-cannot-be-represented-in-config-file 2024-02-23 08:56:35 +00:00
Czar Echavez
5836ea68e3 #2257: rearrange software pages + creating a list of applications and services which is hopefully a single point that should be referred to 2024-02-23 08:55:32 +00:00
Chris McCarthy
771a68dccb #2238 - Implement NMNE detection and logging in NetworkInterface.
- Enhance NicObservation for detailed NMNE event monitoring.
- Add nmne_config options to simulation settings for customizable NMNE capturing.
- Update documentation and tests for new NMNE features and simulation config.
2024-02-22 22:43:14 +00:00
Czar Echavez
98fb28cbbc #2257: setting up application and service docs 2024-02-21 18:19:16 +00:00