Commit Graph

1249 Commits

Author SHA1 Message Date
Chris McCarthy
cf0674ce22 #2326 - Network Interface port name/num fixed so that it carries through to sys log and PCAP outputs. 2024-02-29 13:00:27 +00:00
Marek Wolan
312b5c35bd Merged PR 283: Refactor episode reset
## Summary
Instead of setting all attributes back to a snapshot, simply recreate the entire game and all agents from a cached copy of the config.

This removes the need for `set_original_state` and `reset_component_for_episode` methods on SimComponents.

## Test process
* Unit tests passing
* I've also tried adding a `__del__` method to simcomponent and agent and press env.reset() to verify that the ref count reaches 0, and nothing is secretly keeping using an old part of the simulation.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2317
2024-02-29 11:11:03 +00:00
Marek Wolan
ce39387f92 Merge 'origin/dev' into feature/2317-refactor-reset 2024-02-29 10:16:42 +00:00
Marek Wolan
8730330f73 Apply PR suggestions 2024-02-29 10:14:31 +00:00
Christopher McCarthy
98427b0e73 Merged PR 279: #2238 - Implement NMNE detection and capture
## Summary

This pull request introduces new features for capturing Malicious Network Events (NMNE) within the `NetworkInterface` class and extends the functionality to the `NicObservation` class .Additionally, it updates the simulation configuration to allow customizable NMNE capturing settings.

### Changes
- `NetworkInterface` Enhancements: Added NMNE capturing capabilities to detect and log specified malicious activities.
- `NicObservation` Integration: Updated to support detailed monitoring and analysis based on NMNE capturing results.
- Simulation Configuration: Introduced nmne_config options allowing users to enable NMNE capturing and define specific keywords, enhancing the adaptability of network security measures.

### New Configuration Options
Added to simulation.yml:
``` yaml
simulation:
  network:
    nmne_config:
      capture_nmne: true
      nmne_capture_keywords:
        - DELETE
```

Tests

Documentation
Updated README and related documentation to guide users on how to utilize the new NMNE capturing features and configure them in their simulations.

## Test process
- **NMNE Capture Testing**: Implemented tests using the UC2 network setup, where DELETE SQL queries are initiated by the database client residing on the web server and targeted towards the database service on the database server. Post-query, the network interface cards (NICs) on both servers are examined to verify accurate counting and logging of NMNE (Malicious Network Events) as expected per configuration.

- **NicObservation Testing**: Introduced additional tests to ensure proper integration of the `NicObservation `class, focusing on its ability to accurately observe and report NMNE occurrences.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [X] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

#2238

Related work items: #2238
2024-02-28 14:57:46 +00:00
Chris McCarthy
63ea5478ab #2238 - Updated uc2_demo.ipynb to explain the NMNE in observation space 2024-02-28 13:56:19 +00:00
Chris McCarthy
d55b6a5b48 #2238 - Fixed the observations issue causing tests to fail 2024-02-28 12:03:58 +00:00
Marek Wolan
33d2ecc26a Apply suggestions from code review. 2024-02-26 16:58:43 +00:00
Marek Wolan
f9cc5af7aa Not sure how this test was passing before 2024-02-26 16:06:58 +00:00
Marek Wolan
2076b011ba Put back default router rules 2024-02-26 14:26:47 +00:00
Marek Wolan
a5043a8fbe Modify tests based on refactoring 2024-02-26 12:15:53 +00:00
Marek Wolan
ccb10f1160 Update docs based on reset refactor 2024-02-26 11:02:37 +00:00
Marek Wolan
e5982c4599 Change agents list in game object to dictionary 2024-02-26 10:26:28 +00:00
Marek Wolan
63c9a36c30 Fix typos 2024-02-25 18:36:20 +00:00
Marek Wolan
994dbc3501 Finalise the refactor. It works well now. 2024-02-25 17:44:41 +00:00
Marek Wolan
a34cf08209 Merge remote-tracking branch 'origin/dev' into feature/2317-refactor-reset 2024-02-25 16:41:40 +00:00
Marek Wolan
c115095157 Fix router from config using wrong method 2024-02-25 16:17:12 +00:00
Chris McCarthy
52677538a8 #2238 - Tidied up code, added more docstrings, and implemented suggestions from PR. 2024-02-23 15:12:46 +00:00
Marek Wolan
f933341df5 eod commit 2024-02-23 10:06:48 +00:00
Chris McCarthy
771a68dccb #2238 - Implement NMNE detection and logging in NetworkInterface.
- Enhance NicObservation for detailed NMNE event monitoring.
- Add nmne_config options to simulation settings for customizable NMNE capturing.
- Update documentation and tests for new NMNE features and simulation config.
2024-02-22 22:43:14 +00:00
Marek Wolan
8f85555709 Merged PR 278: Enable the red agent to vary its start node
## Summary
- Made the data manipulation red agent be able to choose between the two clients to start operating on
- changed the attacker name in the config to 'data_manipulation_attacker' to because it is no longer tied to any client
- Updated the documentation notebook accordingly.
- Fixed a bug where the database client made a new connection every time it sent a SQL query (it tries to reuse its most recent one instead)
- Fixed a bug where link loads were not being cleared between episodes (?)

**warning** - the green agents are not working properly after reset right now, but I'm gonna fix this in the next ticket where I refactor episode reset.

## Test process
- unit tests pass
- UC2 notebook passes with both clients. (currently this doesn't work after an episode reset, but the very next thing I'm gonna work on is refactoring the reset, so I don't want to waste time fixing this.)

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [x] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [n] attended to any **TO-DOs** left in the code

Related work items: #2232
2024-02-20 20:22:20 +00:00
Marek Wolan
72f4cc0a50 Remove reset methods from most classes 2024-02-20 16:56:25 +00:00
Marek Wolan
f82506023b Delete set_original_state method definitions 2024-02-20 16:29:27 +00:00
Marek Wolan
64b9ba3ecf Make environment reset reinstantiate the game 2024-02-20 16:21:03 +00:00
Marek Wolan
88f8e9cb42 Add todo comment. 2024-02-20 12:09:32 +00:00
Marek Wolan
f7c1da3118 Update MARL config. 2024-02-20 12:06:30 +00:00
Marek Wolan
76db5dbaa2 Update changelog 2024-02-20 12:05:02 +00:00
Marek Wolan
4a3c66bdc6 Clear notebook code cells. 2024-02-20 12:04:07 +00:00
Marek Wolan
701781b23e Clear link load in new timestep 2024-02-20 11:05:09 +00:00
Marek Wolan
945db1341b Make database client try to use most recent connection instead of generating new one 2024-02-20 11:04:53 +00:00
Czar Echavez
23a56ca59f Merged PR 270: #2258: DoSBot cannot be added via configuration file
## Summary
- Added ability to set DoSBot in nodes via config file
- Fixed missing configuration items in other services/applications
- Created a variable which contains the list of applications and services
- Moved `DatabaseClient` from list of services to list of applications

## Test process
Created a configuration file where a node has all services and applications installed, this is then used by an integration test which loads the configuration file. The test checks to make sure that the configuration file is correctly read and the Service and Application objects are properly created

## Checklist
- [X] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2258
2024-02-15 09:36:08 +00:00
Nick Todd
0af2eef89c Merged PR 274: Add INSERT query functionality into database service.
## Summary
Add support for SQL INSERT query in database_service.py.

## Test process
Updated test_database_on_node.py to test for database INSERTions.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2306
2024-02-14 16:51:25 +00:00
Nick Todd
8520f22e22 2306: Updated documentation 2024-02-14 13:35:08 +00:00
Nick Todd
4a38672fea 2306: Handle INSERT query 2024-02-14 13:18:20 +00:00
Nick Todd
07a934ab66 2306: Update tests to verify INSERT query. 2024-02-14 12:00:08 +00:00
Czar Echavez
98a4fbecb5 Merge remote-tracking branch 'origin/dev' into feature/2258-dosbot-cannot-be-added-via-configuration-file 2024-02-13 16:04:02 +00:00
Christopher McCarthy
9835b31dc5 Merged PR 272: #2205 - Firewall Node
## Description:

This pull request introduces the Firewall class and extends the ACLRule functionality within PrimAITE to provide comprehensive network traffic management and security capabilities. These enhancements enable detailed control over data flow through network simulations, mimicking real-world firewall operations and ACL configurations. The updates focus on the addition of a Firewall node that extends the Router class functionalities and the enhancement of ACLRule to support IP ranges through wildcard masking, thus offering granular traffic filtering based on IP addresses, protocols, ports, and more.

## Key Features:

**Firewall Class:** A new class that extends the Router class, incorporating firewall-specific logic for inspecting, directing, and filtering traffic between the internal, external, and DMZ (De-Militarized Zone) network interfaces. The Firewall class supports configuring network interfaces and applying Access Control Lists (ACLs) for inbound and outbound traffic control.

**Enhanced ACLRule:** The ACLRule class has been updated to support IP ranges using wildcard masking. This allows for more flexible rule definitions, enabling users to specify broad network ranges or individual IP addresses in ACL rules.

**Comprehensive ACL Configuration:** Six distinct ACLs (internal inbound, internal outbound, DMZ inbound, DMZ outbound, external inbound, and external outbound) provide meticulous control over traffic flow, ensuring robust network security. Examples included in the documentation illustrate how to configure ACLs for common scenarios, such as blocking external threats, permitting specific services, and restricting access to sensitive internal resources.

**Intuitive Interface and ACL Management:** Simplified methods for configuring firewall interfaces and ACL rules enhance usability. The Firewall class offers intuitive functions for rule management, including adding, removing, and listing ACL rules.

**Detailed Documentation and Examples:** Accompanying the code updates, comprehensive documentation and example configurations are provided, detailing the use and configuration of the Firewall node and ACL rules within PrimAITE simulations.

## Impact:

The introduction of the Firewall class and the enhancement of ACLRule significantly broaden PrimAITE's capabilities for simulating realistic network security scenarios. Users can now accurately model the behavior of firewalls in their network simulations, applying complex ACLs to control traffic flow and enforce security policies. This update enables more detailed network security analyses, teaching, and experimentation within the PrimAITE environment.

## Test process
Extensive unit tests have been added to cover the new functionality, ensuring reliability and correctness. Tests include scenarios for firewall configuration, ACL rule application, traffic filtering based on various criteria, and interaction between different network zones.

## Checklist
- [X] PR is linked to ...
2024-02-13 13:56:56 +00:00
Chris McCarthy
7b64d99a63 #2205 - Final suggestions from PR 2024-02-13 12:56:41 +00:00
Chris McCarthy
426c0a6682 2205 - Slimmed down the capability of the wireless router for now 2024-02-13 10:18:06 +00:00
Nick Todd
a6f3c458d5 Merged PR 273: 2297: Convert NTP Client and Server to UDP
## Summary
Replaced references to TCP with UDP in NTP client and server code. Updated docs.

## Test process
Re-ran and passed existing tests.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

2297: Convert NTP Client and Server to UDP

Related work items: #2297
2024-02-13 09:43:09 +00:00
Nick Todd
4c66d2b252 2297: Change missed reference TCP to UDP. 2024-02-12 17:24:28 +00:00
Nick Todd
697e53def8 2297: Doc update. 2024-02-12 17:12:59 +00:00
Nick Todd
fa08e53b15 2297: Convert NTP Client and Server to UDP 2024-02-12 17:01:53 +00:00
Chris McCarthy
add09a0280 #2205 - Tidied up interface creation and applied some suggestions from PR 2024-02-12 14:08:55 +00:00
Chris McCarthy
cfd64333e2 #2205 - Added wireless router tests and documentation. Refactored some code based on PR suggestions. 2024-02-12 12:31:08 +00:00
Czar Echavez
7beacfd95f #2258: missing some configuration items + added more tests 2024-02-12 11:41:55 +00:00
Czar Echavez
da92d74236 #2258: remove unnecessary ntp server check 2024-02-12 09:01:30 +00:00
Czar Echavez
66a3e8d02c Merge remote-tracking branch 'origin/dev' into feature/2258-dosbot-cannot-be-added-via-configuration-file 2024-02-12 08:57:47 +00:00
Chris McCarthy
9df7ceed3d #2205 - feat: Implement AirSpace and WirelessRouter for Enhanced Network Simulations
This commit introduces the AirSpace and WirelessRouter classes, expanding the PrimAITE's capabilities to simulate wireless networking environments. The AirSpace class manages wireless communications, ensuring seamless transmission across different frequencies. Meanwhile, the WirelessRouter class integrates both wired and wireless networking functionalities.
2024-02-10 23:44:08 +00:00
Chris McCarthy
a8c1e2b9d9 #2205 - Fixed ACLRule.is_permitted function by returning a bool that indicates whether the rule was matched or not to allow the AccessControlList to know whether to pay attention to the rule or not when it's iterating over them. 2024-02-10 21:32:13 +00:00