Commit Graph

650 Commits

Author SHA1 Message Date
Christopher McCarthy
69c3f97cd9 Merged PR 178: Database Client/Server Simulation
## Summary
This pull request focuses on implementing key functionalities needed for network frame processing and database interactions. The primary changes are:

1. **Internal Frame Processing:** The logic has been implemented in various components like `NIC`, `Node`, `SessionManager`, and `SoftwareManager`. These changes enable the system to process incoming and outgoing network frames in a structured manner.
2. **Database Service and Client:** The `DatabaseService` simulates a SQL database server, while the `DatabaseClient` provides a client interface for connecting to this service. These functionalities have been built and integrated into the existing architecture.
3. **Networking and Communication:** Tests have been added to confirm that database queries can be sent over the network, demonstrating end-to-end functionality.

## Commits

- #1816 Simplified a bunch of stuff in the file system in prep for services and applications. Started adding the database logic. Waiting for the software manager/session manager work from another tick. Merge branch 'dev' into feature/1816_Database-Service-(Network-and-User-Interaction)
- #1816 Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- #1816 DatabaseService now uses the send function when responding.
- #1816 Added database client. Installed the database client on the Web Server node in the UC2 network. Updated the integration test to query the DB server using the DB client.
- #1816 Added full documentation on the database client/server, and the internal frame processing process
- #1816 Fixed tests. Used node and link added number (id) in observation space.

## Test process
For testing these functionalities, the following steps were taken:

1. **Unit Tests:** Tests have been written to confirm that database queries can be sent over the network successfully.
2. **Integration Tests:** Manually tested the frame processing flow from NIC to Service/Application, ensuring the functionality behaves as expected.
3. **Database Queries** Executed sample SQL queries using the `DatabaseClient` to make sure it interacts correctly with the `DatabaseService`.

## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [ ] I have run **pre-commit** checks for code style

Related work items: #1816
2023-09-12 13:31:08 +00:00
Chris McCarthy
695b3ceab4 #1816 - Updated the DataManipulationBot to subclass DatabaseClient. Extended logging. Dropped the Link loading logging as it was clogging up the terminal output. 2023-09-11 16:15:03 +01:00
Chris McCarthy
f19dc9892b #1816 - Fixed tests. Used node and link added number (id) in observation space. 2023-09-11 11:31:03 +01:00
Chris McCarthy
388176b8bd #1816 - Added full documentation on the database client/server, and the internal frame processing process 2023-09-11 09:30:40 +01:00
Chris McCarthy
b1d8666c16 #1816 - Added database client. Installed the database client on the Web Server node in the UC2 network. Updated the integration test to query the DB server using the DB client. 2023-09-08 16:50:49 +01:00
Chris McCarthy
ceac89e778 #1816 - DatabaseService now uses the send function when responding. 2023-09-08 10:15:26 +01:00
Chris McCarthy
2f744af34e #1816 - Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- Implemented a basic sim DB that matches UC2 data manipulation DB in IY.
- Added a test that confirms DB queries can be sent over the network.
2023-09-06 22:26:23 +01:00
Chris McCarthy
6b41bec32a #1816 - Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- Implemented a basic sim DB that matches UC2 data manipulation DB in IY.
- Added a test that confirms DB queries can be sent over the network.
2023-09-06 22:01:51 +01:00
Chris McCarthy
65b027bc06 Merge branch 'dev' into feature/1816_Database-Service-(Network-and-User-Interaction) 2023-09-06 13:37:06 +01:00
Czar Echavez
6b9928c9b9 Merged PR 175: Red Service Data Manipulator Bot
## Summary
Implementation of the Data Manipulation Bot. This service sends a SQL query payload to the database server (or a given machine IP and port)

## Test process
Added a test
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/175?_a=files&path=/tests/unit_tests/_primaite/_simulator/_system/_services/_red_services/test_data_manipulator_service.py

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [X] I have update the **change log**
- [x] I have run **pre-commit** checks for code style

Related work items: #1814
2023-09-06 11:40:29 +00:00
Chris McCarthy
7c157d27d7 #1800 - Simplified a bunch of stuff in the file system in prep for services and applications. Started adding the database logic. Waiting for the software manager/session manager work from another ticket. 2023-09-06 11:35:41 +01:00
Czar Echavez
597c7664bc #1814: update changelog 2023-09-06 11:19:30 +01:00
Czar Echavez
d503e51c2d #1814: Remove hardcoded values + added test + remove unnecessary private parent attribute 2023-09-06 11:12:03 +01:00
Czar Echavez
c349bb4484 #1814: initial implementation of data manipulator service 2023-09-05 17:14:47 +01:00
Marek Wolan
0493c2062c Merged PR 173: Database service (without networking)
## Summary
- add a database service
- change how `SimComponent` adds actions to allow inheritance of actions
- add service-based actions, like start, stop, pause, and compromise

## Test process
New test cases were added.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [x] I have run **pre-commit** checks for code style

Related work items: #1801
2023-09-04 19:41:17 +00:00
Chris McCarthy
1385adf3fe Merge remote-tracking branch 'devops/dev' into feature/1801-Database
# Conflicts:
#	src/primaite/simulator/network/container.py
#	src/primaite/simulator/network/hardware/base.py
2023-09-04 19:45:29 +01:00
Christopher McCarthy
0892a976fd Apply suggestions from code review 2023-09-04 18:37:05 +00:00
Christopher McCarthy
c547f12f4e Merged PR 158: Sync with the downstream GitHub repo
DSTL requested some changes to docs.

Related work items: #1711
2023-09-04 18:21:19 +00:00
Christopher McCarthy
482a5fc5e9 Merged PR 174: Router Node
**_*Disclaimer, I used AI to write this PR description for me, you know, for efficiency._**

## Summary
In this pull request, we've made several key enhancements and fixes to improve the routing and network simulation functionalities of our codebase. At the heart of this PR is the introduction of a `Router` class, complete with `ACLRule` and `RouteTableEntry` classes to facilitate easier management of routes and permissions. We've also fixed a lingering issue with the `.ping` functionality, ensuring that it properly checks for replies before returning a boolean value.

To make our `Router` class more inspectable, `.show()` methods have been added which can output information in both plain text and markdown formats. Additionally, we've removed the `gateway` field from the `NIC` class and relocated it to the `Node` class. This move allows each `Node` to have a single default gateway. To further refine the system, we've added `Computer` and `Server` classes, and updated our restructured text formatted docstrings and documentation. Finally, a UC2 network has been added.

Changes have been fully tested to ensure that the existing functionalities remain intact, and new unit tests have been added to verify routing between different subnets. The code has been synced with the dev branch.

## Test process
We've added new unit tests and integration tests to check the routing capabilities when a router is placed between different subnets. These tests specifically focus on the ping functionality to verify that it works as expected after the changes. Existing tests were also run to make sure no existing functionality was broken.

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [X] I have run **pre-commit** checks for code style

Related work items: #1800
2023-09-04 16:56:41 +00:00
Marek Wolan
289f818266 Apply suggestions from code review 2023-09-04 16:11:47 +00:00
Chris McCarthy
dc9601bf30 Merge remote-tracking branch 'devops/dev' into downstream_github_sync 2023-09-04 16:46:14 +01:00
Chris McCarthy
f3f8d02fcb Merge remote-tracking branch 'github/dev' into downstream_github_sync 2023-09-04 16:45:23 +01:00
Chris McCarthy
596ad20cc6 #1800 - Added better logging and error messages to AccessControlList class. Updated usage of extra following pydantic deprecated warning "pydantic.config.Extra is deprecated, use literal values instead (e.g. extra='allow')" 2023-09-04 16:44:29 +01:00
Chris McCarthy
ccad5ba8a3 #1800 - Ran pre-commit 2023-09-04 16:34:55 +01:00
Chris McCarthy
5981bc7255 Merge remote-tracking branch 'devops/feature/1800_model-a-basic-router-subclass-of-node' into feature/1800_model-a-basic-router-subclass-of-node 2023-09-04 14:58:52 +01:00
Chris McCarthy
3075d1985b #1800 - Renamed all ip fields so that they're post-fixed with ip_address 2023-09-04 14:58:34 +01:00
Christopher McCarthy
d9feb67e02 Apply suggestions from code review 2023-09-04 11:20:40 +00:00
Chris McCarthy
05959e5408 #1800 - Moved the Switch code to a dedicated switch.py module.
- Added more switch tests.
- Updated ACL tests to use router acl.
- Updated more docs.
- Moved the Jupyter notebooks to _package_data and fixed up the setup to move all notebooks to ~/primaite/notebooks/example_notebooks.
2023-09-04 12:14:24 +01:00
Chris McCarthy
5111affeeb #1800 - Added more docstrings and rst docs.
- Extended the .show functionality to enable markdown format too.
2023-09-01 16:58:21 +01:00
Marek Wolan
d55fd835c7 Merge remote-tracking branch 'origin/dev' into feature/1801-Database 2023-08-31 14:56:31 +01:00
Marek Wolan
61fa83a00d Fix failing test 2023-08-31 14:55:14 +01:00
Chris McCarthy
89ad22aceb #1800 - Synced with dev.
- Added the UC2 network.
- Added a Computer class.
2023-08-31 13:35:56 +01:00
Chris McCarthy
415ee3b3a7 Merge branch 'dev' into feature/1800_model-a-basic-router-subclass-of-node 2023-08-31 11:39:21 +01:00
Marek Wolan
bd5aacaf0c Remove todo comments 2023-08-31 11:32:11 +01:00
Marek Wolan
f60f775f03 Improve logging 2023-08-31 11:27:52 +01:00
Marek Wolan
7759c178bb Add logging and service restarting 2023-08-31 11:20:16 +01:00
Marek Wolan
3368376596 Merged PR 172: Methods for adding, removing, connecting and disconnecting nodes in the network.
## Summary
- Add public API for adding, removing, connecting and disconnecting nodes in the network.
- Introduce parent of simcomponent to keep track of parent as per this diagram ![Component ownership tree - Page 1 (2).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/172/attachments/Component%20ownership%20tree%20-%20Page%201%20%282%29.png)
- Update tutorial notebook with new way of doing things.

## Test process
Unit and integration tests written and passing.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [x] I have run **pre-commit** checks for code style

Related work items: #1798
2023-08-31 10:04:11 +00:00
Chris McCarthy
e73d7f49d6 #1800 - Fixed routing and processing of ICMP packets in the Router class 2023-08-31 11:03:38 +01:00
Marek Wolan
62be66205c Fix unit tests 2023-08-31 10:57:45 +01:00
Chris McCarthy
1bf51c7741 #1800 - Added ACL and routing classes.
- Added .show() methods to new router classes to enable inspection of the components as you would a real router.
- Removed gateway from the NIC and added default_gateway to Node so that Node has a single default gateway.
- Added some routing tests to check that ping can be performed when router between subnets.
2023-08-30 21:38:55 +01:00
Marek Wolan
40d3e04e64 Move init action manager function to the top 2023-08-29 14:33:28 +01:00
Marek Wolan
f0b82cbdfb Add ability to uninstall service 2023-08-29 14:15:49 +01:00
Marek Wolan
9c8b7a7b7a Merge remote-tracking branch 'origin/feature/1789-add-convenience-methods-to-network-object' into feature/1801-Database 2023-08-29 13:22:14 +01:00
Marek Wolan
94325d1fde Add Install method to software. 2023-08-29 13:21:34 +01:00
Marek Wolan
7b61322e70 Add service actions 2023-08-29 12:34:41 +01:00
Marek Wolan
1eff41c786 Update docs based on new action options 2023-08-29 11:10:29 +01:00
Marek Wolan
319e87d200 Make changes to the way actions work 2023-08-28 22:34:20 +01:00
Marek Wolan
6e602aa151 Fix unit tests by removing warning checks 2023-08-25 17:56:05 +01:00
Marek Wolan
ae6e835955 Apply suggestions from code review. 2023-08-25 15:58:07 +01:00
Marek Wolan
a9e969aa13 init work on database 2023-08-25 15:29:53 +01:00