Commit Graph

696 Commits

Author SHA1 Message Date
Czar.Echavez
4699e87ea8 Merge branch 'dev' into feature/1943-service-web-server 2023-10-04 16:35:23 +01:00
Czar Echavez
c71e989ac6 Merged PR 187: Database Service backup using FTP
## Summary
- Used FTP changes to allow the database service to create a copy of the database.db file in the backup server
- Used FTP changes to allow the database service to restore the database.db file from a copy made in the backup server

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

Related work items: #1752, #1796, #1816, #1866, #1916
2023-10-04 14:06:32 +00:00
Czar.Echavez
97f0267539 #1796: apply PR suggestions + fix tests 2023-10-04 11:33:18 +01:00
Czar.Echavez
82da21b073 \#1943:
- changelog added
- added documentation + example of using web server + web browser
- extended web server so that it also accepts ip addresses
- web server can differentiate between a normal page request and one that propagates into a DB request
- rename WebServerService -> WebServer
2023-10-03 16:56:35 +01:00
Czar.Echavez
4b5a73bd32 #1943: web server + client + tests + a few improvements to syslogging 2023-10-03 14:59:48 +01:00
Czar.Echavez
fdebfce406 #1796: Fix test + making the restore test better 2023-10-02 12:14:59 +01:00
Chris McCarthy
84405d7ed3 #1796 - Added docstring to the test_uc2_data_manipulation_scenario.py 2023-09-29 20:19:26 +01:00
Chris McCarthy
3dc8a0f222 #1796 - Made the FTP copy real files. Hardcoded the DatabaseService backup folder and filename. Added db restore and final query check to the data manipulation e2e test. 2023-09-29 20:14:42 +01:00
Czar.Echavez
bca3e6344e #1796: documentation 2023-09-28 14:09:32 +01:00
Czar.Echavez
6202d320a6 #1796: Add ability to create and restore database backups + add more sys log messages + remove the link size checks temporarily 2023-09-28 12:23:49 +01:00
Czar.Echavez
c096d06bcd #1796: pre installing system software 2023-09-26 15:14:24 +01:00
Czar Echavez
161d441ad6 Merged PR 185: FTP - client/server
## Summary
Initial implementation of an FTP Client/Server service used to emulate a transfer of files across nodes

## Test process
Added an integration and unit test

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #1752, #1816, #1916
2023-09-26 13:38:42 +00:00
Czar.Echavez
79615243e4 #1916: Added example usage for FTP 2023-09-26 12:09:41 +01:00
Czar.Echavez
0140fe7c48 #1916: fix a problem with process_ftp_command method 2023-09-25 15:59:31 +01:00
Czar.Echavez
2520b67889 #1916:
- Added FTP to changelog
- Added FTP to documentation
- Added documentation in code
- Clean up of methods - prevent repeats of the same code
2023-09-25 14:31:57 +01:00
Czar.Echavez
98ca71aaba Merge branch 'dev' into feature/1916-ftp-client-server 2023-09-25 10:40:51 +01:00
Czar.Echavez
2c234ab67a #1916: Setting up a connected states + added tests + error states for if service is interacted with when not running 2023-09-22 15:38:01 +01:00
Czar.Echavez
58edb6d3e4 #1916: Created FTP superclass + working retrieve file method for FTP 2023-09-21 15:13:30 +01:00
Czar.Echavez
2e76b3f162 #1916: FTP client STOR request to FTP server 2023-09-20 16:23:35 +01:00
Marek Wolan
fd1a4403f4 Merged PR 179: Updated pull_request_template.md
## Summary
*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?*

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [ ] I have run **pre-commit** checks for code style

Updated pull_request_template.md

Related work items: #1866
2023-09-19 12:43:17 +00:00
Czar.Echavez
f913294058 #1916: moved services into their own subdirectories 2023-09-18 14:20:19 +01:00
Czar Echavez
f180c6f680 Merged PR 177: DNS Server and Client
## Summary
Carrying on where Sunil left the task. Added more functionality to the DNS.
- DNS Server can keep a list of domain names that the DNS Client can request
- DNS Client has a cache of domain names that are valid from the DNS Server

## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/177?_a=files&path=/tests/unit_tests/_primaite/_simulator/_system/_services/test_dns.py

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [X] I have update the **change log**
- [x] I have run **pre-commit** checks for code style

Related work items: #1752
2023-09-18 10:31:16 +00:00
Chris McCarthy
939de40f1e #1752 - Moved dns_server ip address from the NIC to the Node. Updated the arcd_uc2_network so that clients and servers have a dns server. Added sys_log entries for DNSServer and DNSClient. MAde the DNSServer always rend a reply, but for the resolved IP address to be empty if it cannot be resolved. 2023-09-18 10:25:26 +01:00
Czar.Echavez
b1e46b4f9e #1752: Apply suggestions from PR review 2023-09-14 20:08:06 +01:00
Czar Echavez
98e103a984 #1752: added documentation for DNS Client and Server 2023-09-13 09:48:38 +01:00
Czar Echavez
b0478f4e88 #1752: added positions to ACL rules for UC2 network to prevent rules being overwritten 2023-09-13 08:46:22 +01:00
Czar Echavez
f29477a50a Merge branch 'dev' into feature/1752-dns-server-and-client 2023-09-12 14:34:15 +01:00
Christopher McCarthy
69c3f97cd9 Merged PR 178: Database Client/Server Simulation
## Summary
This pull request focuses on implementing key functionalities needed for network frame processing and database interactions. The primary changes are:

1. **Internal Frame Processing:** The logic has been implemented in various components like `NIC`, `Node`, `SessionManager`, and `SoftwareManager`. These changes enable the system to process incoming and outgoing network frames in a structured manner.
2. **Database Service and Client:** The `DatabaseService` simulates a SQL database server, while the `DatabaseClient` provides a client interface for connecting to this service. These functionalities have been built and integrated into the existing architecture.
3. **Networking and Communication:** Tests have been added to confirm that database queries can be sent over the network, demonstrating end-to-end functionality.

## Commits

- #1816 Simplified a bunch of stuff in the file system in prep for services and applications. Started adding the database logic. Waiting for the software manager/session manager work from another tick. Merge branch 'dev' into feature/1816_Database-Service-(Network-and-User-Interaction)
- #1816 Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- #1816 DatabaseService now uses the send function when responding.
- #1816 Added database client. Installed the database client on the Web Server node in the UC2 network. Updated the integration test to query the DB server using the DB client.
- #1816 Added full documentation on the database client/server, and the internal frame processing process
- #1816 Fixed tests. Used node and link added number (id) in observation space.

## Test process
For testing these functionalities, the following steps were taken:

1. **Unit Tests:** Tests have been written to confirm that database queries can be sent over the network successfully.
2. **Integration Tests:** Manually tested the frame processing flow from NIC to Service/Application, ensuring the functionality behaves as expected.
3. **Database Queries** Executed sample SQL queries using the `DatabaseClient` to make sure it interacts correctly with the `DatabaseService`.

## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [ ] I have update the **change log**
- [ ] I have run **pre-commit** checks for code style

Related work items: #1816
2023-09-12 13:31:08 +00:00
Czar Echavez
8b6bc84321 #1752: simplifying the DNS implementation - switch to TCP + fixing the DNS integration test 2023-09-12 13:37:11 +01:00
Marek Wolan
275115f4dd Updated pull_request_template.md 2023-09-12 12:12:38 +00:00
Czar Echavez
fb96ef18c0 #1752: remove unnecessary changes 2023-09-12 09:32:28 +01:00
Czar Echavez
1a81285b76 #1752: Added send+receive functionality for DNS client and server + tests + added simulation_output to gitignore 2023-09-12 08:46:07 +01:00
Czar Echavez
ee730d4ab0 Merge branch 'feature/1816_Database-Service-(Network-and-User-Interaction)' into feature/1752-dns-server-and-client 2023-09-11 16:22:18 +01:00
Chris McCarthy
695b3ceab4 #1816 - Updated the DataManipulationBot to subclass DatabaseClient. Extended logging. Dropped the Link loading logging as it was clogging up the terminal output. 2023-09-11 16:15:03 +01:00
Czar Echavez
4c67365d4f Merge branch 'feature/1816_Database-Service-(Network-and-User-Interaction)' into feature/1752-dns-server-and-client 2023-09-11 15:23:39 +01:00
Chris McCarthy
f19dc9892b #1816 - Fixed tests. Used node and link added number (id) in observation space. 2023-09-11 11:31:03 +01:00
Czar Echavez
5a101d5426 Merge branch 'feature/1816_Database-Service-(Network-and-User-Interaction)' into feature/1752-dns-server-and-client 2023-09-11 09:50:16 +01:00
Chris McCarthy
388176b8bd #1816 - Added full documentation on the database client/server, and the internal frame processing process 2023-09-11 09:30:40 +01:00
Czar Echavez
5cd1e2edf0 Merge branch 'feature/1816_Database-Service-(Network-and-User-Interaction)' into feature/1752-dns-server-and-client 2023-09-11 08:58:13 +01:00
Chris McCarthy
b1d8666c16 #1816 - Added database client. Installed the database client on the Web Server node in the UC2 network. Updated the integration test to query the DB server using the DB client. 2023-09-08 16:50:49 +01:00
Czar Echavez
97a38efc27 Merge branch 'feature/1816_Database-Service-(Network-and-User-Interaction)' into feature/1752-dns-server-and-client 2023-09-08 10:41:03 +01:00
Chris McCarthy
ceac89e778 #1816 - DatabaseService now uses the send function when responding. 2023-09-08 10:15:26 +01:00
Czar Echavez
6f2f23e04f #1752: update changelog 2023-09-07 15:59:46 +01:00
Czar Echavez
47dd23311b #1752: added more functionality to DNS client and server + tests 2023-09-07 15:45:37 +01:00
Chris McCarthy
2f744af34e #1816 - Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- Implemented a basic sim DB that matches UC2 data manipulation DB in IY.
- Added a test that confirms DB queries can be sent over the network.
2023-09-06 22:26:23 +01:00
Chris McCarthy
6b41bec32a #1816 - Added the final pieces of the puzzle to get data up from NIC → session manager → software manager → service.
- Implemented a basic sim DB that matches UC2 data manipulation DB in IY.
- Added a test that confirms DB queries can be sent over the network.
2023-09-06 22:01:51 +01:00
Czar Echavez
2cb0c238c9 Merge branch 'dev' into feature/1752-dns-server-and-client 2023-09-06 13:42:07 +01:00
Chris McCarthy
65b027bc06 Merge branch 'dev' into feature/1816_Database-Service-(Network-and-User-Interaction) 2023-09-06 13:37:06 +01:00
Czar Echavez
6b9928c9b9 Merged PR 175: Red Service Data Manipulator Bot
## Summary
Implementation of the Data Manipulation Bot. This service sends a SQL query payload to the database server (or a given machine IP and port)

## Test process
Added a test
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/175?_a=files&path=/tests/unit_tests/_primaite/_simulator/_system/_services/_red_services/test_data_manipulator_service.py

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have written/updated **design docs** if this PR implements new functionality
- [X] I have update the **change log**
- [x] I have run **pre-commit** checks for code style

Related work items: #1814
2023-09-06 11:40:29 +00:00
Chris McCarthy
7c157d27d7 #1800 - Simplified a bunch of stuff in the file system in prep for services and applications. Started adding the database logic. Waiting for the software manager/session manager work from another ticket. 2023-09-06 11:35:41 +01:00