2675 Commits

Author SHA1 Message Date
Marek Wolan
e609f8eb50 Fix misconfiguration in uc2 config and session 2023-11-24 14:56:17 +00:00
Marek Wolan
82116e6c92 Merged PR 218: Get Ray agents working
## Summary
- PrimaiteSession class was separated into 'game' and 'session' classes. Game manages the interfacing of agents and the simulator. Session manages the environment, IO, and RL algorithms, as well as the game.
- Add ray single and multi agent support
- Add environments that can be interacted with directly so that users who want to manually configure RL policies can do so.
- Add notebooks demonstrating manual environment interfacing.

## Test process
Unit tests added, all pass. Example notebooks run to demonstrate compatibility with SB3 and Ray.

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

![PrimAITE High Level Design - Page 1 (4).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/218/attachments/PrimAITE%20High%20Level%20Design%20-%20Page%201%20%284%29.png)

Related work items: #1970, #1971
2023-11-24 13:57:54 +00:00
Marek Wolan
64c7dd3c84 Skip slow tests for now. 2023-11-24 12:03:46 +00:00
Chris McCarthy
dfb08b8cf3 #1859 - DB query now returns false if the query isn't ran due to the node being off 2023-11-24 11:52:33 +00:00
Nick Todd
dd351f8143 #2068: Remove duplicated index entries. 2023-11-24 11:21:25 +00:00
Jake Walker
ff8b773c10 Database Manipulation Bot bug fixes 2023-11-24 11:10:34 +00:00
Marek Wolan
d8975078b3 Fix game reset test. 2023-11-24 10:50:10 +00:00
Jake Walker
4e104a4d99 Update data manipulation bot 2023-11-24 10:33:19 +00:00
Jake Walker
178d911be0 Update data manipulation bot 2023-11-24 10:05:36 +00:00
Jake Walker
92dabe59f7 Fix data manipulation bot configuration 2023-11-24 10:04:19 +00:00
Marek Wolan
bfd5f054dc Merge 'origin/dev' into feature/1971-ray-agents-2 2023-11-24 09:50:37 +00:00
Marek Wolan
abba1ef86b Remove hardcoded checkpoint frequency in rllib 2023-11-24 09:37:26 +00:00
Marek Wolan
6754dbf541 Remove GATE and fix a few spelling mistakes. 2023-11-24 09:28:50 +00:00
Jake Walker
b13a9d3daf Add application execution action for data manipulation bot 2023-11-24 09:25:55 +00:00
Marek Wolan
50c9ef16cb Move policy module into session 2023-11-24 09:18:18 +00:00
Marek Wolan
bd109a7cfc Complete session->game rename refactor 2023-11-24 09:14:55 +00:00
Christopher McCarthy
0caeb60e01 Merged PR 216: #2034 - Implemented the Simulation reset functionality
## Summary
Implemented the Simulation reset functionality by doing a deepcopy of the Simulation object inside the PrimaiteSession upon instantiation and assigning it to a `_simulation_initial_state` variable. Then when reset called, the sessions `simulation` variable is assigned the value of the `_simulation_initial_state` variable.

## Test process
Added a test that uninstalls a service before performing a reset then checks that the service reappears.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

#2034 - Implemented the Simulation reset functionality by doing a deepcopy of the Simulation object inside the PrimaiteSession upon instantiation. Added a test that uninstalls a service before performing a reset then checks that the service reappears.

Related work items: #2034
2023-11-23 22:33:12 +00:00
Czar Echavez
8aa743188f #2064: fix layout of test so it passes in pipeline 2023-11-23 22:28:08 +00:00
Czar Echavez
2ce03e0262 #2064: turn on everything when node is turned on 2023-11-23 22:10:53 +00:00
Czar Echavez
f0fc6518a0 #2064: add handling of offline service to dns, ftp and database 2023-11-23 21:48:11 +00:00
Chris McCarthy
dfe07ad926 #2034 - Fixed deepcopy reference issue 2023-11-23 21:25:52 +00:00
Czar Echavez
bd6c27244c #2064: Edited services and applications to handle when they are shut down 2023-11-23 19:49:03 +00:00
Nick Todd
3dfd7a2e14 #2068: Fix malformed Windows path 2023-11-23 17:57:51 +00:00
Nick Todd
3894a9615d #2068: Replace refs to OpenAI Gym with Gymnasium 2023-11-23 17:42:26 +00:00
Nick Todd
47112aafcf #2068: Removed references to ARCD GATE 2023-11-23 16:19:39 +00:00
Jake Walker
5f1a5af1b4 Add data manipulation bot action manager 2023-11-23 16:06:19 +00:00
Nick Todd
87dde6ee0b #2042: Test tidying changes. 2023-11-23 15:55:58 +00:00
Jake Walker
c93705867f Move configuration from agent to data manipulation bot 2023-11-23 15:53:47 +00:00
Christopher McCarthy
76939fb8e8 Apply suggestions from code review 2023-11-23 15:45:22 +00:00
Chris McCarthy
efeaa4c1cc #2034 - Implemented the Simulation reset functionality by doing a deepcopy of the Simulation object inside the PrimaiteSession upon instantiation. Added a test that uninstalls a service before performing a reset then checks that the service reappears. 2023-11-23 15:31:06 +00:00
Marek Wolan
4ee29efd1f Merged PR 213: Replace SQLite database implementation with a simulation
## Summary
Remove SQLLite and real sql queries and replace them with `SELECT` and `DELETE` queries that interact with the node's file system and service operating status.

## Test process
Existing unit tests pass after logic was changed.

## Checklist
- [x] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [x] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

Related work items: #1971
2023-11-23 10:18:06 +00:00
Nick Todd
8584fa8f51 # 2041: Minor test changes 2023-11-23 10:04:52 +00:00
Marek Wolan
dd63563ba1 Update doc page on primaite session. 2023-11-23 03:07:39 +00:00
Marek Wolan
f1f516c51a Add multi agent session test 2023-11-23 02:51:31 +00:00
Marek Wolan
8a2279c6cb Update end to end tests after session changes 2023-11-23 01:40:27 +00:00
Marek Wolan
14ae8be5e2 Update session after it was split from game 2023-11-23 00:54:19 +00:00
Marek Wolan
1fd5298fc5 Fix multi agent system 2023-11-22 20:22:34 +00:00
Jake Walker
061e508187 Add repeat option to data manipulation bot 2023-11-22 16:24:17 +00:00
Nick Todd
006a37d268 #2042: extract code into request_time() method. 2023-11-22 14:40:44 +00:00
Nick Todd
3f76e09521 #2042: remove apply_timestep() return value 2023-11-22 14:13:50 +00:00
Marek Wolan
9070fb44d4 Check that ray single agent training works 2023-11-22 13:26:29 +00:00
Marek Wolan
b81dd26b71 Add Ray env class 2023-11-22 13:12:08 +00:00
Marek Wolan
1138644a4b Update to make things work with new layout 2023-11-22 12:59:33 +00:00
Marek Wolan
afd64e4674 Separate game, environment, and session 2023-11-22 11:59:25 +00:00
Nick Todd
dd7c2b05f8 #2041: Add RST doc 2023-11-22 08:54:39 +00:00
Nick Todd
984d165364 #2041: Fix long line 2023-11-21 17:24:50 +00:00
Nick Todd
eb2e37429a #2042: Add time attribute 2023-11-21 17:24:24 +00:00
Jake Walker
aa65c53a95 Pass probability of success through to functions 2023-11-21 15:09:51 +00:00
Jake Walker
48af022963 Run agent at configured timesteps 2023-11-21 13:41:38 +00:00
Nick Todd
243f2dd938 #2041: Update CHANGELOG 2023-11-21 12:11:30 +00:00