Commit Graph

858 Commits

Author SHA1 Message Date
Marek Wolan
f1f516c51a Add multi agent session test 2023-11-23 02:51:31 +00:00
Marek Wolan
8a2279c6cb Update end to end tests after session changes 2023-11-23 01:40:27 +00:00
Marek Wolan
14ae8be5e2 Update session after it was split from game 2023-11-23 00:54:19 +00:00
Marek Wolan
1fd5298fc5 Fix multi agent system 2023-11-22 20:22:34 +00:00
Marek Wolan
9070fb44d4 Check that ray single agent training works 2023-11-22 13:26:29 +00:00
Marek Wolan
b81dd26b71 Add Ray env class 2023-11-22 13:12:08 +00:00
Marek Wolan
1138644a4b Update to make things work with new layout 2023-11-22 12:59:33 +00:00
Marek Wolan
afd64e4674 Separate game, environment, and session 2023-11-22 11:59:25 +00:00
Marek Wolan
66d5612e92 Merge remote-tracking branch 'origin/feature/1972-remove-sqlite' into feature/1971-ray-agents-2 2023-11-18 04:54:17 +00:00
Marek Wolan
fdb48c0ded Update docs for Database 2023-11-18 03:46:24 +00:00
Marek Wolan
7d800cbbc9 Merge remote-tracking branch 'origin/dev' into feature/1972-remove-sqlite 2023-11-18 03:41:22 +00:00
Marek Wolan
e9d2437db8 Merged PR 212: GATE-less agents
## Summary
Remove gate and integrate SB3 agent natively.
This includes:
- agent saving
- checkpoints
- session output directory
- simulation output (such as syslogs) are now in the session output directory
- agent loading
- training progress bar (with `enlighten`)

## Test process
End to end tests that verify that a session is being created correctly with a network
They also check the outputs exist.

## Checklist
- [y] PR is linked to a **work item**
- [upcoming PR] **acceptance criteria** of linked ticket are met
- [y] performed **self-review** of the code
- [loading not tested but need to move onto Ray now] written **tests** for any new functionality added with this PR
- [n] updated the **documentation** if this PR changes or adds functionality
- [n] written/updated **design docs** if this PR implements new functionality
- [n] updated the **change log**
- [y] ran **pre-commit** checks for code style
- [n] attended to any **TO-DOs** left in the code

Related work items: #1971
2023-11-21 09:33:29 +00:00
Marek Wolan
77f3806ba7 Remove real database in favour of simulated 2023-11-18 03:40:08 +00:00
Marek Wolan
9d0a98b221 Apply suggestions from code review 2023-11-17 20:30:07 +00:00
Marek Wolan
5d9d8fe92c merge PR updates from parent branch 2023-11-17 17:59:45 +00:00
Marek Wolan
3fb7bce3ce Get RLLib to stop crashing. 2023-11-17 17:57:57 +00:00
Marek Wolan
c5b4ae45be Remove problematic progress bars 2023-11-17 11:40:36 +00:00
Marek Wolan
6e5e1e6456 Begin rllib 2023-11-17 11:38:29 +00:00
Marek Wolan
5bda952ead Fix sim output 2023-11-17 10:20:26 +00:00
Marek Wolan
ba580b00b4 Improve config validation and fix tests 2023-11-16 16:14:50 +00:00
Marek Wolan
0861663cc1 Add agent loading 2023-11-16 15:40:49 +00:00
Marek Wolan
e52d1fbd45 Add enlighten dependency 2023-11-16 15:29:48 +00:00
Marek Wolan
0b9bdedebd Fix typehints 2023-11-16 15:28:38 +00:00
Marek Wolan
13c49bf3ea Fix session path monkeypatch 2023-11-16 15:19:14 +00:00
Marek Wolan
7545c25a46 Make pytest patch with temporary session dir 2023-11-16 15:11:03 +00:00
Marek Wolan
829500a60f Get sb3 checkpoints saving during training 2023-11-16 14:37:37 +00:00
Marek Wolan
4cc7ba1522 Add ability to save sb3 final model 2023-11-15 16:59:56 +00:00
Marek Wolan
64e8b3bcea Add basic primaite session e2e tests 2023-11-15 16:04:16 +00:00
Marek Wolan
6182b53bfd Fix incorrect number of steps per episode 2023-11-15 14:49:44 +00:00
Marek Wolan
c8f2f193bd Implement agent training with sb3 2023-11-15 12:52:18 +00:00
Marek Wolan
e6ead6e532 Update agent interface to work better with envs 2023-11-14 15:10:07 +00:00
Marek Wolan
1cb54da2dd Remove more GATE stuff 2023-11-13 17:12:50 +00:00
Marek Wolan
08e88e52b0 Begin implementing training loop in session 2023-11-13 16:35:35 +00:00
Marek Wolan
707f2b59af Add SB3 RL agent 2023-11-13 16:08:39 +00:00
Marek Wolan
21c06dbea1 Remove GATE-related code. 2023-11-13 16:04:25 +00:00
Czar Echavez
815bdfe603 Merged PR 206: 1962-Folder and File restore
## Summary
- Implement the folder/file restore
- Split the FileSystem and its classes into their own files so that it is easier to maintain.
- Reimplement how deleted folders and files are stored in a deleted list, so that it is possible to restore later

## Test process
See https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/206?_a=files&path=/tests/unit_tests/_primaite/_simulator/_file_system
- Unit tests written to make sure methods work as intended
- Integration tests with the apply_request method which tests that the agents can send the request to the simulation to affect it

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #1962
2023-11-09 11:55:53 +00:00
Czar.Echavez
6e1f9bd63d #1962: add gate installation for docs build pipeline 2023-11-08 13:00:07 +00:00
Czar.Echavez
be4a467877 #1962: revert pulling from src 2023-11-08 10:48:41 +00:00
Czar.Echavez
f5df1366a7 #1962: merge dev into branch + fix minor diffs + ensure that imports pull from src 2023-11-08 10:36:47 +00:00
Marek Wolan
c060991084 Merged PR 207: Game layer
## Summary
- Remove v2-specific code, documentation, and tests
- Reconnect CLI, to new v3 classes and functions
- Add Game layer
- Minor changes to simulation layer to ensure compatibility with game layer actions and observations

### Game layer
- Add a primaite session, which is the main entry point to running a session. This encompasses agents, the simulation, and a connection with GATE.
- Add an agent interface, random agents, and RL agents.
- Add new observation space management
- Add new action space management
- Add new reward function management

## Test process
Ran primaite session without crash. Majority of integration and unit tests have not been written yet. (sorry)

I know it's not ideal to merge in this not-tested-state, but it doesn't make sense to continue diverging `dev` and `dev-game-layer`.

Instead of reviewing the diff, I would recommend downloading this code and getting familiarised with the new `game` module by reading the code and playing around with it.

Related work items: #1622, #1759, #1760, #1761, #1764, #1765, #1766, #1767, #1768, #1879, #1924, #1986, #1987, #1988, #1989, #1990, #1998
2023-11-07 16:03:43 +00:00
Czar Echavez
729ef2ebac Merged PR 208: #2025: added GATE installation as part of pipeline
## Summary
Installing GATE before running the primaite setup command so that the pipeline passes

## Test process
*How have you tested this (if applicable)?*

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

#2025: added GATE installation as part of pipeline

Related work items: #2025
2023-11-07 15:03:00 +00:00
Czar.Echavez
d4679bb0e3 #2025: Missing character for GATE path 2023-11-07 14:09:22 +00:00
Czar.Echavez
e0d694a736 #2025: added GATE installation as part of pipeline 2023-11-07 14:00:08 +00:00
Marek Wolan
0ea282166d Merge remote-tracking branch 'origin/dev' into dev-game-layer 2023-11-06 16:23:29 +00:00
Czar.Echavez
eba10ae5ef #1962: clean up tests + improve the show command which shows the folders and files in file system 2023-11-06 11:56:44 +00:00
Czar.Echavez
535c1b19ab #1962: attempt to make the timestep actions look neater + adding logic that allows restoring a folder take multiple timesteps 2023-11-06 11:12:06 +00:00
Czar.Echavez
e70ceec716 #1962: folder/file restore logic 2023-11-06 10:22:08 +00:00
Czar.Echavez
51713bad74 #1962: split tests into managable files + implement deletion of folders and files + tests 2023-11-03 15:15:18 +00:00
Czar.Echavez
b2c3e273b7 #1962: separating file system into more managable files 2023-11-02 15:10:51 +00:00
Czar.Echavez
b67eb1bb34 #1962: separating file system into more managable files 2023-11-02 13:14:08 +00:00