Commit Graph

833 Commits

Author SHA1 Message Date
Chris McCarthy
1ddfca6459 #1566 - Refactored the test_train_eval_episode_steps.py to sue TempPrimaiteSession.
- Fixed all errors that were caused b fixing the above.
- Some tests still fail, these are for SS to fix.
- Dropped the old run_generic stuff from conftest.py
2023-07-07 15:50:14 +01:00
SunilSamra
4902ccd2b4 Merge remote-tracking branch 'origin/dev' into feature/901-change-functionality-acl-rules 2023-07-07 15:14:05 +01:00
Chris McCarthy
8edb26a65c Merge remote-tracking branch 'origin/dev' into 1566-configure-episode-steps-learn-eval
# Conflicts:
#	src/primaite/agents/rllib.py
2023-07-07 14:34:20 +01:00
Marek Wolan
2effe31523 Reworded observation description 2023-07-07 14:24:37 +01:00
SunilSamra
79d98e977b 1566 - added test file and edited configs to include types of num steps and modifed agents to use correct step and episode counts 2023-07-07 14:13:47 +01:00
Marek Wolan
179301e068 Update observation space documentation 2023-07-07 13:52:14 +01:00
Brian Kanyora
15ca48ec6c Added a getting started file 2023-07-07 11:37:57 +01:00
Marek Wolan
1d1f3f2403 Merge remote-tracking branch 'origin/dev' into feature/1572-fix-docs-formatting 2023-07-07 10:30:11 +01:00
Marek Wolan
5618283cc5 Standardise docstring summary line placement. 2023-07-07 10:28:00 +01:00
Czar Echavez
4acb220c0c Merged PR 89: #1386 Enable a repeatable/deterministic baseline test
## Summary
- Added the fix from #1535 with minor changes to make sure that the `primaite_env.step()` function can properly parse the action
- added the config deterministic and seed to training config
- added the deterministic and seed to the Training config class, with defaults `False` and `None` respectively
- minor fix to `primaite_env.close()` function so that it now works

## Test process
Added e2e tests for generic, ppo and a2c which evaluates a trained agent twice to make sure that the seeding and deterministic action works

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

#1386: added the ability to set deterministic and seeding RNG when training and evaluating + the fix provided in #1535

Related work items: #1386, #1535
2023-07-07 09:22:47 +00:00
Czar Echavez
76997f403e Merge branch 'dev' into feature/1386-enable-a-repeatable-or-deterministic-baseline-test 2023-07-06 22:22:37 +01:00
Christopher McCarthy
5ac196b3cb Merged PR 109: Auto save agent at end of training
## Summary
* Made RLlib and SB3 agents save at the end of each learning session by default using a common file naming format. Also now agents only checkpoint every n and not on the final episode.

## Test process
*Tests saved agent file in the test_primaite_session test.

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

Related work items: #1593
2023-07-06 16:29:48 +00:00
Marek Wolan
87710ec22b Merged PR 108: Divide default rewards by 10000
## Summary
As per the discussion this morning, this PR reimplements changes that were made by ADSP to make the default rewards smaller. This also adds type hints rewards as floats.

## Test process
I checked that sessions are able to run and that they report values similar to what we are used to but smaller by a factor of 10000. I did not change the reward values in the integration test configs, and the tests still pass.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #889, #1586
2023-07-06 15:17:47 +00:00
Marek Wolan
653d76ec62 Added docstrings to class intialisers 2023-07-06 16:08:51 +01:00
Marek Wolan
9167816896 Removed reference to file that no longer exists 2023-07-06 15:18:49 +01:00
Marek Wolan
8d466accf5 Add __init__ to class special members doc 2023-07-06 15:18:33 +01:00
Marek Wolan
eb068e22b6 undeleted api (lol) 2023-07-06 15:05:39 +01:00
Marek Wolan
70bde700b7 Deleted icon 2023-07-06 15:04:46 +01:00
Chris McCarthy
ddabf991ce #1593 - Ran pre-commit hook 2023-07-06 14:18:49 +01:00
Chris McCarthy
fc98441a11 #1593 - Check that agent saved file exists 2023-07-06 14:13:02 +01:00
Chris McCarthy
1e7f5b62f3 #1963 - Made RLlib and SB3 agents save at the end of each learning session by default using a common file naming format. Also now agents only checkpoint every n and not on the final episode 2023-07-06 13:56:12 +01:00
Czar Echavez
08220ff6ea #1386: remove redundant config files + test fixtures + fixing deterministic and seed config description in documentation to avoid misunderstandings 2023-07-06 13:27:44 +01:00
Marek Wolan
33f6f8bc34 Updated rewards type description in docs 2023-07-06 12:56:24 +01:00
Marek Wolan
dd8593e489 Change reward to float and divide by 10000 2023-07-06 12:52:14 +01:00
Czar Echavez
bb9bfc50a5 #1386: remove setting of global seed + running pre-commit checks 2023-07-06 12:10:26 +01:00
Chris McCarthy
a35c363345 #1386 - Updated tests in test_seeding_and_deterministic_session.py to use TempPrimaiteSession.
- Added test_seeded_learning test and test_deterministic_evaluation test.
- Passed config values seed and deterministic to ppo agent
- Dropped deterministic override in evaluate functions
- TempPrimaiteSession now writes files to a UUID folder rather than datetime
- Added seed to Ray RLlib agent setup in rllib.py
- Added seed to SB3 agent setup in sb3.py
2023-07-06 11:35:44 +01:00
SunilSamra
3ff081ea71 1566 - added train_episodes, train_steps, eval_episodes and eval_steps to training_config_main.yaml 2023-07-06 11:12:51 +01:00
SunilSamra
7a02661c66 901 - changed how acl rules are added to access control list and added structure to AccessControlList observation 2023-07-06 11:07:21 +01:00
Marek Wolan
30b08fd48b Rescaled default rewards by a factor of 1/10000 2023-07-06 10:51:34 +01:00
Marek Wolan
d9394d274d Updated documentation 2023-07-06 10:34:27 +01:00
Marek Wolan
b9549497d2 Removed duplicated function definitions 2023-07-06 10:23:14 +01:00
Chris McCarthy
f92d2fb65d temp 2023-07-06 10:07:54 +01:00
Marek Wolan
ead02ed691 Updated docstrings 2023-07-05 16:46:23 +01:00
Marek Wolan
013dcb94a8 Add docstrings and type hints. 2023-07-05 16:19:43 +01:00
Marek Wolan
d598ffaa65 Merge branch 'bugfix/1587-hardcoded-agent' into feature/1575-docstring-param-desc 2023-07-05 15:22:13 +01:00
Czar Echavez
0068092d8b #1386: remove unneeded configs + setting the seed globally + temp test 2023-07-05 15:02:41 +01:00
Marek Wolan
8f4e8bf538 typo 2023-07-05 14:50:03 +01:00
Marek Wolan
5f98c9b1bd Fix minor typos in docstrings 2023-07-05 14:13:43 +01:00
Marek Wolan
376ff9f597 Imported ADSP function for ACL 2023-07-05 14:10:52 +01:00
Marek Wolan
0664389bdc Changed hardcoded agent helper for new obs space 2023-07-05 13:58:46 +01:00
Czar Echavez
b0c83d7148 #1386: fix saving of agent 2023-07-05 11:41:18 +01:00
SunilSamra
35045f4842 901 - merged with dev 2023-07-05 11:34:15 +01:00
Marek Wolan
247136ed6d Merge branch 'feature/1572-fix-docs-formatting' of https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE into feature/1572-fix-docs-formatting 2023-07-05 10:14:20 +01:00
Marek Wolan
fa6dbd8338 Move class docstrings out of init function. 2023-07-05 10:14:16 +01:00
Marek Wolan
d7bf90e6f4 Updated access_control_list.py 2023-07-05 09:00:41 +00:00
Marek Wolan
b81c29d46e Update some param descriptions for hardcoded agent 2023-07-05 09:54:50 +01:00
Marek Wolan
24a4f96ed0 Add blank lines at the end of file. 2023-07-05 09:22:49 +01:00
Marek Wolan
17b5c6bf92 Add missing module level docstrings. 2023-07-05 09:19:58 +01:00
SunilSamra
766ee9624a 901 - updated observations.py to change and add new mapping of ACL rules to represent no rule present in list 2023-07-05 09:08:03 +01:00
Czar Echavez
818d64f330 #1386: fix bug with agent zip file not being saved after run 2023-07-04 16:30:31 +01:00