Commit Graph

382 Commits

Author SHA1 Message Date
SunilSamra
a2f43b5abc #901 - merged dev into branch 2023-07-17 19:54:07 +01:00
SunilSamra
95b6211781 Merge remote-tracking branch 'origin/feature/901-change-functionality-acl-rules' into feature/901-change-functionality-acl-rules 2023-07-17 19:42:29 +01:00
SunilSamra
3aab6a3738 #901 - ran black pre-commit over observations.py to fix it 2023-07-17 19:42:05 +01:00
Sunil Samra
1721f2eb84 Apply suggestions from code review 2023-07-17 18:36:13 +00:00
Christopher McCarthy
35af1e9d1e Merged PR 121: #1629 - Added rllib test
## Summary
Quick test that uses RLLIB in a session

## Test process
The learning session completes then we check that the number of rows in both the average reward per episode and all transactions csv files.

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

#1629 - Added rllib test

Related work items: #1629
2023-07-17 17:28:51 +00:00
Czar Echavez
e1ac628793 Merged PR 119: Loading SB3 Agents + Loading agent via PrimaiteSession
## Summary
- Added a feature which allows a user to load a previous SB3 session
- Added a feature which allows a user to load a previous PrimaiteSession
- Added a feature which allows a user to load a previous session via the CLI: `primaite session --load "<SESSION_PATH>"`
- RLlib is TODO in another ticket #1626
- Parallel tests via the [pytest-xdist](https://pypi.org/project/pytest-xdist/) dependency (MIT licensed)
- Moved hardcoded agent into hardcoded_abc.py
- renamed agent.py to agent_abc.py to clarify it is an abstract base class
- Added documentation to clarify how to use the feature via CLI or using the run function via main.py

## Test process
Created [test_session_loading.py](https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/119?_a=files&path=/tests/test_session_loading.py) which loads a previously run session and then performs a learn and evaluation run on the loaded agent/Primate session.

The test copies the saved session into a temporary folder, which is then set as the test session path. Once the test is done, the temporary folder should then be deleted

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

Related work items: #1595
2023-07-17 15:26:42 +00:00
SunilSamra
257be9532f #901 - Changed num_eval_steps back to 1 in ppo_seeded_training_config.yaml 2023-07-17 15:54:15 +01:00
SunilSamra
57157db08c Merge remote-tracking branch 'origin/dev' into feature/901-change-functionality-acl-rules 2023-07-17 15:51:01 +01:00
Czar Echavez
39b30460cd Merge branch 'dev' into feature/1595-finalise-and-test-the-loading-of-trained-agents 2023-07-17 15:23:46 +01:00
Sunil Samra
dd21f9440f Apply suggestions from code review 2023-07-17 14:21:37 +00:00
Brian Kanyora
a432822bcb Merged PR 113: feature/1597-Getting-Started
## Summary
Add a Getting started page to the docs file.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [na] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #1597
2023-07-17 14:19:16 +00:00
SunilSamra
ec938ce761 #901 - Changed num_eval_steps back to 1 in ppo_seeded_training_config.yaml 2023-07-17 14:06:33 +01:00
SunilSamra
007a0c4b98 Merge remote-tracking branch 'origin/feature/901-change-functionality-acl-rules' into feature/901-change-functionality-acl-rules 2023-07-17 14:01:32 +01:00
SunilSamra
2526427f2f #901 - Fixed bug in implicit rule - comparing it to string ALLOW or DENY in access_control_list.py 2023-07-17 13:58:06 +01:00
Sunil Samra
cc09fe9079 Removed apply_implicit_rule comment 2023-07-17 12:45:31 +00:00
SunilSamra
78d7f39342 #901 - Removed flatten from training configs
- Added flatten operation in observations.py when there are multiple obs components
- Updated config.rst docs
2023-07-17 13:44:16 +01:00
SunilSamra
da20c0e9e6 #901
- Removed bool apply_implicit_rule
- Set default implicit_rule to EXPLICIT DENY
- Added position to ACLs in laydown configs
- Removed apply_implicit_rule from training configs
2023-07-17 13:00:58 +01:00
Chris McCarthy
360eb38c2b #1629 - Added assertion in the test that checks the length of the all transactions file too.
- Added supporting function on the TempPrimaiteSession class that reads the all transactions csv file.
- Some renaming of the functions.
2023-07-17 12:14:47 +01:00
Czar Echavez
6b76214eb2 #1595: set default tc and ldc to None for AgentABC and PrimaiteSession + adding a comment for cli load flag 2023-07-17 11:54:54 +01:00
Chris McCarthy
75c91b9eb9 #1629 - Added rllib test 2023-07-17 11:50:07 +01:00
Brian Kanyora
9a0b14b111 Merge remote-tracking branch 'origin/dev' into feature/1597-Getting-Started 2023-07-17 10:33:59 +01:00
SunilSamra
707d8f6189 #901
- Added check in access_control_list.py which sets implicit permission to NA if boolean is False
- Changed the defaults in training_config.py so that each scenario has an EXPLICIT ALLOW rule as default implicit rule
- Updated the test_seeding_and_deterministic_session.py because of change no2 adds an extra rule to that scenario
2023-07-17 10:27:56 +01:00
Brian Kanyora
50697c6f75 Apply suggestions from code review 2023-07-17 09:23:11 +00:00
SunilSamra
9df8d132fc #901 - added to config.rst and added new ACL main config options 2023-07-17 10:08:12 +01:00
Czar Echavez
ea7c1519fe #1595: minor fix to cli command 2023-07-14 16:04:34 +01:00
SunilSamra
8aa71c3ff8 #901 - amended comment in observations.py 2023-07-14 16:04:13 +01:00
Czar Echavez
7c2ff55da2 #1595: added loading sessions to run command + test + documentation for how to use loading sessions 2023-07-14 15:51:38 +01:00
SunilSamra
1b6244d13f #901 - amended comment in training_config_main.yaml 2023-07-14 15:49:18 +01:00
SunilSamra
661c865108 #901 -
- Added comments in access_control_list.py
- Changed obs_shape to max_number_acl_rules from max_number_acl_rules + 1 as index starts from 1
- Commented episode and step print line from test_single_action_space.py
2023-07-14 15:27:37 +01:00
SunilSamra
eb75d15722 901 - Added another test and tidied up comments in test_observation_space.py and tidied up comments in observations.py 2023-07-14 14:51:26 +01:00
SunilSamra
4e53564670 901 - Changed the default expected_mean_reward_per_episode values in test_seeding_and_deterministic_session.py 2023-07-14 14:26:10 +01:00
Czar.Echavez
8e2f105d57 #1595:
- Added ability to load sessions via PrimaiteSession
- PrimaiteSession loading test
- Added a NotImplemented RLlib loading for now
- Added the ability to load sessions for hardcoded agents
- Moved Session metadata parsing to utils
2023-07-14 14:14:03 +01:00
Chris McCarthy
f9c7cafe87 #901 - Dropped temp_primaite_sessiion_2 from conftest.py.
- Re-added the hard-coded mean rewards per episode values from a rpe-trained agent to the deterministic test in test_seeding_and_deterministic_session.py
- Partially tidies up some tests in test_observation_space.py; Still some work to be done on this at a later date.
2023-07-14 14:13:11 +01:00
SunilSamra
e743b2380c 901 - fixed test_observation_space.py, added test fixture for test_seeding_and_deterministic_session.py and increased default max number of acls 2023-07-14 12:29:50 +01:00
Czar Echavez
436448beed #1595: fix poorly merged tests + files 2023-07-14 11:21:59 +01:00
Czar Echavez
7b929109dc #1595: test to make sure that the loaded agent trains + remove unnecessary files + fixing agent save output name 2023-07-14 10:56:28 +01:00
Czar Echavez
118b05ede0 Merge branch 'dev' into feature/1595-finalise-and-test-the-loading-of-trained-agents 2023-07-14 08:39:52 +01:00
SunilSamra
558223e8b6 901 - removed print statements and merged with dev 2023-07-13 17:14:59 +01:00
SunilSamra
77f717c649 Merge remote-tracking branch 'origin/dev' into feature/901-change-functionality-acl-rules 2023-07-13 16:48:02 +01:00
Czar.Echavez
738e5b5dca #1595 missed hardcoded_abc file in commit 2023-07-13 16:24:30 +01:00
Czar.Echavez
606354614a #1595:
- SB3 Agent loading
- rename agent.py -> agent_abc.py
- rename hardcoded.py -> hardcoded_abc.py
- Tests
- Added in test asset that is used to load the SB3 Agent
2023-07-13 16:24:03 +01:00
SunilSamra
0ab4dab72a 901 - fixed test_single_action_space.py test 2023-07-13 11:45:23 +01:00
SunilSamra
f8cb18c654 901 - changed acl current obs from list to numpy.array, changed default ACL list in training_config.py to FALSE, and tried to make test_seeding_and_deterministic_session.py test without fixed reward results 2023-07-13 11:04:11 +01:00
Brian Kanyora
fd2ab39edf feature/1597:
Added dependencies to the index.rst since v1.1.0.
2023-07-13 09:36:04 +01:00
Brian Kanyora
f5e1ef7491 feature/1597:
Added dependencies to the index.rst since v1.1.0.
2023-07-13 09:35:42 +01:00
Marek Wolan
c61770825a Merged PR 116: Update documentation
## Summary
* Update observation space information and standardise formatting of code blocks in that section
* Remove non-ascii quotation characters
* Update custom blue agent page to match new AgentSession classes.
* Introduce glossary
* Provide a first draft of migration guide for 1.2 to 2.0 (probably not comprehensive)

## Test process
Sphinx is able to build the documentation as checked on my local machine

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #1602
2023-07-12 14:28:41 +00:00
Sunil Samra
a19fbd1e98 Merged PR 115: Configure Different Episode and Step Counts for Training and Evaluation
## Summary
Training configs now have 2 different types of episode and step counts - one for train and one for evaluation.

`num_train_episodes`
`num_train_steps`
`num_eval_episodes`
`num_eval_steps`

## Test process
A test file `test_train_eval_episode_steps.py` has been implemented which runs train and evaluation session on two particular configs.

The train and evaluation sessions have different episodes and step count and the test checks that the output log files have the correct number of `total_steps` and `total_episodes`.

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

Related work items: #1566, #1589
2023-07-12 13:34:58 +00:00
Czar Echavez
85c360548b #1595: run tests in parallel 2023-07-12 12:04:26 +01:00
SunilSamra
06c20f6984 Merge remote-tracking branch 'origin/dev' into feature/901-change-functionality-acl-rules
# Conflicts:
#	src/primaite/acl/access_control_list.py
2023-07-12 10:45:03 +01:00
SunilSamra
96b48aad79 1566 - removed redundant config file 2023-07-12 09:52:54 +01:00