Commit Graph

47 Commits

Author SHA1 Message Date
Christopher McCarthy
3186fcb489 Merged PR 121: #1629 - Added rllib test
## Summary
Quick test that uses RLLIB in a session

## Test process
The learning session completes then we check that the number of rows in both the average reward per episode and all transactions csv files.

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

#1629 - Added rllib test

Related work items: #1629
2023-07-17 17:28:51 +00:00
Chris McCarthy
0732eb2f5c #1629 - Added rllib test 2023-07-17 11:50:07 +01:00
Czar Echavez
dce0d10383 #1595: fix poorly merged tests + files 2023-07-14 11:21:59 +01:00
SunilSamra
bd8d9f278d 1566 - removed redundant config file 2023-07-12 09:52:54 +01:00
SunilSamra
728ff01076 1566 - fixed the test_training_config.py test file by removing num_steps from init 2023-07-10 13:24:34 +01:00
SunilSamra
9738d2b889 1566 - added correct num_train_episodes etc values to configs, fixed test_reward.py 2023-07-10 11:25:26 +01:00
SunilSamra
af4fd4f66b 1566 - updated configs to correct values of step count and number of episodes 2023-07-07 16:26:12 +01:00
Chris McCarthy
1ddfca6459 #1566 - Refactored the test_train_eval_episode_steps.py to sue TempPrimaiteSession.
- Fixed all errors that were caused b fixing the above.
- Some tests still fail, these are for SS to fix.
- Dropped the old run_generic stuff from conftest.py
2023-07-07 15:50:14 +01:00
Chris McCarthy
8edb26a65c Merge remote-tracking branch 'origin/dev' into 1566-configure-episode-steps-learn-eval
# Conflicts:
#	src/primaite/agents/rllib.py
2023-07-07 14:34:20 +01:00
SunilSamra
79d98e977b 1566 - added test file and edited configs to include types of num steps and modifed agents to use correct step and episode counts 2023-07-07 14:13:47 +01:00
Czar Echavez
76997f403e Merge branch 'dev' into feature/1386-enable-a-repeatable-or-deterministic-baseline-test 2023-07-06 22:22:37 +01:00
Czar Echavez
08220ff6ea #1386: remove redundant config files + test fixtures + fixing deterministic and seed config description in documentation to avoid misunderstandings 2023-07-06 13:27:44 +01:00
Chris McCarthy
a35c363345 #1386 - Updated tests in test_seeding_and_deterministic_session.py to use TempPrimaiteSession.
- Added test_seeded_learning test and test_deterministic_evaluation test.
- Passed config values seed and deterministic to ppo agent
- Dropped deterministic override in evaluate functions
- TempPrimaiteSession now writes files to a UUID folder rather than datetime
- Added seed to Ray RLlib agent setup in rllib.py
- Added seed to SB3 agent setup in sb3.py
2023-07-06 11:35:44 +01:00
Czar Echavez
0068092d8b #1386: remove unneeded configs + setting the seed globally + temp test 2023-07-05 15:02:41 +01:00
Czar Echavez
c7de7bf21b Merge branch 'dev' into feature/1386-enable-a-repeatable-or-deterministic-baseline-test 2023-07-04 09:41:07 +01:00
Chris McCarthy
c36ddfa03f #917 - Synced with dev (at the point of random red agent) 2023-07-03 17:25:21 +01:00
Chris McCarthy
e11fd2ced4 #917 - Fixed the RLlib integration
- Dropped support for overriding the num_episodes and num_steps at the agent level. It's just not needed and will add complexity when overriding and writing output files.
2023-06-30 16:52:57 +01:00
Chris McCarthy
7b1f889415 #917 - Integrated the PrimaiteSession into all tests.
- Ran a full pre-commit hook and thus encountered tons of fixes required
2023-06-30 09:08:13 +01:00
Chris McCarthy
7b0f47d6f8 #917 -Finished integrating all agents to either train (policy agents) or evaluate (hard-coded agents). Still some fixing up to do, tidying up, loading etc. also docs. But this is all now working. 2023-06-20 16:06:55 +01:00
Czar Echavez
9fb30ffe1b #1386: added the ability to set deterministic and seeding RNG when training and evaluating + the fix provided in #1535 2023-06-20 10:41:30 +01:00
Chris McCarthy
40686031e6 temp commit 2023-06-13 09:42:54 +01:00
Chris McCarthy
fc6ec69dee Merge remote-tracking branch 'origin/dev' into feature/915_PRI-31_Packaging_Deployment
# Conflicts:
#	docs/source/about.rst
#	src/primaite/main.py
#	src/primaite/nodes/node.py
2023-06-12 16:42:26 +01:00
Chris McCarthy
86b813afa1 #915 - Synced with dev 2023-06-09 15:49:48 +01:00
Chris McCarthy
2fccbbd9df Merge remote-tracking branch 'origin/dev' into feature/915_PRI-31_Packaging_Deployment
# Conflicts:
#	docs/source/about.rst
#	docs/source/config.rst
#	src/primaite/common/config_values_main.py
#	src/primaite/environment/primaite_env.py
#	src/primaite/main.py
#	tests/config/multidiscrete_obs_space_laydown_config.yaml
#	tests/config/obs_tests/laydown.yaml
#	tests/conftest.py
#	tests/test_observation_space.py
2023-06-09 13:41:05 +01:00
Chris McCarthy
3a5d982991 #915 - Synced with dev to bring in changes from #898 2023-06-09 13:11:14 +01:00
Chris McCarthy
6c2c95b329 Merge remote-tracking branch 'origin/dev' into feature/915_PRI-31_Packaging_Deployment
# Conflicts:
#	tests/conftest.py
#	tests/test_observation_space.py
#	tests/test_reward.py
2023-06-09 10:35:14 +01:00
Marek Wolan
00a30bcb7a Merge remote-tracking branch 'origin/dev' into feature/1468-observations-class 2023-06-09 09:01:54 +01:00
Chris McCarthy
98fc1e4c71 #915 - Created app dirs and set as constants in the top-level init.
- renamed _config_values_main to training_config.py and renamed the ConfigValuesMain class to TrainingConfig.
Moved training_config.py to src/primaite/config/training_config.py
- Renamed all training config yaml file keys to make creating an instance of TrainingConfig easier.
Moved action_type and num_steps over to the training config.
- Decoupled the training config and lay down config.
- Refactored main.py so that it can be ran from CLI and can take a training config path and a lay down config path.
- refactored all outputs so that they save to the session dir.
- Added some necessary setup scripts that handle creating app dirs, fronting example config files to the user, fronting demo notebooks to the user, performing clean-up in between installations etc.
- Added functions that attempt to retrieve the file path of users example config files that have been fronted by the primaite setup.
- Added logging config and a getLogger function in the top-level init.
- Refactored all logs entries logged to use a logger using the primaite logging config.
- Added basic typer CLI for doing things like setup, viewing logs, viewing primaite version, running a basic session.
- Updated test to use new features and config structures.
- Began updating docs. More to do here.
2023-06-07 22:40:16 +01:00
Marek Wolan
4329c65211 Apply suggestions from code review. 2023-06-07 15:25:11 +01:00
SunilSamra
6089fb6950 893 - removed unnecessary functions from utils.py and changed single_action_space_fixed_blue_actions_main_config.yaml back to GENERIC agentIdentifier after PR comments 2023-06-07 14:39:52 +01:00
SunilSamra
a45f0d1979 Merge remote-tracking branch 'origin/dev' into feature/893-node-acl-into-one-action-space 2023-06-06 11:56:52 +01:00
SunilSamra
1a7d629d5a 893 - added new tests to test action space size and node is completing both sets of actions in a single episode and created new main config 2023-06-06 11:00:41 +01:00
Marek Wolan
f14910ca96 Fix Link Traffic Levels observation encoding 2023-06-02 13:15:38 +01:00
Marek Wolan
14096b3dd1 Add tests for observations 2023-06-02 12:59:01 +01:00
Marek Wolan
602bf9ba9a Edit configs for observation space 2023-06-02 09:10:53 +01:00
SunilSamra
f72a80c9d2 1443 - added in print test statements 2023-06-01 16:27:25 +01:00
Marek Wolan
2b2c7ad22d Improve observation space test 2023-06-01 09:45:46 +01:00
SunilSamra
ae2f4d472e 1443 - reverted changes made to observation space and added config files for testing 2023-05-31 14:11:15 +01:00
Marek Wolan
41910797b5 Revert config changes by removing observations 2023-05-31 10:52:57 +01:00
Marek Wolan
317549225f Merge remote-tracking branch 'origin/dev' into feature/1463-multidiscrete-observation-option 2023-05-31 10:46:18 +01:00
SunilSamra
82371b210e Merge remote-tracking branch 'origin/dev' into feature/893-node-acl-into-one-action-space 2023-05-31 10:34:42 +01:00
Marek Wolan
082e3b7307 Add test for new multidiscrete spaces 2023-05-30 15:48:11 +01:00
Marek Wolan
7bc938229b Update configs and transactions to include new obs 2023-05-30 15:24:13 +01:00
SunilSamra
7aa8d89dfe 1443 - updated test_reward.py to reflect updates to reward.py so that the correct config values are called i.e. compromisedShouldBeGood on the correct steps during the training run 2023-05-30 11:50:54 +01:00
SunilSamra
3cd5864f25 1429 - created new branch from dev, added enums to enums.py, created agents package and utils.py file, added option to primaite_env.py for ANY action type and changed the action spaces are defined using ADSP branch 2023-05-26 10:17:45 +01:00
Chris McCarthy
32a4d9e459 #1355 - Carried out full renaming in node.py, active_node.py, passive_node.py, and service_node.py to make params and variable names explicit.
- Made the same renaming in the yaml laydown config files.
- Added Type hints wherever I've been.
- Added a custom NodeType in custom_typing.py to encompass the Union of ActiveNode, PassiveNode, ServiceNode.
2023-05-25 21:03:11 +01:00
Chris McCarthy
fc1b374cb2 #902 - replaced 'final_node_<placeholder>' with 'reference_node_<placeholder>' in methods for scoring of os_state, file_system_state, service state and operating state. This fixed the reward function so it is checked at each step for node operating system state, operating state, file system state and service state.
- Added unit tests.
2023-05-25 14:05:53 +01:00