Commit Graph

265 Commits

Author SHA1 Message Date
Chris McCarthy
b6d93ad33f #917 - Began the process of reloading existing agents into the session 2023-06-28 19:54:00 +01:00
Chris McCarthy
4866722911 #917 - Overhauled transaction and mean reward writing.
- Separated out learning outputs from evaluation outputs
2023-06-28 16:34:00 +01:00
Chris McCarthy
a9ebfd7917 #917 - Synced with dev and added better logging 2023-06-28 12:01:01 +01:00
Marek Wolan
e086d419ad Attempt to add flat spaces 2023-06-28 11:07:45 +01:00
Chris McCarthy
edab1a393d Merge remote-tracking branch 'origin/dev' into feature/917_Integrate_with_RLLib
# Conflicts:
#	src/primaite/config/training_config.py
#	src/primaite/main.py
2023-06-28 10:11:03 +01:00
Marek Wolan
d28db68c02 Merged PR 95: Apply precommits and add precommit to build pipeline
## Summary
The code changes are purely cosmetic- the result of applying pre-commit to all our files. I also added a pre-commit step to the build pipeline to reject non-conforming PRs

## Test process
I saw that the build pipeline passes with this new step.

## Checklist
- [ ] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #1557
2023-06-28 08:14:49 +00:00
SunilSamra
9623b1450a 1555 - added specific steps to doc string 2023-06-27 16:59:43 +01:00
SunilSamra
f368aae982 Merge remote-tracking branch 'origin/dev' into feature/1555-update-test-reward-doc-string 2023-06-27 16:55:00 +01:00
Marek Wolan
3e620c5633 Merged PR 94: Fix ier reward calculation
## Summary
Logic error with negation of booleans.

## Test process
Run with debug logging to verify that no longer getting warnings about reference IERS being blocked.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Fix ier reward calculation

Related work items: #1554
2023-06-27 15:10:19 +00:00
Chris McCarthy
20b65ae9ab Merge remote-tracking branch 'origin/bugfix/1554-fix-not-learning-iers' into feature/917_Integrate_with_RLLib 2023-06-27 15:56:56 +01:00
Marek Wolan
afe5bf8fe8 Merge branch 'dev' into feature/build-pipeline-precommit 2023-06-27 15:49:49 +01:00
Marek Wolan
349a18a4eb Fix ier reward calculation 2023-06-27 15:27:56 +01:00
Czar Echavez
be041f5e5c Merge branch 'dev' into feature/1386-enable-a-repeatable-or-deterministic-baseline-test 2023-06-27 14:16:10 +01:00
SunilSamra
7bed0d59cb Merge remote-tracking branch 'origin/dev' into feature/1555-update-test-reward-doc-string 2023-06-27 14:09:36 +01:00
Marek Wolan
33f7e9f506 Add pre-commit 2023-06-27 13:07:54 +00:00
Marek Wolan
f71d4488b8 Merged PR 92: Fix reference IERs
## Summary
As per the ticket and James's explanation, there are now separate reference IERs which are used for the reference environment.

## Test process
I verified that the training can occur.
![image.png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/92/attachments/image.png)

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [n/a] I have written **tests** for any new functionality added with this PR
- [n/a] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Fix reference IERs

Related work items: #1554
2023-06-27 13:07:39 +00:00
Marek Wolan
a3e50293b7 Add pre-commits to build pipeline 2023-06-27 12:07:33 +00:00
Marek Wolan
beae1e5c4f Cosmetic changes to satisfy pre-commit 2023-06-27 13:06:10 +01:00
Marek Wolan
de91a50581 Improve readability 2023-06-27 12:56:15 +01:00
Marek Wolan
cdeb6abf60 More descriptive debug msg 2023-06-27 12:44:42 +01:00
SunilSamra
36f1dff9b8 1555 - updated doc-string to make test understanding easier 2023-06-27 12:27:57 +01:00
Marek Wolan
dc43e5dc15 rename to prevent confusion 2023-06-27 10:45:45 +00:00
Marek Wolan
3774fb8319 apply pre-commits 2023-06-27 11:20:18 +01:00
Marek Wolan
cd991a7d61 Fix reference IERs 2023-06-27 11:10:21 +01:00
Czar Echavez
2e2b098a3c #1386: Apply suggestions from code review - make seed an optional variable 2023-06-23 07:57:31 +00:00
Brian Kanyora
57315a6789 feature\1522:
Create random red agent behaviour.
2023-06-22 15:34:13 +01:00
Chris McCarthy
8f6e930ba2 #917 - Updated main config 2023-06-22 14:10:38 +01:00
Chris McCarthy
dce6fe55ee #917 - Got things working'ish 2023-06-20 22:29:46 +01:00
Chris McCarthy
7b0f47d6f8 #917 -Finished integrating all agents to either train (policy agents) or evaluate (hard-coded agents). Still some fixing up to do, tidying up, loading etc. also docs. But this is all now working. 2023-06-20 16:06:55 +01:00
Czar Echavez
65024320dd #1386: fix README.md 2023-06-20 11:22:29 +01:00
Czar Echavez
99399cbda6 #1386: added documentation + dealing with pre-commit checks 2023-06-20 11:19:05 +01:00
Czar Echavez
9fb30ffe1b #1386: added the ability to set deterministic and seeding RNG when training and evaluating + the fix provided in #1535 2023-06-20 10:41:30 +01:00
Chris McCarthy
10c94954a5 #917 - Almost there. All output files being writen for SB3/RLLIB PPO & A2C. Just need to bring in the hardcoded agents then update the testa and docs. 2023-06-19 21:53:25 +01:00
Chris McCarthy
3670f16766 #917 - Integrated both SB3 and RLlib agents into PrimaiteSession 2023-06-19 20:27:08 +01:00
Chris McCarthy
c09874edbe #917 - Got RLlib fully training in PrimAITE. Started integrating the the other agents into the Session class 2023-06-18 22:40:56 +01:00
Chris McCarthy
31eb36c75a #917 - started working on the Agent abstract classes and sub-classes 2023-06-15 09:48:44 +01:00
Chris McCarthy
40686031e6 temp commit 2023-06-13 09:42:54 +01:00
Christopher McCarthy
dc0349c37b Merged PR 81: #915 Packaging & Deployment
## Summary
- Created app dirs and set as constants in the top-level init.
- Renamed _config_values_main to training_config.py and renamed the ConfigValuesMain class to TrainingConfig.
- Moved training_config.py to src/primaite/config/training_config.py
- Renamed all training config yaml file keys to make creating an instance of TrainingConfig easier.
- Moved action_type and num_steps over to the training config.
- Decoupled the training config and lay down config.
- Refactored main.py so that it can be ran from CLI and can take a training config path and a lay down config path.
- Refactored all outputs so that they save to the session dir.
- Added some necessary setup scripts that handle creating app dirs, fronting example config files to the user, fronting demo notebooks to the user, performing clean-up in between installations etc.
- Added functions that attempt to retrieve the file path of users example config files that have been fronted by the primaite setup.
- Added logging config and a getLogger function in the top-level init.
- Refactored all logs entries logged to use a logger using the primaite logging config.
- Added basic typer CLI for doing things like setup, viewing logs, viewing primaite version, running a basic session.
- Updated test to use new features and config structures.
- Made tests log to temp directory
- typer==0.9.0 added to pyproject.toml
- Refactored documentation and included APi docs, dependencies.
- Make files now re-build autosummary and deps file.
- Added typer and platformdirs to deps in pyproject.toml.
- Made root_is_pure = True in setup.py as platform/python specific wheels don't need to be built but the option is there should we need to.

## Test process
- Added an e2e test for primaite.main.run func.
- Added legacy config file conversion tests
- added

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

Related work items: #915
2023-06-12 18:17:46 +00:00
Chris McCarthy
9563a8a121 Synced with dev 2023-06-12 16:59:31 +01:00
Chris McCarthy
fc6ec69dee Merge remote-tracking branch 'origin/dev' into feature/915_PRI-31_Packaging_Deployment
# Conflicts:
#	docs/source/about.rst
#	src/primaite/main.py
#	src/primaite/nodes/node.py
2023-06-12 16:42:26 +01:00
Brian Kanyora
173e6d77a3 Merged PR 75: Fixing the functionality of resetting a node
## Summary:
Split the ticket into two task

Task 1: Fixed the resetting operating state to set compromised or overwhelmed services or operating system back to a good state. Added a reset count that switches the node into a good state.

Task 2: Created a "SHUTTING DOWN" operating state to last for a (configurable) and a "BOOTING" operating state to last for a (configurable).

## Test process
First test was to test the reset changes the node to a good state when its set to a COMPROMISED state. The last two test makes sure that the node boots and shutdowns correctly.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #898, #1438
2023-06-12 15:21:47 +00:00
Chris McCarthy
4596687e35 Merge remote-tracking branch 'origin/dev' into feature/898-Fix-the-functionality-of-resetting-a-node 2023-06-12 14:20:16 +01:00
Chris McCarthy
4c4f351534 #951 - Can not view and change the log level from the cli.
- Fixed write transaction issue in transactions_to_file.py
2023-06-09 22:23:45 +01:00
Christopher McCarthy
3016f4b984 Apply suggestions from code review 2023-06-09 20:31:12 +00:00
Chris McCarthy
149a534851 #915 - Annotated logs func in cli.py to take -n.
- Fixed entry point on main.py
- Commented out the print reward line in step func of primaite_env.py.
- Added jupyterlab==3.6.1 to pyproject.toml
2023-06-09 16:44:49 +01:00
Chris McCarthy
1a7d158d77 #915 - Force app dir creation before config file sink 2023-06-09 16:04:56 +01:00
Chris McCarthy
86b813afa1 #915 - Synced with dev 2023-06-09 15:49:48 +01:00
Chris McCarthy
2fccbbd9df Merge remote-tracking branch 'origin/dev' into feature/915_PRI-31_Packaging_Deployment
# Conflicts:
#	docs/source/about.rst
#	docs/source/config.rst
#	src/primaite/common/config_values_main.py
#	src/primaite/environment/primaite_env.py
#	src/primaite/main.py
#	tests/config/multidiscrete_obs_space_laydown_config.yaml
#	tests/config/obs_tests/laydown.yaml
#	tests/conftest.py
#	tests/test_observation_space.py
2023-06-09 13:41:05 +01:00
Chris McCarthy
3a5d982991 #915 - Synced with dev to bring in changes from #898 2023-06-09 13:11:14 +01:00
Marek Wolan
495357a4bc Merged PR 69: Configurable observation space.
## Summary
This PR implements a new module called `observations` within `primaite.environment`.

The module is able to keep track of the observation space and to generate observations for the blue agent. It builds the observation space from components. Each component can be configured by supplying parameters at instantiation. For example, the Link Traffic Levels component lets the user customise how many levels there should be.

Note: If a space contains multiple components, they are combined into a 'gym.spaces.Tuple' Space. This is not compatible with some learning agents so we may need to add the options to flatten the observation space.

## Test process
I was able to run the main script with a single-component obs space. I also wrote several unit and integration tests for the new functionality.

## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [x] I have written **tests** for any new functionality added with this PR
- [x] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

If you review this, please check the linked tickets and make sure you agree that this PR addresses them fully.

Related work items: #886, #924, #1468, #1469
2023-06-09 09:52:47 +00:00