2675 Commits

Author SHA1 Message Date
Chris McCarthy
27e22edaf1 #917 - Reinstalled the pre-commit hook 2023-07-03 20:40:38 +01:00
Chris McCarthy
e271a28bf0 #917 - Synced with dev and integrated the new observation space 2023-07-03 20:36:21 +01:00
Chris McCarthy
1716786441 Merge remote-tracking branch 'origin/dev' into feature/917_Integrate_with_RLLib
# Conflicts:
#	src/primaite/config/_package_data/training/training_config_main.yaml
#	src/primaite/environment/primaite_env.py
#	src/primaite/main.py
#	src/primaite/transactions/transaction.py
#	src/primaite/transactions/transactions_to_file.py
2023-07-03 19:51:52 +01:00
Chris McCarthy
c36ddfa03f #917 - Synced with dev (at the point of random red agent) 2023-07-03 17:25:21 +01:00
Chris McCarthy
72b0050b1b Merge remote-tracking branch 'origin/feature/917_Integrate_with_RLLib' into feature/917_Integrate_with_RLLib 2023-07-03 17:12:03 +01:00
Marek Wolan
adf70b7d04 Merged PR 103: Change build pipeline to enable installing from wheel on windows
## Summary
Just splits the install primaite step into two depending if agent is using windows or not.

## Test process
Ran a build successfully.

## Checklist
- [ ] This PR is linked to a **work item**
- [ ] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [ ] I have run **pre-commit** checks for code style
2023-07-03 16:10:09 +00:00
Marek Wolan
0fcc5c0179 Updated azure-ci-build-pipeline.yaml 2023-07-03 16:02:59 +00:00
Czar Echavez
409743dee9 Merge branch 'dev' into feature/1386-enable-a-repeatable-or-deterministic-baseline-test 2023-07-03 16:56:44 +01:00
Marek Wolan
2a481bc57c Merged PR 100: Flatten observation spaces and improve transactions for observations
## Summary
*Replace this text with an explanation of what the changes are and how you implemented them. Can this impact any other parts of the codebase that we should keep in mind?*

## Test process
I ran some training sessions to ensure that the outputted transaction list has the correct data and headers. I was also able to verify that the agent is able to train with observation spaces containing multiple components.

I trained an agent on laydown 3 with NODE_LINK_TABLE both as normal and flattened spaces and the agent learned in both instances.
![image.png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/100/attachments/image.png)  ![image (2).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/100/attachments/image%20%282%29.png)
## Checklist
- [x] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #1558
2023-07-03 15:54:00 +00:00
Chris McCarthy
d55225dd41 Merge remote-tracking branch 'origin/dev' into feature/917_Integrate_with_RLLib
# Conflicts:
#	src/primaite/config/_package_data/training/training_config_main.yaml
#	src/primaite/environment/primaite_env.py
2023-07-03 15:07:09 +01:00
Marek Wolan
93881e5d2c Merge remote-tracking branch 'origin/dev' into feature/1558-flatten-spaces 2023-07-03 15:03:10 +01:00
Marek Wolan
8a95e66b63 Merge branch 'dev' into feature/1558-flatten-spaces 2023-07-03 15:01:56 +01:00
Marek Wolan
9964ac4cea Updated azure-ci-build-pipeline.yaml 2023-07-03 13:36:33 +00:00
Czar Echavez
9ff85207c6 Merged PR 102: 1522 Red Agent random behaviour
## Summary
Ported over ADSP changes regarding the randomised red agent.
Red agent currently only works on laydown configs which contain links.

Each episode generates random red agent instructions

## Test process
Written a test that ensures that the random red agent produces random red agent instructions

| Random red agent | Laydown                | Agent Identifier | Run 1                                                                              | Run 2                                                                              | Run 3                                                                              |
|------------------|------------------------|------------------|------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
| NONE             | Very Basic (Laydown 3) | A2C              | ![image (4).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%284%29.png)  | ![image (8).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%288%29.png)  | ![image (9).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%289%29.png)  |
| RANDOM           | Very Basic (Laydown 3) | A2C              | ![image (5).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%285%29.png)  | ![image (6).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%286%29.png)  | ![image (7).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%287%29.png)  |
| NONE             | Very Basic (Laydown 3) | PPO              | ![image (10).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%2810%29.png) | ![image (11).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%2811%29.png) | ![image (12).png](https://dev.azure.com/ma-dev-uk/b50a61ee-86c4-48bc-9a0b-a67645ba12ee/_apis/git/repositories/2825053e-bd3b-45b2-8680-1281809eefa2/pullRequests/102/attachments/image%20%2812%29.png) |
| RANDOM           | Very Basic (Laydown 3) | PPO              ...
2023-07-03 13:09:32 +00:00
Marek Wolan
fa231057a8 Updated azure-ci-build-pipeline.yaml 2023-07-03 12:44:01 +00:00
Marek Wolan
4b56e7bd3a Updated azure-ci-build-pipeline.yaml 2023-07-03 12:40:02 +00:00
Marek Wolan
0a5d25fe01 Add windows build option 2023-07-03 12:37:08 +00:00
Czar Echavez
a7913487b8 #1522: create_random_red_agent -> _create_random_red_agent + converting NodeStateInstructionRed into a dataclass 2023-07-03 13:36:14 +01:00
Czar Echavez
befd183b2c #1522: refactor red_agent_identifier -> random_red_agent so that it is a boolean + documentation 2023-07-03 12:18:58 +01:00
Chris McCarthy
9999ed9b00 #917 - Added Windows and MacOS to build pipeline. Updated so that runs only Python 3.8 and 3.10 (middle version not required) 2023-07-03 12:03:36 +01:00
Chris McCarthy
676bca70bb Merge remote-tracking branch 'origin/dev' into feature/917_Integrate_with_RLLib
# Conflicts:
#	tests/test_reward.py
2023-07-03 11:59:16 +01:00
Christopher McCarthy
7fe46ef99c Apply suggestions from code review 2023-07-03 10:47:26 +00:00
Czar Echavez
6b4530bded #1522: run pre-commit 2023-07-03 10:08:25 +01:00
Czar Echavez
22849c18da Merge branch 'dev' into feature/1522-Random-Red-Agent-Behaviour 2023-07-03 09:59:25 +01:00
Czar Echavez
68457aa0b2 #1522: added a check for existing links in laydown + test that checks if red agent instructions are random 2023-07-03 09:46:52 +01:00
Sunil Samra
78751de717 Merged PR 93: 1555 - updated doc-string to make test understanding easier
## Summary
Changed doc-string of test_reward.py to reflect the new test and what it is trying to do rather than the old outdated one.

## Test process
NA - no logic changes

## Checklist
- [X] This PR is linked to a **work item**
- [X] I have performed **self-review** of the code
- [X] I have written **tests** for any new functionality added with this PR
- [X] I have updated the **documentation** if this PR changes or adds functionality
- [X] I have run **pre-commit** checks for code style

1555 - updated doc-string to make test understanding easier

Related work items: #1555, #1556
2023-07-03 08:10:17 +00:00
Marek Wolan
046937d838 Apply suggestions from code review 2023-07-03 08:00:51 +00:00
SunilSamra
b3d9a6e79d 901 - merged with changes made to dev 2023-07-03 08:17:52 +01:00
SunilSamra
5f2e8b547a Merge remote-tracking branch 'origin/dev' into feature/1555-update-test-reward-doc-string 2023-07-03 08:10:28 +01:00
Chris McCarthy
06d5004695 #917 - Dropped VerboseLevel in enums.py and changed OutputVerboseLevel to SB3OutputVerboseLevel 2023-06-30 17:09:50 +01:00
Chris McCarthy
e11fd2ced4 #917 - Fixed the RLlib integration
- Dropped support for overriding the num_episodes and num_steps at the agent level. It's just not needed and will add complexity when overriding and writing output files.
2023-06-30 16:52:57 +01:00
Marek Wolan
7e6fe2759b Fix flattening when there are no components. 2023-06-30 15:43:15 +01:00
Marek Wolan
d86489a9c2 revert unnecessary changes. 2023-06-30 13:16:30 +01:00
Chris McCarthy
00185d3dad #917 - Fixed primaite_config.yaml issue in cli.py
- Added kaleido to deps in pyproject.toml
2023-06-30 11:40:26 +01:00
Marek Wolan
09883e13c2 Update docs 2023-06-30 10:44:04 +01:00
Marek Wolan
99ba05c6ee Remove redundant cols from transactions 2023-06-30 10:41:56 +01:00
Czar Echavez
4e1e0ef4b4 #1522: remove numpy randomisation + added random red agent config 2023-06-30 10:37:23 +01:00
Chris McCarthy
cf09202e96 #917 - Added tensorflow to main deps for RLlib.
- Dropped support for Python 3.11 due to not supported on Ray RLlib.
- Made release pipeline only run once as we're now no longer using pure path wheels.
2023-06-30 10:24:59 +01:00
Marek Wolan
fb48f75adf Remove temporary file 2023-06-30 09:54:34 +01:00
Chris McCarthy
7b1f889415 #917 - Integrated the PrimaiteSession into all tests.
- Ran a full pre-commit hook and thus encountered tons of fixes required
2023-06-30 09:08:13 +01:00
Marek Wolan
c9f58fdb2a Fix observation representation in transactions 2023-06-29 15:26:07 +01:00
Czar Echavez
10e432eb01 #1522: fixing create random red agent function 2023-06-29 15:03:11 +01:00
Czar Echavez
15b3bad5d4 Merge branch 'dev' into feature/1522-Random-Red-Agent-Behaviour 2023-06-29 14:17:41 +01:00
Chris McCarthy
b6d93ad33f #917 - Began the process of reloading existing agents into the session 2023-06-28 19:54:00 +01:00
Chris McCarthy
4866722911 #917 - Overhauled transaction and mean reward writing.
- Separated out learning outputs from evaluation outputs
2023-06-28 16:34:00 +01:00
Chris McCarthy
a9ebfd7917 #917 - Synced with dev and added better logging 2023-06-28 12:01:01 +01:00
Marek Wolan
e086d419ad Attempt to add flat spaces 2023-06-28 11:07:45 +01:00
Chris McCarthy
edab1a393d Merge remote-tracking branch 'origin/dev' into feature/917_Integrate_with_RLLib
# Conflicts:
#	src/primaite/config/training_config.py
#	src/primaite/main.py
2023-06-28 10:11:03 +01:00
Marek Wolan
d28db68c02 Merged PR 95: Apply precommits and add precommit to build pipeline
## Summary
The code changes are purely cosmetic- the result of applying pre-commit to all our files. I also added a pre-commit step to the build pipeline to reject non-conforming PRs

## Test process
I saw that the build pipeline passes with this new step.

## Checklist
- [ ] This PR is linked to a **work item**
- [x] I have performed **self-review** of the code
- [ ] I have written **tests** for any new functionality added with this PR
- [ ] I have updated the **documentation** if this PR changes or adds functionality
- [x] I have run **pre-commit** checks for code style

Related work items: #1557
2023-06-28 08:14:49 +00:00
SunilSamra
9623b1450a 1555 - added specific steps to doc string 2023-06-27 16:59:43 +01:00