Commit Graph

1621 Commits

Author SHA1 Message Date
Nick Todd
bfb8fbd1bf Merged PR 365: Run SB3 training in SubprocVecEnv
## Summary
Demonstration notebook show how to use SB3 SubprocVecEnv to vectorise environments to speed up training.
## Test process
Successfully run notebook on Windows and Linux.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2442
2024-05-03 16:02:34 +00:00
Nick Todd
e510c3ceff #2442: Removed 'primaite setup' call 2024-05-03 15:10:24 +01:00
Marek Wolan
727a05f291 Merged PR 363: Make ftp client only default on computers, not servers
## Summary
REMOVE ftp client from host node default software and add it to computer default software, effectively removing it from servers.

Ensure tests are using the uc2 yaml file instead of outdated network generator function.

Fix some tests to make sure it all passing

## Test process
I have tested it.

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Make ftp client only default on computers, not servers

Related work items: #2546
2024-05-03 12:40:17 +00:00
Marek Wolan
535e064af5 unflake test 2024-05-03 11:05:33 +01:00
Marek Wolan
4f38184959 remove redundant comment 2024-05-03 09:08:26 +01:00
Nick Todd
689400e95c Merge branch 'dev' into bugfix/2442-add_SubprocVecEnv_support 2024-05-02 17:04:43 +01:00
Marek Wolan
4a02d1d8e5 Readd import statement 2024-05-02 17:00:29 +01:00
Nick Todd
3c5e107f81 Merge branch 'dev' into bugfix/2442-add_SubprocVecEnv_support 2024-05-02 16:59:08 +01:00
Nick Todd
5a3b6ade2f #2442: Updated changelog. 2024-05-02 16:52:35 +01:00
Marek Wolan
6ef6a5cd18 Merge remote-tracking branch 'origin/dev' into feature/remove-ftp-client-default-app 2024-05-02 16:52:24 +01:00
Nick Todd
7c689a0d35 #2442: Markdown changes. 2024-05-02 16:32:54 +01:00
Nick Todd
a37ed4e487 #2442: Fix config file path to be OS independent. 2024-05-02 16:16:36 +01:00
Marek Wolan
b9b922e776 Make ftp client only default on computers, not servers 2024-05-02 15:51:06 +01:00
Nick Todd
2774442561 Merge branch 'bugfix/2442-add_SubprocVecEnv_support' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into bugfix/2442-add_SubprocVecEnv_support 2024-05-02 15:48:30 +01:00
Nick Todd
db29d33629 #2442: Add multi-processing notebook example. 2024-05-02 15:44:18 +01:00
Czar Echavez
a5794e5df1 Merged PR 359: Improved Dev CLI tools
## Summary
Improved dev tools:
- can configure what the dev mode can override
  - sys log level
  - outputting sys log
  - outputting pcap logs
  - outputting to terminal
  - the location where to output session (by default will print to the primaite repo root)

## Test process
https://dev.azure.com/ma-dev-uk/PrimAITE/_git/PrimAITE/pullrequest/359?_a=files&path=/tests/integration_tests/cli/test_dev_cli.py

and a lot of manually checking if the thing outputs to the correct directories

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2447, #2533
2024-05-02 13:37:16 +00:00
Czar Echavez
e599e03c10 #2533: cleaner output text for dev-mode show + consistent colors 2024-05-02 12:21:24 +01:00
Czar Echavez
ec6d2bf640 #2533: fix missing implementation of output_dir override in io.py + moving the dev mode override in SIM_OUTPUT to prevent the override from being overridden 2024-05-01 16:53:37 +01:00
Czar Echavez
952f6ee225 #2533: fix is_dev_mode for missing or outdated configs 2024-05-01 16:23:44 +01:00
Czar Echavez
25807653f4 Merge remote-tracking branch 'origin/dev' into feature/2533-improve-dev-tools-in-primaite-cli 2024-05-01 16:11:52 +01:00
Czar Echavez
d7572dc18b #2533: changelog 2024-05-01 16:10:27 +01:00
Czar Echavez
61e7a4e439 #2533: documentation on use of dev-mode 2024-05-01 16:04:18 +01:00
Czar Echavez
5516fbc6fc #2533: output directory for sessions 2024-05-01 14:48:10 +01:00
Czar Echavez
b6b5ce91c2 #2533: clean up temp items in tests + fixing the CLI tests 2024-05-01 14:33:33 +01:00
Czar Echavez
a2fb04e6f6 #2533: set default dev output path + clean up 2024-05-01 14:02:40 +01:00
Nick Todd
831f80f617 Merge branch 'bugfix/2442-add_SubprocVecEnv_support' of ssh.dev.azure.com:v3/ma-dev-uk/PrimAITE/PrimAITE into bugfix/2442-add_SubprocVecEnv_support 2024-05-01 11:22:08 +01:00
Nick Todd
b7fa826d95 #2442: Initial commit of MP test script 2024-05-01 11:16:45 +01:00
Christopher McCarthy
4763344835 Merged PR 358: #2519 - refactored all air-space usage to that a new instance of AirSpace is...
## Summary
- Refactored all air-space usage to that a new instance of `AirSpace` is created for each instance of Network. This 1:1 relationship between network and airspace will allow parallelization.

## Test process
- Original test remains the same. TDD, init.
- Added additional `WirelessRouter` test that tests wireless connectivity from config file too.
``` text
C:\Users\ChristopherMcCarthy\source\azure_devops\ma-dev-uk\PrimAITE\venv\Scripts\python.exe "C:/Program Files/JetBrains/PyCharm 2023.3.5/plugins/python/helpers/pycharm/_jb_pytest_runner.py" --path C:\Users\ChristopherMcCarthy\source\azure_devops\ma-dev-uk\PrimAITE\tests\integration_tests\network\test_wireless_router.py
Testing started at 15:33 ...
Launching pytest with arguments C:\Users\ChristopherMcCarthy\source\azure_devops\ma-dev-uk\PrimAITE\tests\integration_tests\network\test_wireless_router.py --no-header --no-summary -q in C:\Users\ChristopherMcCarthy\source\azure_devops\ma-dev-uk\PrimAITE\tests\integration_tests\network

============================= test session starts =============================
collecting ... collected 2 items

test_wireless_router.py::test_cross_wireless_wan_connectivity
test_wireless_router.py::test_cross_wireless_wan_connectivity_from_yaml

======================== 2 passed, 1 warning in 0.14s =========================
+----------------+-------------------+-------------+---------------+--------------+---------+
| Connected Node |    MAC Address    |  IP Address |  Subnet Mask  |  Frequency   |  Status |
+----------------+-------------------+-------------+---------------+--------------+---------+
|    router_1    | da:9a:08:17:80:a4 | 192.168.1.1 | 255.255.255.0 | WiFi 2.4 GHz | Enabled |
|    router_2    | 0d:67:76:5e:c2:fb | 192.168.1.2 | 255.255.255.0 | WiFi 2.4 GHz | Enabled |
+----------------+-------------------+-------------+---------------+--------------+---------+
PASSED     [ 50%]+----------------+-------------------+-------------+---------------+--------------+---------+
| Connected Node |    MAC Address    |  IP Address |  Subnet Mask  |  Frequency   |  Status |
+----------------+-------------------+-------------+---------------+--------------+---------+
|    router_1    | 1a:46:f6:cb:8c:15 | 192.168.1.1 | 255.255.255.0 | WiFi 2.4 GHz | Enabled |
|    router_2    | 7a:9f:d8:2b:4b:90 | 192.168.1.2 | 255.255.255.0 | WiFi 2.4 GHz | Enabled |
+----------------+-------------------+-------------+---------------+--------------+---------+
PASSED [100%]
Process finished with exit code 0
```

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] atte...
2024-05-01 10:12:28 +00:00
Chris McCarthy
7552872430 #2519 - Fixed the broked multi-lan network topology diagrams while I was in doing a PR too 2024-05-01 10:28:54 +01:00
Chris McCarthy
d72622f3f7 Merge branch 'dev' into feature/2519_Make-airspace-an-attribute-of-Simulation
# Conflicts:
#	CHANGELOG.md
2024-05-01 10:22:44 +01:00
Czar Echavez
95643d3255 #2533: optimise so we are not reading from file all the time 2024-04-30 19:36:22 +01:00
Czar Echavez
729f9c5064 #2533: fix primaite config recreated when running setup 2024-04-30 19:15:54 +01:00
Czar Echavez
73f64cd89c Merge remote-tracking branch 'origin/dev' into feature/2533-improve-dev-tools-in-primaite-cli 2024-04-30 18:27:05 +01:00
Charlie Crane
13cabf8e21 Merged PR 360: 2523 - Logging Rewards updates.
## Summary
This PR updates `PrimaiteGymEnv` and `PrimaiteRayMARLEnv` within `environment.py` to log reward values per step and per episode.
Corrects an import statement within `io.py` - removing the use of `src`.
Fixes an error within notebooks that saw them failing following a rename of `game_config` to `env_config`.

## Test process
 - Tests all continue to pass, and notebooks now run without error. This PR does not change any functionality, just adds some logging.
 - Searched for the catch-22 that would trip me up as part of this ticket. A 3 line change felt like too simple an implementation...

## Checklist
- [X] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [X] attended to any **TO-DOs** left in the code

Related work items: #2523
2024-04-30 17:13:50 +00:00
Charlie Crane
9c89fe4143 #2523 - Typo corrections within notebooks that stopped them from running 2024-04-30 16:14:34 +01:00
Chris McCarthy
04eb33e28a #2519 - Dropped failed unit test in test_wireless_router.py as it didn't actually test any functionality and simply just tested the config parser, which we now do in the integration test. 2024-04-30 16:07:38 +01:00
Charlie Crane
bb99e46651 #2523 - Updating CHANGELOG.md 2024-04-30 16:03:31 +01:00
Czar Echavez
ab3d23785d #2533: precommit 2024-04-30 15:50:53 +01:00
Chris McCarthy
bc573dfed0 #2519 - refactored all air-space usage to that a new instance of AirSpace is created for each instance of Network. This 1:1 relationship between network and airspace will allow parallelization. Added additional WirelessRouter test that tests wireless connectivity from config file too. 2024-04-30 15:46:16 +01:00
Czar Echavez
59990813f5 #2533: added tests + log level for output 2024-04-30 15:43:25 +01:00
Charlie Crane
9b3699389a #2523 - Adding in some additional logging for other agent classes. This currently prints total_reward instead of average reward 2024-04-30 15:36:59 +01:00
Christopher McCarthy
a82ee2b8e1 Merged PR 354: Network focussed dev config documentation
## Summary
Built three networks, one is the minimal network, one is a basic LAN, and another is a nice beefy multi-LAN setup with ISP and DNS along with applications, services. Documented the process when building the config files. Included the config files as example configs for use.

## Test process
Tested the shit out of the big multi-lan network.

## Checklist
- [ ] PR is linked to a **work item**
- [ ] **acceptance criteria** of linked ticket are met
- [ ] performed **self-review** of the code
- [ ] written **tests** for any new functionality added with this PR
- [ ] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [ ] updated the **change log**
- [ ] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

Related work items: #2266
2024-04-30 13:53:28 +00:00
Charlie Crane
b8c46a92e9 #2523 - Minor typos/corrections to make things run locally. Added one logging line to record Blue agent reward per step in environment.py 2024-04-30 14:16:29 +01:00
Chris McCarthy
a141c8d212 #2266 - fixed typos from PR suggestions 2024-04-30 14:08:12 +01:00
Nick Todd
137f2a9648 Merged PR 306: Fix issues with check_hash function.
## Summary
The current implementation of check_hash() as triggered by NODE_FILE_CHECKHASH and NODE_FOLDER_CHECKHASH is incorrect. However, a correct implementation is a future task so the code is disabled and a deprecation warning has been added.

## Test process
Disabled tests that exercise check_hash() functionality. Added tests to show that deprecation warning is present.

## Checklist
- [X ] PR is linked to a **work item**
- [X] **acceptance criteria** of linked ticket are met
- [X] performed **self-review** of the code
- [X] written **tests** for any new functionality added with this PR
- [X] updated the **documentation** if this PR changes or adds functionality
- [ ] written/updated **design docs** if this PR implements new functionality
- [X] updated the **change log**
- [X] ran **pre-commit** checks for code style
- [ ] attended to any **TO-DOs** left in the code

2299: Remove calls to corrupt.

Related work items: #2299
2024-04-30 12:08:19 +00:00
Nick Todd
3d0ebe1aba #2299: Correct possible mis-merge. 2024-04-30 10:39:33 +01:00
Nick Todd
9906ba20ae Merge branch 'dev' into bugfix/2299-check_hash_function_corrupts_files_and_folders 2024-04-30 09:17:40 +01:00
Marek Wolan
8a97d197b9 Merged PR 355: Document request success
## Summary
Add a doc page and a notebook to go over the request manager.

## Test process
Notebook runs.

## Checklist
- [x] PR is linked to a **work item**
- [x] **acceptance criteria** of linked ticket are met
- [x] performed **self-review** of the code
- [na] written **tests** for any new functionality added with this PR
- [x] updated the **documentation** if this PR changes or adds functionality
- [x] written/updated **design docs** if this PR implements new functionality
- [na] updated the **change log**
- [x] ran **pre-commit** checks for code style
- [x] attended to any **TO-DOs** left in the code

Related work items: #2514
2024-04-29 17:36:29 +00:00
Czar Echavez
c94fa653f7 Merge remote-tracking branch 'origin/dev' into feature/2533-improve-dev-tools-in-primaite-cli 2024-04-29 16:24:19 +01:00
Nick Todd
5b03013e3f Merge branch 'dev' into bugfix/2299-check_hash_function_corrupts_files_and_folders 2024-04-29 15:44:06 +01:00