| Case | Step | Event | airmarket | iart_dss | onesky | <None> | ||
|---|---|---|---|---|---|---|---|---|
| 1 | 📓 |
Tested USS: airmarket
|
||||||
| 2 | 📓 |
Control USS: onesky
|
||||||
| Prerequisites check | Verify area is clear | 3 | 🌐 |
|
🌐 | |||
| 4 | ✅ | DSS responses | ✅ | |||||
| 5 | ✅ | Area is clear of op intents | ✅ | |||||
| Attempt to plan flight in conflict | Plan Flight 2 | 6 | 🌐 |
|
🌐 | |||
| 7 | ✅ | DSS responses | ✅ | |||||
| 8 | 🌐 |
|
🌐 | |||||
| 9 | ✅ | Failure | ✅ | |||||
| 10 | ✅ | Successful planning | ✅ | |||||
| 11 | 🌐 |
|
🌐 | |||||
| 12 | ✅ | DSS responses | ✅ | |||||
| 13 | ✅ | Operational intent shared correctly | ✅ | |||||
| 14 | ✅ | Operational intent state is correct | ✅ | |||||
| 15 | 🌐 |
|
🌐 | |||||
| 16 | ✅ | Operational intent details retrievable | ✅ | |||||
| 17 | ✅ | Operational intent details data format | ✅ | |||||
| 18 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 19 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 20 | ✅ | Correct operational intent details | ✅ | |||||
| 21 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 22 | ✅ | Off-nominal volumes | ✅ | |||||
| 23 | ✅ | Vertices | ✅ | |||||
| Attempt to plan Flight 1 | 24 | 🌐 |
|
🌐 | ||||
| 25 | ✅ | DSS responses | ✅ | |||||
| 26 | 🌐 |
|
🌐 | |||||
| 27 | ✅ | Failure | ✅ | |||||
| 28 | ✅ | Incorrectly planned | ✅ | |||||
| 29 | 🌐 |
|
🌐 | |||||
| 30 | ✅ | DSS responses | ✅ | |||||
| 31 | ✅ | Operational intent not shared | ✅ | |||||
| Delete Flight 2 | 32 | 🌐 |
|
🌐 | ||||
| 33 | ✅ | Successful deletion | ✅ | |||||
| Attempt to modify planned flight in conflict | Plan Flight 1 | 34 | 🌐 |
|
🌐 | |||
| 35 | ✅ | DSS responses | ✅ | |||||
| 36 | 🌐 |
|
🌐 | |||||
| 37 | ✅ | Failure | ✅ | |||||
| 38 | ✅ | Successful planning | ✅ | |||||
| 39 | 🌐 |
|
🌐 | |||||
| 40 | ✅ | DSS responses | ✅ | |||||
| 41 | ✅ | Operational intent shared correctly | ✅ | |||||
| 42 | ✅ | Operational intent state is correct | ✅ | |||||
| 43 | 🌐 |
|
🌐 | |||||
| 44 | ✅ | Operational intent details retrievable | ✅ | |||||
| 45 | ✅ | Operational intent details data format | ✅ | |||||
| 46 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 47 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 48 | ✅ | Correct operational intent details | ✅ | |||||
| 49 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 50 | ✅ | Off-nominal volumes | ✅ | |||||
| 51 | ✅ | Vertices | ✅ | |||||
| Plan Flight 2 | 52 | 🌐 |
|
🌐 | ||||
| 53 | ✅ | Retrieve pre-existing notifications | ✅ | |||||
| 54 | 🌐 |
|
🌐 | |||||
| 55 | ✅ | Retrieve pre-existing notifications | ✅ | |||||
| 56 | 🌐 |
|
🌐 | |||||
| 57 | ✅ | DSS responses | ✅ | |||||
| 58 | 🌐 |
|
🌐 | |||||
| 59 | ✅ | Failure | ✅ | |||||
| 60 | ✅ | Successful planning | ✅ | |||||
| 61 | 🌐 |
|
🌐 | |||||
| 62 | ✅ | DSS responses | ✅ | |||||
| 63 | ✅ | Operational intent shared correctly | ✅ | |||||
| 64 | ✅ | Operational intent state is correct | ✅ | |||||
| 65 | 🌐 |
|
🌐 | |||||
| 66 | ✅ | Operational intent details retrievable | ✅ | |||||
| 67 | ✅ | Operational intent details data format | ✅ | |||||
| 68 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 69 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 70 | ✅ | Correct operational intent details | ✅ | |||||
| 71 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 72 | ✅ | Off-nominal volumes | ✅ | |||||
| 73 | ✅ | Vertices | ✅ | |||||
| Check for conflict notifications | 75 | 🌐 |
|
🌐 | ||||
| 76 | ✅ | Retrieve notifications | ✅ | |||||
| 77 | 🌐 |
|
🌐 | |||||
| 78 | ✅ | Retrieve notifications | ✅ | |||||
| 79 | 🌐 |
|
🌐 | |||||
| 80 | ✅ | Retrieve notifications | ✅ | |||||
| 81 | 🌐 |
|
🌐 | |||||
| 82 | ✅ | Retrieve notifications | ✅ | |||||
| 83 | 🌐 |
|
🌐 | |||||
| 84 | ✅ | Retrieve notifications | ✅ | |||||
| 85 | 📓 |
scd0090_notification: onesky notification latency: >0s
|
||||||
| Attempt to modify planned Flight 1 in conflict | 86 | 🌐 |
|
🌐 | ||||
| 87 | ✅ | DSS responses | ✅ | |||||
| 88 | 🌐 |
|
🌐 | |||||
| 89 | ✅ | Failure | ✅ | |||||
| 90 | ✅ | Incorrectly modified | ✅ | |||||
| 91 | 🌐 |
|
🌐 | |||||
| 92 | ✅ | DSS responses | ✅ | |||||
| 93 | ✅ | Operational intent shared correctly | ✅ | |||||
| 94 | ✅ | Operational intent state is correct | ✅ | |||||
| 95 | 🌐 |
|
🌐 | |||||
| 96 | ✅ | Operational intent details retrievable | ✅ | |||||
| 97 | ✅ | Operational intent details data format | ✅ | |||||
| 98 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 99 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 100 | ✅ | Correct operational intent details | ✅ | |||||
| 101 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 102 | ✅ | Off-nominal volumes | ✅ | |||||
| 103 | ✅ | Vertices | ✅ | |||||
| Attempt to activate flight in conflict | Attempt to activate conflicting Flight 1 | 104 | 🌐 |
|
🌐 | |||
| 105 | ✅ | DSS responses | ✅ | |||||
| 106 | 🌐 |
|
🌐 | |||||
| 107 | ✅ | Failure | ✅ | |||||
| 108 | ✅ | Incorrectly activated | ✅ | |||||
| 109 | 🌐 |
|
🌐 | |||||
| 110 | ✅ | DSS responses | ✅ | |||||
| 111 | ✅ | Operational intent shared correctly | ✅ | |||||
| 112 | ✅ | Operational intent state is correct | ✅ | |||||
| 113 | 🌐 |
|
🌐 | |||||
| 114 | ✅ | Operational intent details retrievable | ✅ | |||||
| 115 | ✅ | Operational intent details data format | ✅ | |||||
| 116 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 117 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 118 | ✅ | Correct operational intent details | ✅ | |||||
| 119 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 120 | ✅ | Off-nominal volumes | ✅ | |||||
| 121 | ✅ | Vertices | ✅ | |||||
| Modify activated flight with pre-existing conflict | Delete Flight 2 | 122 | 🌐 |
|
🌐 | |||
| 123 | ✅ | Successful deletion | ✅ | |||||
| Activate Flight 1 | 124 | 🌐 |
|
🌐 | ||||
| 125 | ✅ | DSS responses | ✅ | |||||
| 126 | 🌐 |
|
🌐 | |||||
| 127 | ✅ | Failure | ✅ | |||||
| 128 | ✅ | Successful activation | ✅ | |||||
| 129 | 🌐 |
|
🌐 | |||||
| 130 | ✅ | DSS responses | ✅ | |||||
| 131 | ✅ | Operational intent for active flight not deleted | ✅ | |||||
| 132 | ✅ | Operational intent shared correctly | ✅ | |||||
| 133 | ✅ | Operational intent state is correct | ✅ | |||||
| 134 | 🌐 |
|
🌐 | |||||
| 135 | ✅ | Operational intent details retrievable | ✅ | |||||
| 136 | ✅ | Operational intent details data format | ✅ | |||||
| 137 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 138 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 139 | ✅ | Correct operational intent details | ✅ | |||||
| 140 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 141 | ✅ | Off-nominal volumes | ✅ | |||||
| 142 | ✅ | Vertices | ✅ | |||||
| Plan Flight 2 | 143 | 🌐 |
|
🌐 | ||||
| 144 | ✅ | DSS responses | ✅ | |||||
| 145 | 🌐 |
|
🌐 | |||||
| 146 | ✅ | Failure | ✅ | |||||
| 147 | ✅ | Successful planning | ✅ | |||||
| 148 | 🌐 |
|
🌐 | |||||
| 149 | ✅ | DSS responses | ✅ | |||||
| 150 | ✅ | Operational intent shared correctly | ✅ | |||||
| 151 | ✅ | Operational intent state is correct | ✅ | |||||
| 152 | 🌐 |
|
🌐 | |||||
| 153 | ✅ | Operational intent details retrievable | ✅ | |||||
| 154 | ✅ | Operational intent details data format | ✅ | |||||
| 155 | ✅ | Operational intent reference reported by USS matches the one published to the DSS | ✅ | |||||
| 156 | ✅ | Operational intent details have not changed without publishing a new version to the DSS | ✅ | |||||
| 157 | ✅ | Correct operational intent details | ✅ | |||||
| 158 | ✅ | Operational intent details extents are contained within reference extents | ✅ | |||||
| 159 | ✅ | Off-nominal volumes | ✅ | |||||
| 160 | ✅ | Vertices | ✅ | |||||
| Activate Flight 2 | 161 | 🌐 |
|
🌐 | ||||
| 162 | ✅ | Retrieve pre-existing notifications | ✅ | |||||
| 163 | 🌐 |
|
🌐 | |||||
| 164 | ✅ | Retrieve pre-existing notifications | ✅ | |||||
| 165 | 🌐 |
|
🌐 | |||||
| 166 | ✅ | DSS responses | ✅ | |||||
| 167 | 🌐 |
|
🌐 | |||||
| 168 | 🛑 |
Failure
[167]
Flight planning activity PlanningActivityResult.Failed leaving flight plan FlightPlanStatus.NotPlanned onesky indicated flight planning activity PlanningActivityResult.Failed leaving flight plan FlightPlanStatus.NotPlanned rather than the expected (Activity Completed, flight plan OkToFly) with notes "Timed out waiting for flight creation/update" |
🛑 | |||||
| 169 | ✅ | Successful activation | ✅ | |||||
| Cleanup | 170 | 🌐 |
|
🌐 | ||||
| 171 | ⚠️ |
Successful flight deletion
[170]
Failed to clean up flight 22ff43ee-719d-486c-b4cf-560b3fb1c020 from airmarket Attempt to delete flight plan returned status 401 rather than 200 as expected Stack trace: Traceback (most recent call last): File "/app/monitoring/uss_qualifier/scenarios/flight_planning/test_steps.py", line 458, in cleanup_flights resp = flight_planner.try_end_flight( flight_id, ExecutionStyle.IfAllowed ) File "/app/monitoring/monitorlib/clients/flight_planning/client_v1.py", line 156, in try_end_flight raise PlanningActivityError( ...<2 lines>... ) monitoring.monitorlib.clients.flight_planning.client.PlanningActivityError: Attempt to delete flight plan returned status 401 rather than 200 as expected |
⚠️ | |||||