9. Endpoint Metrics Test Cases

9.1 CMN/EP/EPMETRICS/METRICS/BV/01: Verify Endpoint has corresponding metrics but not invalid metrics enabled.

9.1.1 Test Purpose

DUT only has the metrics that correspond to enabled protocol stacks, with no irrelevant metrics present.

9.1.2 Initial Conditions

• DUT is registered with Amazon Sidewalk network.

9.1.3 Test Procedure

sequenceDiagram
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over DUT,Upper_Tester: iterate: Each Sidewalk operation mode
    Note over DUT,Upper_Tester: Sidewalk application started
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Note over DUT,Upper_Tester: end of iteration

9.1.4 Expected Results

9.1.4.1 Pass Verdict

DUT maintains only metrics associated with currently enabled protocol stacks, with no irrelevant metrics present.

9.2 BLE/EP/EPMETRICS/REPORT/BV/01: Verify Endpoint sends Endpoint Metrics to Cloud over BLE in a specific time interval.

9.2.1 Test Purpose

DUT demonstrates capability to transmit endpoint metrics through BLE connectivity based on the configured metrics periodicity.

9.2.2 Initial Conditions

• The Sidewalk stack (BLE) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in BLE mode.

9.2.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 0, BLE disconnected
    Note over Lower_Tester,Upper_Tester: Wait: metrics periodicity cycle - 1 minute post initial time synchronization
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Note over Lower_Tester,Upper_Tester: Wait: remaining 1 minute to complete periodicity cycle
    DUT->>Lower_Tester: Beacon with connection request
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.2.4 Expected Results

9.2.4.1 Pass Verdict

Following time synchronization, DUT successfully transmits metrics over BLE according to the configured periodicity.

DUT receives Cloud acknowledgment for the transmitted metrics package.

DUT resets metrics after periodic reporting completion.

9.3 BLE/EP/EPMETRICS/REPORT/BV/02: Verify Endpoint sends remaining Endpoint Metrics to Cloud over BLE via piggybacking on a periodic reporting.

9.3.1 Test Purpose

DUT implements metrics piggybacking over subsequent BLE uplinks to complete metrics transmission when single periodic uplink capacity is insufficient.

9.3.2 Initial Conditions

• The Sidewalk stack (BLE+FSK), (BLE+LORA) or (BLE+FSK+LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in BLE mode.

9.3.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 0, BLE disconnected
    Note over Lower_Tester,Upper_Tester: Configure individual metrics to their respective boundary values
    Upper_Tester->>DUT: metrics_set(Category_ID, Metric_ID, boundary_value)
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    DUT->>Lower_Tester: Beacon with connection request
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all remaining metrics are piggybacked via BLE
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.3.4 Expected Results

9.3.4.1 Pass Verdict

DUT transmits remaining metrics from periodic reporting via BLE uplink piggybacking.

9.4 BLE/EP/EPMETRICS/REPORT/BV/03: Verify Endpoint Metrics are uploaded via Piggybacking over BLE if the explicit reporting is disabled.

9.4.1 Test Purpose

When metrics capability is disabled, DUT is able to transmit endpoint metrics via BLE piggybacking instead of periodic reporting during regular reporting intervals.

9.4.2 Initial Conditions

• The Sidewalk stack (BLE) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability disabled.

• Lower Tester operates in BLE mode.

9.4.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 0, BLE disconnected
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    DUT->>Lower_Tester: Beacon with connection request
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all metrics are piggybacked via BLE
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.4.4 Expected Results

9.4.4.1 Pass Verdict

DUT transmits all metrics through BLE piggybacking instead of periodic reporting.

9.5 BLE/EP/EPMETRICS/UL/BV/01: Verify Uplink messages Metrics.

9.5.1 Test Purpose

DUT captures BLE uplink metrics.

9.5.2 Initial Conditions

• The Sidewalk stack (BLE) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in BLE mode.

9.5.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 0, BLE disconnected
    Upper_Tester->>DUT: sid_ble_bcn_connection_request(set)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Beacon with connection request
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_ul: 0"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_ul: 1"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_ul: 2"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: Continued on next page.

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: Continued from previous page
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_ul: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.5.4 Expected Results

9.5.4.1 Pass Verdict

DUT’s BLE uplink metrics increment after each BLE uplink transmission.

9.6 BLE/EP/EPMETRICS/DL/BV/01: Verify Downlink messages Metrics.

9.6.1 Test Purpose

DUT captures BLE downlink metrics.

9.6.2 Initial Conditions

• The Sidewalk stack (BLE) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in BLE mode.

9.6.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 0, BLE disconnected
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_dl: 0"
    Note over Lower_Tester,Upper_Tester: Continued on next page.

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: Continued from previous page
    Lower_Tester->>DUT: Establish BLE connection (triggered by upcoming downlink packets)
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_dl: 1"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_dl: 2"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_dl: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.6.4 Expected Results

9.6.4.1 Pass Verdict

DUT’s BLE downlink metrics increment after each BLE downlink reception.

9.7 BLE/EP/EPMETRICS/CONN-BV-01: Verify Connection attempts Metrics.

9.7.1 Test Purpose

DUT captures BLE connection attempts metrics.

9.7.2 Initial Conditions

• The Sidewalk stack (BLE) is initialized on the DUT.

• DUT is registered with Amazon Sidewalk network.

• Lower Tester operates in BLE mode.

9.7.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: The Sidewalk stack BLE is initialized but not started
    Upper_Tester->>DUT: sid_start(ble_start: true)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_conn_attempts: 1"
    DUT->>Lower_Tester: Beacon with time sync request
    Lower_Tester->>DUT: Establish BLE connection
    Lower_Tester->>DUT: RESP_TIME
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_conn_attempts: 1"
    Upper_Tester->>DUT: sid_ble_bcn_connection_request(set)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Beacon with connection request
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_conn_attempts: 2"
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    Note over Lower_Tester,Upper_Tester: Wait: 30 sec
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: metrics_send()
    DUT->>Lower_Tester: Beacon with connection request
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_BLE_LINK)
    DUT->>Upper_Tester: Print BLE_LINK metrics "num_conn_attempts: 3"
    Lower_Tester->>DUT: Establish BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 1, BLE connected"
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop BLE connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, BLE Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, BLE Disconnected"

9.7.4 Expected Results

9.7.4.1 Pass Verdict

DUT’s BLE connection attempts metrics increment after BLE start and each DUT-initiated BLE connection request.

9.8 FSK/EP/EPMETRICS/REPORT/BV/01: Verify Endpoint sends Endpoint Metrics to Cloud over FSK in a specific time interval.

9.8.1 Test Purpose

DUT demonstrates capability to transmit endpoint metrics through FSK connectivity based on the configured metrics periodicity.

9.8.2 Initial Conditions

• The Sidewalk stack (FSK) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in FSK mode.

9.8.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 2, FSK connected
    Note over Lower_Tester,Upper_Tester: Wait: metrics periodicity cycle - 1 minute post initial time synchronization
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Note over Lower_Tester,Upper_Tester: Wait: remaining 1 minute to complete periodicity cycle
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop FSK connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, FSK Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, FSK Disconnected"

9.8.4 Expected Results

9.8.4.1 Pass Verdict

Following time synchronization, DUT successfully transmits metrics over FSK according to the configured periodicity.

DUT receives Cloud acknowledgment for the transmitted metrics package.

DUT resets metrics after periodic reporting completion.

9.9 FSK/EP/EPMETRICS/REPORT/BV/02: Verify Endpoint sends remaining Endpoint Metrics to Cloud over FSK via piggybacking on a periodic reporting.

9.9.1 Test Purpose

DUT implements metrics piggybacking over subsequent FSK uplinks to complete metrics transmission when single periodic uplink capacity is insufficient.

9.9.2 Initial Conditions

• The Sidewalk stack (FSK) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in FSK mode.

9.9.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 2, FSK connected
    Note over Lower_Tester,Upper_Tester: Configure individual metrics to their respective boundary values
    Upper_Tester->>DUT: metrics_set(Category_ID, Metric_ID, boundary_value)
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Note over Lower_Tester,Upper_Tester: Continued on next page.

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: Continued from previous page
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all remaining metrics are piggybacked via FSK
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop FSK connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, FSK Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, FSK Disconnected"

9.9.4 Expected Results

9.9.4.1 Pass Verdict

DUT transmits remaining metrics from periodic reporting via FSK uplink piggybacking.

9.10 FSK/EP/EPMETRICS/REPORT/BV/03: Verify Endpoint Metrics are uploaded via Piggybacking over FSK if the explicit reporting is disabled.

9.10.1 Test Purpose

When metrics capability is disabled, DUT is able to transmit endpoint metrics via FSK piggybacking instead of periodic reporting during regular reporting intervals.

9.10.2 Initial Conditions

• The Sidewalk stack (FSK) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability disabled.

• Lower Tester operates in FSK mode.

9.10.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 2, FSK disconnected
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all metrics are piggybacked via FSK
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop FSK connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, FSK Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, FSK Disconnected"

9.10.4 Expected Results

9.10.4.1 Pass Verdict

DUT transmits all metrics through FSK piggybacking instead of periodic reporting.

9.11 FSK/EP/EPMETRICS/UL/BV/01: Verify Uplink messages Metrics and Total number of TX unique packets Metrics.

9.11.1 Test Purpose

DUT captures FSK uplink metrics and TX packets metrics.

9.11.2 Initial Conditions

• The Sidewalk stack (FSK) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in FSK mode.

9.11.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 2, FSK connected
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_ul: 0"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_TX: 0"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_ul: 1"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_TX: 1"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_ul: 2"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_TX: 2"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_ul: 3"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_TX: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop FSK connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, FSK Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, FSK Disconnected"

9.11.4 Expected Results

9.11.4.1 Pass Verdict

DUT’s FSK uplink metrics increment after each FSK uplink transmission.

DUT’s FSK TX packets metrics increment after each FSK packet transmission.

9.12 FSK/EP/EPMETRICS/DL/BV/01: Verify Downlink messages Metrics and Total number of RX unique packets Metrics.

9.12.1 Test Purpose

DUT captures FSK downlink metrics and RX packets metrics.

9.12.2 Initial Conditions

• The Sidewalk stack (FSK) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in FSK mode.

9.12.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 2, FSK connected
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_dl: 0"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_RX: 0"
    Note over Lower_Tester,Upper_Tester: Continued on next page.

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: Continued from previous page
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_dl: 1"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_RX: 1"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_dl: 2"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_RX: 2"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_LINK)
    DUT->>Upper_Tester: Print FSK_LINK metrics "num_dl: 3"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_FSK_MAC)
    DUT->>Upper_Tester: Print FSK_MAC metrics "num_RX: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop FSK connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, FSK Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, FSK Disconnected"

9.12.4 Expected Results

9.12.4.1 Pass Verdict

DUT’s FSK downlink metrics increment after each FSK downlink reception.

DUT’s FSK RX packets metrics increment after each FSK packet reception.

9.13 LORA/EP/EPMETRICS/REPORT/BV/01: Verify Endpoint sends Endpoint Metrics to Cloud over LoRa in a specific time interval.

9.13.1 Test Purpose

DUT demonstrates capability to transmit endpoint metrics through LoRa connectivity based on the configured metrics periodicity.

9.13.2 Initial Conditions

• The Sidewalk stack (LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in LoRa mode.

9.13.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 4, LoRa connected
    Note over Lower_Tester,Upper_Tester: Wait: metrics periodicity cycle - 1 minute post initial time synchronization
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Note over Lower_Tester,Upper_Tester: Wait: remaining 1 minute to complete periodicity cycle
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: Print all applicable metrics for enabled stacks
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop LoRa connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, LoRa Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, LoRa Disconnected"

9.13.4 Expected Results

9.13.4.1 Pass Verdict

Following time synchronization, DUT successfully transmits metrics over LoRa according to the configured periodicity.

DUT receives Cloud acknowledgment for the transmitted metrics package.

DUT resets metrics after periodic reporting completion.

9.14 LORA/EP/EPMETRICS/REPORT/BV/02: Verify Endpoint sends remaining Endpoint Metrics to Cloud over LoRa via piggybacking on a periodic reporting.

9.14.1 Test Purpose

DUT implements metrics piggybacking over subsequent LoRa uplinks to complete metrics transmission when single periodic uplink capacity is insufficient.

9.14.2 Initial Conditions

• The Sidewalk stack (LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability enabled.

• Lower Tester operates in LoRa mode.

9.14.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 4, LoRa connected
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    DUT->>Lower_Tester: NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_sent()
    Lower_Tester->>DUT: ACK_NTFY_READ_METRICS_FWK
    DUT->>Upper_Tester: on_metrics_received()
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all remaining metrics are piggybacked via LoRa
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop LoRa connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, LoRa Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, LoRa Disconnected"

9.14.4 Expected Results

9.14.4.1 Pass Verdict

DUT transmits remaining metrics from periodic reporting via LoRa uplink piggybacking.

9.15 LORA/EP/EPMETRICS/REPORT/BV/03: Verify Endpoint Metrics are uploaded via Piggybacking over LoRa if the explicit reporting is disabled.

9.15.1 Test Purpose

When metrics capability is disabled, DUT is able to transmit endpoint metrics via LoRa piggybacking instead of periodic reporting during regular reporting intervals.

9.15.2 Initial Conditions

• The Sidewalk stack (LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• DUT is configured with metrics capability disabled.

• Lower Tester operates in LoRa mode.

9.15.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 4, LoRa disconnected
    Note over Lower_Tester,Upper_Tester: Manually trigger periodic metrics reporting
    Upper_Tester->>DUT: metrics_send()
    Note over Lower_Tester,Upper_Tester: loop: Send uplinks until all metrics are piggybacked via LoRa
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: 1B)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD (with 1B of Command data and piggybacked metrics)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: end of loop
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop LoRa connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, LoRa Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, LoRa Disconnected"

9.15.4 Expected Results

9.15.4.1 Pass Verdict

DUT transmits all metrics through LoRa piggybacking instead of periodic reporting.

9.16 LORA/EP/EPMETRICS/UL/BV/01: Verify Uplink messages Metrics and Total number of TX normal packets Metrics.

9.16.1 Test Purpose

DUT captures LoRa uplink metrics and TX normal packets metrics.

9.16.2 Initial Conditions

• The Sidewalk stack (LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in LoRa mode.

9.16.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 4, LoRa connected
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_ul: 0"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_TX: 0"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_ul: 1"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_TX: 1"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_ul: 2"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Note over Lower_Tester,Upper_Tester: Continued on next page.

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: Continued from previous page
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_TX: 2"
    Upper_Tester->>DUT: sid_put_msg(SID_MSG_TYPE_NOTIFY, Command data: any)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE "TX CRID: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_ul: 3"
    DUT->>Lower_Tester: CUSTOM_NOTIFY_CMD(with the Command data)
    DUT->>Upper_Tester: on_msg_sent()
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_TX: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop LoRa connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, LoRa Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, LoRa Disconnected"

9.16.4 Expected Results

9.16.4.1 Pass Verdict

DUT’s LoRa uplink metrics increment after each LoRa uplink transmission.

DUT’s LoRa TX packets metrics increment after each LoRa packet transmission.

9.17 LORA/EP/EPMETRICS/DL/BV/01: Verify Downlink messages Metrics and Total number of RX packets Metrics.

9.17.1 Test Purpose

DUT captures LoRa downlink metrics and RX packets metrics.

9.17.2 Initial Conditions

• The Sidewalk stack (LoRa) is initialized and started on the DUT.

• DUT is registered with Amazon Sidewalk network.

• DUT is time-synced with Amazon Sidewalk network.

• Lower Tester operates in LoRa mode.

9.17.3 Test Procedure

sequenceDiagram
    participant Lower_Tester as Lower Tester
    participant DUT as DUT
    participant Upper_Tester as Upper Tester
    Note over Lower_Tester,Upper_Tester: DUT Status: Registered, Time Synced, Link Mask 4, LoRa connected
    Upper_Tester->>DUT: clear_metrics(SID_METRICS_CAT_ALL)
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_dl: 0"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_RX: 0"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_dl: 1"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_RX: 1"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_dl: 2"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_RX: 2"
    Lower_Tester->>DUT: CUSTOM_NOTIFY_CMD (with any Command data)
    DUT->>Upper_Tester: on_msg_received() "Command data: any"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_LINK)
    DUT->>Upper_Tester: Print LORA_LINK metrics "num_dl: 3"
    Upper_Tester->>DUT: print_metrics(SID_METRICS_CAT_LORA_MAC)
    DUT->>Upper_Tester: Print LORA_MAC metrics "num_RX: 3"
    Upper_Tester->>DUT: sid_stop()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Lower_Tester: Drop LoRa connection
    DUT->>Upper_Tester: on_status_changed() "Time Synced, Link Mask 0, LoRa Disconnected"
    Upper_Tester->>DUT: sid_deinit()
    DUT->>Upper_Tester: #35;SID_ERROR_NONE
    DUT->>Upper_Tester: on_status_changed() "No Time, Link Mask 0, LoRa Disconnected"

9.17.4 Expected Results

9.17.4.1 Pass Verdict

DUT’s LoRa downlink metrics increment after each LoRa downlink reception.

DUT’s LoRa RX packets metrics increment after each LoRa normal packet reception.

---