1. Introduction
1.1 Scope
This document provides the Amazon Sidewalk reference test cases for certification of compliance with the Amazon Sidewalk specification.
1.1.1 Application
The specification includes test cases for Endpoint products, referred to as the Device Under Test (DUT). At this time there is no requirement for gateway certification, as all gateways on the Amazon Sidewalk network are Amazon devices, such as Echo and Ring products.
1.1.2 Naming Convention
Each test case has a short, unique identifier, and a title. The general format of a test case identifier is:
<AREA>/<ROLE>/<FEATURE>/<FUNCTION>/<CLASS>-<ID>
<AREA>can be: CMN (Common), BLE, FSK, LORA, or MULTI (multi-radio)<ROLE>can be: EP (Endpoint), or GW (Gateway, reserved for future use)<FEATURE>can be: BCN, CONN, DATA, API<FUNCTION>is<FEATURE>dependent<CLASS>can be: BV (Behaviour Valid), or BI (Behaviour Invalid)<ID>is a unique integer to differentiate multiple test cases under the preceding fields
For example FSK/EP/BCN/JOIN/BV-01 - an FSK Endpoint test case for discovering a beacon, with valid behavior.
1.1.2.1 Behaviour Valid (BV) Tests
These tests verify that the DUT reacts in conformity with the specification when receiving valid message sequences.
1.1.2.2 Behaviour Invalid (BI) Tests
These tests verify that the DUT reacts in conformity with the specification when receiving sequences containing invalid messages, missing messages, messages of incorrect type or content, and is expected to recover and resume the protocol function.
1.2 General Test Requirements
Each test case has an Expected Results section. In all cases this will describe the criteria for a Pass Verdict. In some cases it may also describe criteria that could lead to an Inconclusive Verdict - this refers to a result that is not recorded as a fail but may indicate an undetected or undesirable condition and should be reviewed.
If a DUT cannot achieve a Pass Verdict or Inconclusive Verdict then it is automatically classified as a test failure, however in some cases additional specific details may be provided under Fail Verdict indicating why a test could fail. For example a test that attempts to send 10 data packets may explicitly state that a Fail Verdict is recorded if the Tester receives 9 or fewer packets.
1.3 Test Environment
The test specification requires one or two tester “roles” - the Upper Tester and the Lower Tester. These refer to the roles undertaken by the test equipment to drive the upper (top of stack) and lower (bottom of stack) interfaces to the Device Under Test. The roles do not mandate a particular physical implementation of the test equipment. Typically a single piece of test equipment would implement both roles, however for some test cases it may be possible to separate the roles across two independent entities.
The DUT lower interface is the radio, therefore the Lower Tester will reproduce the functionality and required behavior of a standard Amazon Sidewalk Gateway.
The DUT upper interface is the on-chip Amazon Sidewalk application API (e.g. the sid_init, sid_start etc. API calls). It is the responsibility of the test equipment provider to specify how the Upper Tester role within the test equipment will communicate with the on-chip API (e.g. via command-line interface, binary transport, or some other mechanism).
1.4 References
TBD