CoAP over GATT (Bluetooth Low Energy Generic Attributes)

Introduction The Constrained Application Protocol (CoAP) can be used with different network and transport technologies, for example UDP on 6LoWPAN networks. Not all those network technologies are available at end user devices in the vicinity of the constrained devices, which inhibits direct communication and necessitates the use of gateway devices or cloud services. In particular, 6LoWPAN is not available at all in typical end user devices, and while 6LoWPAN-over-BLE (IPSP, the Internet Protocol Support Profile of Bluetooth Low Energy (BLE), ) might be compatible from a radio point of view, many operating systems or platforms lack support for it, especially in a user-accessible way. As a workaround to access constrained CoAP devices from end user devices, this document describes a way encapsulate generic CoAP exchanges in Bluetooth GATT (Generic Attribute Profile). This is explicitly not designed as means of communication between two devices in full control of themselves -- those should rather build an IP based network and transport CoAP as originally specified. It is intended as a means for an application to escape the limitations of its environment, with a special focus on web applications that use the Web Bluetooth . In that, it is similar to CoAP-over-WebSockets . GATT, which has read and write semantics, is not a perfect match for CoAP's request/response semantics; this specification bridges the gap in order to make CoAP transportable over what is sometimes the only available protocol.

Application example Consider a network of home automation light bulbs and switches, which internally uses CoAP on a 6LoWPAN network and whose basic pairing configuration can be done without additional electronic devices. Without CoAP-over-GATT, an application that offers advanced configuration requires the use of a dedicated gateway device or a router that is equipped and configured to forward between the 6LoWPAN and the local network. In practice, this is often delivered as a wired gateway device and a custom app. With CoAP-over-GATT, the light bulbs can advertise themselves via BLE, and the configuration application can run as a web site. The user navigates to that web site, and it asks permission to contact the light bulbs using Web Bluetooth. The web application can then exchange CoAP messages directly with the light bulb, and have it proxy requests to other devices connected in the 6LoWPAN network. For browsers that do not support Web Bluetooth, the same web application can be packaged into an native application consisting of a proxy process that forwards requests received via CoAP-over-WebSockets on the loopback interface to CoAP-over-GATT, and a browser view that runs the original web application in a configuration to use WebSockets rather than CoAP-over-GATT. That connection is no replacement when remote control of the system is desired (in which case, again, a router is required that translates 6LoWPAN to the rest of the network), but suffices for many commissioning tasks.

Alternatives Several approaches were considered, but considered unsuitable for the intended use cases:

CoAP over 6LoWPAN over BLE (BLE IPSP): While this is the natural choice for transporting CoAP over BLE, it is unavailable on typical end user devices. There is no clear path toward how that would be integrated in platforms like Android or iOS, and even if it were, creating a network connection to a nearby device from within an application might not be possible (if how WLAN networks are managed is any indication). [ TBD: Illustrate how easy IPSP is when only working link-local like CoAP-over-GATT does, see also https://gitlab.com/chrysn/coap-over-gatt/-/issues/10. ]
GoldenGate : This introduces significant network overhead, and burdens the end user device application with shipping a full network stack that is executed in a position where it can not integrate fully with the operating system's network stack. Moreover, this places a retransmission layer on top of a partially reliable transport (GATT), duplicating effort and possibly aggravating congestion situations.
CoAP over UDP over SLIP over GATT UART : This is similar to the GoldenGate approach, but built on the GATT UART provided with Nordic Semiconductor's libraries. This shares the network stack duplication and retransmission concerns of GoldenGate.
slipmux over BLE GATT UART service: This is similar to the previous item; the stack duplication concern is addressed, but retransmissions are still active atop of a service that already provides some reliability.

Protocol description

Boundary conditions: GATT properties [ This section may be shortened in later iterations, but is kept around while the protocol is being developed to easily fix mistakes made from wrong assumptions. ] CoAP-over-GATT has different properties than UDP transported over the Internet:

Messages sent by one party are received by the other party in the order in which they are sent. There is no re-ordering. (There is also a total order on messages sent by any party, but that property is not useful because it's often not accessible through the Bluetooth stacks.)
There is limited reliabiliy built into the protocol. Data transmissions initiated by the data source can be unreliable ("write without response", "notify") or reliable ("write with response", "indicate"). The caveat with their relability is that acknowledgements are sent by the BLE stack, without consulting with the application. (This is not only done for simplicity but also for power efficiency: There is only a short time window in which the data source is listening for confirmations). Thus, these confirmations can not serve to acknowledge that the a CoAP request contained in the event was read, understood and is being processed. The reliability mechanisms are still useful, though: Both "write" and "notify"/"indicate" update the GATT characteristic's state, and while a slow application may miss data when sent in fast succession, it is reasonable to expect from the BLE stack to deliver the last data to the application when no more data is sent.
Reads and writes may be subtly confused: When a characteristic is written to, and it is read before the BLE server application has had time to interact with its BLE stack, the written value may be echoed back at read time. This is likely not problematic when "notify"/"indicate" is used instead of polling reads, but it seems prudent to take precautions.

Requests and responses CoAP-over-GATT uses a GATT Characteristics to transport requst and response messages. Similar CoAP-over-UDP it offers both reliable and unreliable transfer and message deduplication, but as GATT's properties (see ) differ from UDP's, it uses a different serialization and a different kind of message IDs. Tokens are used like with other CoAP transports, and allow keeping multiple requests active at the same time. A GATT server announces service of UUID 8df804b7-3300-496d-9dfa-f8fb40a236bc (abbreviated US in this document), with one or more pairs of characteristics of UUID 8bf52767-5625-43ca-a678-70883a366866 (the downstream characteristic, abbreviated UCD) and ab3720c8-7fc0-41f8-aa2a-9a45c2c01a4b (the upstream characteristic, abbreviated UCU) through BLE advertisements from a BLE peripheral (typically a constrained device), which are discovered by a BLE central (typically an end user device). The server and client roles of CoAP and GATT are independent of each other: either BLE participant can send requests in a CoAP client role. It is expected that as this document matures, shorter (16 or 32 bit) identifiers will be requested and assigned. [ See also https://gitlab.com/chrysn/coap-over-gatt/-/issues/7. ]

Message sub-layer At the UCU/UCD pair of CoAP-over-GATT characteristics, each party maintains a single bit Message ID (initialized at 1 when a connection is created), and the last Message ID sent by the peer (initialized at 0 when a connection is created). Messages are serialized as GATT values. The GATT client sends a message by writing it to UCD (reliably using the "write with response" or unreliably using "write without response" operation); the GATT server sends them reliably using an "indicate" or unreliably "notify" event on UCU. The serialization format is the same for all, and illustrated in :

Components of a message

a single message description byte, compose of 4 bits R (reserved), M (Message ID), C (Confirm) and A (Acknowledge ID), followed by 4 bits of token length (TKL).
Code, token, options, payload marker and payload as in . Unlike there, there is no 16-bit Message ID field (a similar role is taken by bits M and A), and in empty messages, the code is not sent.

The bits are set as follows:

The R bit is reserved for future extensions; it MUST be written as 0, and writes with values of 1 MUST be ignored.
The Message ID bit is always set to the current Message ID of the sender.
The Confirm bit is set if the sender asks the peer to acknowledge that the message has been noted.
The Acknowledge ID is always set to the peer's last sent Message ID that had the Confirm bit set.

When receiving a message with the C bit set, the recipient MUST eventually send a response message with radio reliability.

Using the message sub-layer [ This section reflects ongoing experimentation with the above serialization format and rules. Senders may use other patterns as long as they do not stall their peer by not sending any messages after the Confirm bit was set. ] To send a message unreliably in terms of CoAP transmission, a sender sets its latest Message ID in the M bit, sets C to 0, and populates the remaining bits per the rules above. It then sends the message unreliably on the radio (it may be sent reliably, especially when the peer set the C bit before). After a CoAP-unreliable message, the sender may send more CoAP-unreliable messages. It should avoid sending multiple messages in the same connection event (because the peer's BLE stack would be likely to not pass on the earlier message). To send a message reliably in terms of CoAP transmission, a sender sets its latest Message ID in the M bit, sets C to 1, and populates the remaining bits per the rules above. It then sends the message reliably on the radio (it may send unreliably if a message is expected from the peer soon, but then needs to be prepared to send the same message again). After sending that message, the sender does not send any other message until a message is received with A equal to the sent message's M bit. The sender may need to send the very same message again if no earlier transmission of the message happened reliably. [ Do we need to give timing guidance here? Probably not, because it only happens if there is some expectation in the first place. ] The sender may cancel the transmission by sending an empty message with the same M and C bits, or by sending different message with these bits (which are then all unreliable transmissions). When receiving a message with the C bit set, it is up to the recipient when to send the radio-reliable message. If it is expected that a radio-reliable message will be sent soon, it is permissible and useful to send unrelated unreliable messages that already account for the set C bit in their A bit.

Message deduplication CoAP-over-GATT participants MUST ignore a message arriving at a characteristic if it is identical to the one received previously in the same connection. (The first message is never ignored). Note that it is not possible to send two identical consecutive messages unreliably. When sending identical requests, the sender may vary the token. Sending identical responses generally is rarely significant, even with the generalized , because the mechanism to make responses "non-matching" in that document's terminology typically incurs variation. When it does not, but the repetition is still significant, sending the messages reliably becomes necessary.

Requests and responses CoAP requests and responses are built on the message sub-layer as they are in : requests are sent with a token chosen by the CoAP client, and the CoAP server sends a response with the same token. Responses and message-layer acknowledgments can happen in the same message. Unlike in , there is no association between a request and its message ID: Any message may serve as an acknowledgement; it is always only the token that matches requests to responses.

Fragmentation Attribute values are limited to 512 Bytes ( Part F Section 3.2.9), practically limiting blockwise operation () to size exponents to 4 (resulting in a block size of 256 byte). Even smaller messages might enhance the transfer efficiency when they avoid fragmentation at the L2CAP level. [ TBD: Verify: ]

Multiple characteristics If a server provides multiple UCU and UCD typed characteristics, they form pairs in the sequence in which they are listed. By using them in parallel, multiple messages can be sent without waiting for individual confirmation. This is similar to using RFC7252 with NSTART > 1, and may be used by the GATT client if the GATT server lists multiple pairs of UCU/UCD characteristics. The GATT server can send messages only through UCU characteristics on which the GATT client enabled "indicate" or "notify"; if the GATT client does not support multiple characteristics, it will just pick any and only enable them on that one. Each characteristic has its independent message ID bits. All characteristics of a service share a single token space, and responses need not necessarily be sent on the characteristic the request was sent on. The use of muliple characteristics is primarily practical when large amounts of data are to be transferred. These transfers can utilize much of BLE's bandwidth because they make it easy to send much data within a single BLE connection event.

Communication example The example illustrated in shows an observation request with reliable and unreliable responses. It chooses the most typical configuration where the GATT server is also the BLE peripheral (and thus sends avertisements). The GATT client is also the CoAP client here.

Example message flow (Pairing in Just-Works mode and discovery not illustrated) <----- Write+Resp. M=1 C=1 A=0 T="01" GET /temp, Observe: 0 (The server sends temperature values unreliably for some time) Notify M=1 C=0 A=1 T="01" 2.05 Content, Obs: 1, "22°C" ---> Notify M=1 C=0 A=1 T="01" 2.05 Content, Obs: 2, "21°C" ---> <----- Write+Resp. M=0 C=1 A=0 T="02" GET /model Indicate M=1 C=1 A=0 T="02" 2.05 Content, "ExampleScan" --> <----- Write+Resp. M=0 C=0 A=1 empty Notify M=0 C=0 A=0 T="01" 2.05 Content, Obs: 3, "20°C" ---> (At this point, the temperature isn't changing for some time, and the server sends a reliable notification) Indicate M=0 C=1 A=0 T="01" 2.05 Content, Obs: 4, "20°C" -> <----- Write+Resp. M=0 C=0 A=0 empty ]]>

Development directions

Is there any good reason to allow read operations? A GATT client that is waiting for a Confirm bit to be acknowledged might attempt a Read (for the case that the confirmation arrived in an unreliable message), but might just as well perform the last write again. Reading would be more efficient (because it can happen without application intervention, and no data is sent), but the added complexity might not be worth the enhancements.
Fragmentation. If the current approach of requiring devices to support large MTU sizes turns out to be impractical, or if GATT level fragmentation vastly outperforms CoAP fragmentation, it may be necessary to use composite reads and writes on GATT. Care has to be taken to use only operations supported by : that API does not expose reads with offsets. Offset based fragmentation may also be incompatible with the write-with-response approach suggested for reliability.
Usability from WebBluetooth WebBluetooth clients may be unaware that two protocol instances are running between the client and the server at the same time, without any indication on the BLE side. Is there anything this protocol can do to help the clients discover (or even resolve) the situation? See also https://gitlab.com/chrysn/coap-over-gatt/-/issues/9.

Addresses The URI scheme associated with CoAP over GATT is "coap" as per the recommendation of . The default value of Uri-Host is the MAC address of the CoAP server, in hexadecimal encoding, followed by .ble.arpa. The use of .ble.alt as defined in was considered instead of .ble.arpa, but rejected for lack of management of its subdomains. Language from the .alt specification may be used when it comes to describing how this is not disturbing DNS operations. User information and port are always absent with this scheme. Assembling the URI of a request for the discovery resource of a BLE device with the MAC address 00:11:22:33:44:55 would thus be assembled, under the rules of , to coap://001122334455.ble.arpa/.well-known/core. Locally defined host or service name registries may be used to create names that are more suitable for human interaction. For DNS, which is widely used for this purpose, no record types are registered that map to Bluetooth MAC addresses at the time of writing. Note that on some platforms (e.g. Web Bluetooth ), the peer's or the own address may not be known application. They may come up with an application-internal registered name component (e. g. coap://id-SomeInternalIdentifier.alt/.well-known/core, in this case using the .alt zone from ), but must be aware that those can not be expressed towards anything outside the local stack -- the same way they would avoid using IPv6 zone identifiers or URIs whose host name is localhost.

Use with persistent addresses When services are meant to provide long-lived and universally usable URIs, addresses based on MAC addresses can be impractical, because they fluctuate on hardware changes. (Moreover, privacy mechanisms on the device or the platform can render them unusable even before hardware changes). In the absence of a usable host or service name registry, implementers may opt for non-GATT addresses right away. provides the means to advertise a different canonical address, and to announce availability of that advertised service on the present transport. When long-lived addresses circumvent privacy preserving measures, considerations concering the tracking of devices [ are TBD along the lines of "don't make it discoverable to unauthorized sources, and in case of doubt let the peer show its credentials first" ].

Compression and reinterpretation of non-CoAP characteristics The use of SCHC is being evaluated in combination with CoAP-over-GATT; the device can use the characteristic UUID to announce the static context used. Together with non-traditional response forms ( and contexts that expand, say, a numeric value 0x1234 to a message like This enables a different use case than dealing with limited environments: Accessing BLE devices via CoAP without application specific gateways. Any required information about the application can be expressed in the SCHC context.

Additional use of advertisements In the current specification, advertisements are used to indicate that CoAP-over-GATT is being used. If Service Data is transported in the advertisement, it contains an identifier of the device in the ble-sd.arpa zone, such that the lower case hexadecimal representation of the Service Data value is prepended to .ble-sd.arpa to form a name for the device. There is no expectation for these names to be globally unique: considerations for beacon lengths may require them to be as short as 2 bytes. They are local alias names, comparable to hostname.local, that help applications filter devices rather than establishing a connection with several devices just to find the intended one. The use of Service Data names has two upsides compared to filtering by MAC address:

Service Data identifiers can be stable across changes in hardware.
Service Data identifiers can be queried even on platforms on which MAC addresses are not accessible, such as on Web Bluetooth.

Two more uses of them are being considered:

Some resource metadata might already be transported in advertisements. These would need to be compact (in the order of magnitude of 10 bytes or less), and could contain data otherwise only discovered by querying the .well-known/core resource, or (hashes of) AS and audience values for ACE to facilitate connection creation with a device known by its managed identity. [ This is largely superseded by Service Data identifiers: The level of per deployment customization for what would and would not be hashed is likely so large that there would not be any interoperability exceeding plain identifiers anyway. ]
Advertisements could contain broadcast CoAP messages. Given that these non-traditional responses can not have embedded requests (as defined in ) due to size contraints, a mechanism such as could be used to distribute some consensus request.