Developer Experience with CBOR on the wire

[c2bo]

#5 is discussing the general serialization format, but also mixes in the dev ux related discussion points.

My take-away from the discussions at IETF 126 was that people were pointing towards CDDL and defining CBOR and JSON in one CDDL. Make sure it is clear that we use CBOR over the wire, but have a clear definition of JSON representations for configuration, debugging, etc.
Copied over from the other issue:

If we use integer based maps in CBOR, we could in theory define in CDDL values that are different between CBOR and JSON representation which would allow us to define keys in the map that are different from CBOR (integer) and JSON (string). Taken from cbor-wg Wiki:

JC<"v", 2>
binary-value = JC<base64-string, bstr>

JC<"v",2> defines the value as "v" for JSON and 2 for CBOR.

RFC9741 introduces control operators that allows to express things like base64 encoding in CDDL:

signature-for-json = text .b64u signature
signature = bytes .cbor COSE_Sign1

[awoie]

WG discussion:

• Notation of CBOR encoded structures #34 and Developer Experience with CBOR on the wire #19 both needed, and should just be tried out.
• @MasterKale volunteered to help with this and review this approach

[awoie]

@c2bo @MasterKale I am wondering if you could provide examples for the CBOR / CDDL used in #29 and check what works best?

[andrewhughes3000]

Please post the examples in this Issue for discussion/review by WG.

[c2bo]

@c2bo @MasterKale I am wondering if you could provide examples for the CBOR / CDDL used in #29 and check what works best?

That was my proposal in last call -> that is definitely the plan. I am on vacation this week, so i am not sure if i will make progress this week from my side.

[MasterKale]

Here's an initial exploration at combining RFC 9165's .feature to define a helper notation to colocate type annotations between CBOR and JSON encodings, with RFC 9741 for some JSON-friendler datatypes.

BTW, since the various structures in #29 are defined for CBOR as the initial/"default" encoding scheme, I think the "JC<>" notation for expressing cross-encoding datatypes should probably be "CJ<>" instead. Accordingly, CJ<..., ...> is used below to indicate "this datatype in a CBOR domain, this other datatype in a JSON domain":

Examples

Using CDDL's .feature to define a "CBOR type, JSON type"...type

CJ<C,J> = C .feature "cbor" / J .feature "json"

Values using CDDL datatype primitives get the CJ<> treatment

label = CJ<int, int> / CJ<tstr,tstr>
values = CJ<any, any>
payload = CJ<bstr, b64u>
empty_or_serialized_map = CJ<bstr .cbor header_map, TODO> / CJ<bstr .size 0, TODO>

Values referring to "complex" CDDL datatypes don't get the CJ<> treatment

reader_auth = COSE_Sign

COSE_Sign = [
    Headers,
    payload : CJ<bstr, b64u> / CJ<nil, nil>,
    signatures : [+ COSE_Signature]
]

COSE_Signature =  [
    Headers,
    signature : CJ<bstr, b64u>
]

Headers = (
    protected : empty_or_serialized_map,
    unprotected : header_map
)

header_map = {
    Generic_Headers,
    * label => values
}

Generic_Headers = (
    ? 1 => CJ<int, int> / CJ<tstr, tstr>  ; algorithm identifier
    ? 2 => [+label],                      ; criticality
    ? 3 => CJ<tstr, tstr> / CJ<int, int>, ; content type
    ? 4 => CJ<bstr, b64u>,                ; key identifier
    ? ( 5 => CJ<bstr, b64u> //            ; IV
        6 => CJ<bstr, b64u> )             ; Partial IV
)

Experimenting with adding CJ<> notation to dchp/pull/29

Pulling out some sections from #29 to update with this syntax

Request Structure

The top-level CredentialRequest contains:

Field	Key	Type	Presence	Description
payload	1	CJ<bytes, b64u>	M	CBOR-encoded RequestPayload
reader_auth	2	COSE_Sign	O	Optional reader authentication signature

The RequestPayload contains:

Field	Key	Type	Presence	Description
ScenarioSets	1	{ + ScenarioRef => ScenarioSet }	M	Map of scenarios the reader is requesting, keyed by ScenarioRef
CredentialQueries	2	{ + CredentialRef => CredentialQuery }	M	Dictionary of all requested credential definitions
encryption_context	3	EncryptionContext	C	Primary response encryption context; absent when ISO/IEC 18013-5 session encryption is used
additional_encryption_contexts	4	{ + CJ<int, int> => EncryptionContext }	O	Additional encryption contexts for multi-key routing, keyed by integer

Credential Queries

A CredentialQuery defines the exact credential requirements for a single credential type.

Field	Key	Type	Presence	Description
credential_format	1	CJ<tstr, tstr>	M	Format identifier (e.g., "mso_mdoc", "vc+sd-jwt")
credential_type	2	CJ<tstr, tstr>	M	Credential type (e.g., "org.iso.18013.5.1.mDL")
elements_dict	3	{ + ElementRef => DataElementDef }	M	Dictionary of requested data elements
requested_elements	4	ElementLogic	M	Boolean logic tree defining which elements are required
general_extensions	5	generalExtensions	O	Protocol-level extensions applicable across formats
format_extensions	6	formatExtensions	O	Format-specific extensions (e.g., mdocExtensions, sdjwtExtensions)
encryption_ref	7	CJ<int, int>	O	Reference to an entry in additional_encryption_contexts; absent means the main encryption_context is used

My Opinions

• Any use of CDDL primitive datatypes (e.g. bstr, tstr, int) should get the CJ<> treatment, no matter how obvious it might seem a datatype maps to a JavaScript type
• Custom CDDL types (e.g. COSE_Signature, Headers, Generic_Headers) can probably go without CJ<> annotations
• CDDL Arrays and Maps have matching types in JavaScript so there's seemingly no need to give them the CJ<> treatment either

I think after going through this exercise I've come to the opinion that the CJ<..., ...> notation style seems reasonable to me. I think I'd personally prefer to read this style over having to Cmd+F and jump between CBOR-specific and JSON-specific structs throughout an already complex document.

[MasterKale]

And I apologize, I'm pretty sure I'm using "CBOR" and "CDDL" a bit too interchangeably above. I don't often talk about data structures using CDDL terminology so bear with me while I learn from the WG through osmosis 😅

[MasterKale]

Make sure it is clear that we use CBOR over the wire, but have a clear definition of JSON representations for configuration, debugging, etc.

I was mulling over my exploration above and agree it'd be necessary, around whatever such notation we ultimately land on, to emphasize that the JSON datatype definitions for any given data structure is only for representing an instance of that data structure in isolation. That is, there'd be no actual JSON-shaped values within the deeper parts of the protocol which expects everything to use CBOR encoding 🤔

[awoie]

WG discussion:

• heavy annotated CDN might be an alternative; CDN is superset of JSON, e.g., can do comments.
• for examples, no JSON notation; JSON notation is for debug/ output mostly.
• in a lot of places types are the same in JSON and CBOR; only in a few places they differ, e.g., bytestr vs b64u. we could agree on this.
• CDN and JC serve different purposes.
• CDDL is well supported, e.g., Rust, which can also do JSON/CBOR conversion.
• Need some experimenting with libraries.
• Next step could be to apply this to the examples of the initial document PR.
• Not a blocker for the remaining specification.