Skip to main content

ADR-002: IPLD Objects With JOSE

hackmd-github-sync-badge

Summary

Implement Object Schemas based on IPLD which support JOSE operations.

Abstract

Objects on the Sonr network provide a way for users and applications to persist data. The problem with modern software architectures is that the security of user data is not guaranteed—a problem Sonr aims to solve with the help of JavaScript Object Signing and Encryption (JOSE).

Additionally, today’s software stack is not built with interoperability in mind. An entire layer must be added to any application to support communication with other applications. IPLD provides a standard and decentralized way for any motor node to make use of the data stored on Sonr.

The combining of these technologies begets a system in which user data is always encrypted and can never be owned by one application, not even the application that created it.

Primer

The content of this ADR is based on the works of ADR-001 as well as a few core technologies—all of which should be well understood before reading ADR-002. Here is a list of resources.

Glossary

JSON Web Signature (JWS)

JavaScript Object Notation (JSON) data structure that represents a cryptographic key. This specification also defines a JWK Set JSON data structure that represents a set of JWKs. Cryptographic algorithms and identifiers for use with this specification are described in the separate JSON Web Algorithms (JWA) specification and IANA registries established by that specification.

JSON Web Encryption (JWE)

JSON Web Encryption (JWE) represents encrypted content using JSON-based data structures. Cryptographic algorithms and identifiers for use with this specification are described in the separate JSON Web Algorithms (JWA) specification and IANA registries defined by that specification. Related digital signature and Message Authentication Code (MAC) capabilities are described in the separate JSON Web Signature (JWS) specification.

JSON Web Key (JWK)

A JSON Web Key (JWK) is a JavaScript Object Notation (JSON) data structure that represents a cryptographic key. This specification also defines a JWK Set JSON data structure that represents a set of JWKs. Cryptographic algorithms and identifiers for use with this specification are described in the separate JSON Web Algorithms (JWA) specification and IANA registries established by that specification.

Schema

A definition that outlines a structure for data to be stored within a persistence layer. These structures conform to a standard of key value pairs where the key is a string and the value is one of a set number of Types.

Identity

An identity is derived from the user's provided key upon registration.

Objective

The objective of ADR-002 is to define how Sonr manages the creation and access of data on its network. A proper solution must have these attributes.

  • Enable motor nodes to store data on behalf of a user.
  • Enable devices which belong to the same user to access the same data.
  • Enable certain data to be shared with other users.
  • Allow for revoking access to both devices and other users.
  • Maintain that all data is encrypted in transit and at rest.

Changes From ADR-001

To accommodate the features proposed by ADR-002, a few changes to terminology will be made from ADR-001.

Rename Object to Schema

Object in ADR-001 represents a type definition in Sonr. This name is misleading as it creates ambiguity between the type definition and the data which is of that type. With the addition of IPLD, the term Schema will better express the concepts that were represented by Object while creating consistency with the libraries we employ.

Client-Side Encryption with JOSE

A core principle of Sonr is to never let private user data pass unencrypted through the network. To achieve this, the motor node must be responsible for encrypting and decrypting all data. This section describes how a user may securely store data on Sonr and maintain the ability to access that data from any motor associated with his/her account.

Encryption

In order for encrypted data to be accessed from multiple devices, there must be a shared key for each piece of data that will be accessible to each motor.

Key Creation

The JWK used during the encryption process is determined by the access levels for the data. As the owner of data, the user bucket will contain a map of encrypted JWKs: map[CID]map[PK]JWK [henceforth referenced to as the Object Encryption Key Map (OEKM)]; where CID is the CID of the object, PK is the public key of a member who has access, and JWK is the JWK used to encrypt the data.

For objects shared with other users, the OEKM may exist in a dedicated bucket, but this is an implementation detail for the developer.

Client

  1. An object is created using a Schema—this is what will be stored.
  2. The motor encrypts the data using its shared key for private data. If the data is meant to be shared with a party, a key specific to that party is used.
    • Any property in the data that is represented as a separate Schema will be encrypted separately and included with the POST request.
    • Each encrypted schema will also have a corresponding CID. Because CIDs are generated predictably, this will be computed by the motor.
  3. The motor then creates a copy of the key it generated and encrypts it with each public key in the root-level DID. This is ultimately what allows another motor to decrypt the data.
  4. A request can now be crafted to Highway with the encrypted data and set of encrypted keys.
    • The request should include each encrypted schema along with the list of encrypted shared keys.

Writing encrypted data

Server

  1. When a request is received, each object is added to IPFS.
  2. The CID for each object will be verified, creating an error if one does not match.
  3. If successful, each CID is added to a record on-chain. Alternatively, each CID may be be part of a bucket update and not stored individually on-chain.

Decrypting

The process for decrypting data becomes simple with the above provisions. The motor will first obtain the shared key—accessible via the vault (more on that later)—which will be used to encrypt data fetched through IPFS. The entire process is as follows.

  1. Fetch the shared key and decrypt it.
    • The shared key will have already been added to the vault by the uploading device. This process is outlined in the next section.
  2. Through IPFS, fetch the relevant data via CID. Then, fetch each referenced CID in the result. Continue this process until all objects have been recovered.
  3. Decrypt each object using the shared key.
  4. Compose the original schema by replacing the IPLD Link with the referenced object.

Revoking Access

If any device or party is to have its access revoked, a new key must be used for the next update to that object. This means the process for revoking access is to simply upload the object again, with the revoked party omitted from the OEKM.

A Note on the Vault

Each user on Sonr will have a bucket created for them when they create an account, known as their vault. The vault acts as a collection of all the user's personal keys or private data. Buckets and vaults will be covered in great detail in ADR-003.

Adding Verification Methods

The process of securing user data starts with the verification method. Simply put, verification methods are a member of the DID Document which contain the public keys used for authentication. New verification methods can be added in order for a user to authenticate with new or different keys. Later sections will detail how these methods are used to create JWKs for secure data storage.

The important thing to note about Verification Methods in ADR-002 is that the process for adding a Verification Method must now include encrypting all shared keys again with the new method’s public key.

Schema Model

A schema added to Sonr will have the following format.

type Schema struct {
Did string
Label string
Fields map[string]SchemaKind
}

type SchemaKind int32

const (
SchemaKind_Invalid SchemaKind = iota
SchemaKind_Map
SchemaKind_List
SchemaKind_Unit
SchemaKind_Bool
SchemaKind_Int
SchemaKind_Float
SchemaKind_String
SchemaKind_Bytes
SchemaKind_Link
SchemaKind_Struct
SchemaKind_Union
SchemaKind_Enum
SchemaKind_Any
)

Did

The DID that uniquely identifies the Schema.

Label

A name for this schema.

Fields

A list of properties for this schema. The property types are defined as SchemaKind.

Object Model

Object data on Sonr will be encrypted, but IPFS will contain a wrapper of that data containing some additional metadata.

type Object struct {
Content []byte
EncryptedJWKs map[PublicKey]EncryptedJWK
}

type PublicKey string
type EncryptedJWK string

Content

The raw content of this object. May be encrypted with a JWK.

EncryptedJWKs

A mapping of each public key that has access to the data with the JWK encrypted with said public key.

Object Schema API (Blockchain)

The following methods will be added to support creating Schemas and Objects.

CreateSchema(label, fields)

Parameters

label (string) : The label for the Schema. Must be alphanumeric and without whitespace.

fields (map[string]int32) : The properties to be included in the Schema. Each key must be alphanumeric and without whitespace. The value is an integer corresponding to a SchemaKind.

Description

Creates a schema for use throughout Sonr.

Creating a schema requires a DID and DID Document to be created.

The model itself will be stored in IPFS, while the DID-CID pair is stored on-chain.

GetSchema()

Description

Returns the schema. Requires no blockchain transactions.

UploadObject(schemaDid, data, encryptedJwks)

Parameters

schemaDid (string) : The DID of the schema this object adheres to.

data ([]byte) : The raw data to be added to IPFS. This can be plaintext or encrypted with the user's root JWK.

encryptedJwks (map[PK]JWK) : A map of encrypted JWKs for anyone who should have access to the data. Each JWK must be encrypted with the PK of the shared party.

Description

Uploads raw data and adds to it IPFS. A CID is returned for addressing the content.

There is no blockchain interactions with this call.

GetObject(cid)

Parameters

cid (string) The CID which addresses the content on IPFS.

Description

Returns the object previously uploaded to IPFS, including the map of JWKs. The map can be used to look up and decrypt the appropraite JWK in order to read the data.

This call requires no blockchain interaction.

WhoIs Updates

// Metadata attached associated with the document
// data is arbitrary, and thus user specified.
Metadata map[string]string

DID Document Verification Definition

// Connected Motors Webauthn credentials get stored as verificationMethod
{
"verificationMethod": [
{
// Set to Motor Nodes Wallet Address
"controller": "did:snr:123",

// Id of Key set to unique value and operating system/architecture
"id": "did:snr:123#ios-arm64-1",

// JWK generated from WebAuthN Credential
"publicKeyJwk": {
"crv": "P-256",
"kty": "EC",
"x": "UANQ8pgvJT33JbrnwMiu1L1JCGQFOEm1ThaNAJcFrWA=",
"y": "UWm6q5n1iXyeCJLMGDInN40bkkKr8KkoTWDqJBZQXRo="
},
"type": "JsonWebKey2020"
}
]
}

IPLD Codec

DAG-JOSE is a codec that implements the IPLD Data Model as a subset of JOSE, which complies with the DAG-CBOR specication. Some additional constraints for hash consistent representations. DAG-CBOR also adds a "link" type using a CBOR tag to bring it in line with the IPLD Data Model. go implementation

IPLD Pathing

Paths are composed of a series of segments, and each segment is an instruction on how to navigate deeper into the filesystem. With filesystems, each step is over a "directory" and leads you to either a "file" or another "directory"; for IPLD, each step is over a "node" and leads you to another "node"!

DID Methods for CBOR-JOSE support

The following methods will need to be implemented within sonr-io/sonr/pkg/did for DAG-JOSE support for Sonr DID Documents.

  • Authenticate
  • CreateJWS
  • DecryptJWE
  • EnctyptJWE
  • VerifyJWS

Models

CreateJWSOptions {
did: string
protected: map[string]interface{} (optional)
linkedBlock: string (optional)
}
DagJWS = {
payload: string
signatures: []JWSSignature
link: CID (optional)
}
VerifyJWSOptions {
atTime: Date (optional)

disableTimecheck: boolean (optional)

issuer: string (optional)

capability: Cacao (optional)

revocationPhaseOutSecs: number (optional)
}
EncryptJWE {
cleartext: []byte,
recipients: []string,
protectedHeader: map[string]interface{}
aad: []byte (optional)
}
DecryptJWEOptions {
did: string (optional)
}

Methods

Authenticate

Description

DID Authentication communicates with the remote origin of DID document to gain the JWS or siganture. Said JWS is then verified. the following operations are important for proper authentication compliance

  • Check the returned kid includes the did from the verification payload.
  • Check experation of the signature

Parameters

provider - DIDProvider. paths - didUrl(s) to authenticate aud - Audience (JWS claim)

Returns

A JWS with general serialization containing the following properties:

nonce : The random string which was given as a challenge

did : The DID which authentication was given for

paths : The paths which was given permission for

exp : Unix timestamp after which the JWS should be considered invalid.

aud : optional audience for the JWS, should match the domain which made the request.

CreateJWS

Generates a JSON web signature object composed of a multi signature key.

{
"link": {
"/": "bafybeig6xv5nwphfmvcnektpnojts33jqcuam7bmye2pb54adnrtccjlsu"
},
"payload": "AXASIN69ets85WVE0ipva5M5b2mAqAZ8LME08PeAG2MxCSuV",
"signatures": [
{
"protected": "eyJhbGciOiJFUzI1NksifQ",
"signature": "SiYGXW7Yi-KxbpIlLNmu0lEhrayV7ypaAC49GAcQ_qpTstZW89Mz6Cp8VlUEX-qVsgYjc-9-1zvLcDYlxOsr1g"
},
{
"protected": "eyJhbGciOiJFUzI1NksifQ",
"signature": "Q8PdTE5A5N3a0ktO2wNdUymumHlSxNF9Si38IvzsMaSZC63yQw-bJNpKf-UeJFPH7cDzY7jLg2G_viejp7NqXg"
}
]
}

Params

payload : The payload to sign, json object or base64url encoded string.

protected : The protected header, json object.

did : The DID that should sign the message, may include the key, fragment.

revocable : Makes the JWS revocable when rotating keys, boolean default to false

Returns: DAGJws

VerifyJWS

Description

We can compute the SHA-256 representation if the specified secret is present within the generated token. Upon querying for an ecrypted entry within ipfs, the DID controller of said object should be able to verify the signature as it should be aware of valid signatures relating to itself.

Params

jws : signature to verify

options : VeriftyJWSOptions

Returns

Object Schemas

IPLD Object Schema

IPLD represents data types which can be stored in IPFS nodes as “Kinds” the following are types which “Kinds” support:

  • boolean
  • integer
  • float
  • map
  • list
  • string
  • null
  • bytes
  • link

Kinds in IPLD are similar to data types supported within “JSON” but also add “bytes” and “link” where link maps to “CID” type.

DSL Representation of Object Schema

Node builders allow you to declare schema within an object representation that then can be transformed to various types. Builders then can be represented in DSL which is a declarative syntax for IPLD schemas similar to JSON. The following is a method stub for creating an object schema for later usage.

CreateObjectSchema

Creation of an object will define a schema for objects to be later stored in IPFS. Schemas define the shape of data for a given object.

PutObjectSchema

func (i *IPFSProtocol) PutObjectSchema(doc *ot.ObjectDoc) (*cid.Cid, error)

Implementation using IPLD Node Builder

Stores a DSL representation of a defined object schema with the following relationship denoting a property of the persistent schema.

{
Label: string
Type: Kind
}

This object will be stored within an aggregate (array structure) within the ObjectDoc and should be iterated over and stored within a node builder, then serialized to a byte representation.

GetObjectSchema

Function Signature

func (i *IPFSProtocol) GetObjectSchema(cid *CID) (Node, error)

Implementation (string repersentation of schema)

Instead of using a node builder to build our schema up from a tree of related objects, the schema structured can be loaded from a string repersentation of the ipld schema:

'
{
"foo": string,
"baz": {
"prop": integer
}
}
'

The above repersentation can be serialized to a Byte Array, and loaded as a static schema structure. Therefore, we can then store this structure as a schema for schemas as the following:

{
"schema": []byte,
}

schema : Schema decleration as an array of bytes.

Interaction Methods for Highway to Motor Communication

The following are interface methods that should be defined on Host and implemented for Host targets.

AccessApp()

User authenticates a Registered Application on Sonr with their DID Based Multisignature key for all their devices. Creates a new Bucket inside the User bucket for the newly provisioned Application.

InteractObject()

Maps user defined object to a object schema. Which is then pushed to a bucket.

ReadObject()

This method begins the request process for reading the individual object values of another User's app bucket. Provisioned users will automatically get access (if they called RequestBucket already) and unprovisioned users will send the RequestBucket to the corresponding peer.