SubMan async event streams (SSE) 1.0-166+8578759 documentation

SubMan async event streams (SSE) 1.0-166+8578759

Every listing endpoint in ldap-subscriber-api.yaml accepts ?subscribe=1, which turns the response into a text/event-stream that stays open and reports every change to the listed records. A single-object endpoint (/realm/{realmId} for example) returns the object and ignores subscribe. This document describes the one stream that every listing endpoint provides, and the events that the stream delivers.

Quickstart for an SSE consumer

const es = new EventSource('/realm/example.org/subscriber?subscribe=1&access_token=' + token);

es.addEventListener('ready', () => {
  // The server has finished replaying the initial state. Every later
  // `entry` is a live change.
});

es.addEventListener('entry', (e) => {
  const { entryUUID, syncOp, body } = JSON.parse(e.data);
  switch (syncOp) {
    case 'add':                                 // new record, full body
    case 'modify':  applyUpsert(body); break;   // changed record, full body
    case 'delete':  applyDelete(body); break;   // removed record, only id and _url
    case 'present':                             // unchanged record, only id and _url
      if (!haveRecord(body._url)) fetchRecord(body._url);
      break;
  }
});

The stream uses two SSE event names:

The server sends ready once, when the initial replay of existing records is complete. Every later entry is a live change. The payload is an empty object, and the id: line carries the resume cookie for the end of the replay.
The server sends entry once per record. syncOp names the kind of change, and body carries the record. Every record includes _url, the path of the REST resource of the record.

Choosing what to follow

A stream is a listing GET that stays open, so the stream accepts the same query parameters as the listing GET:

The id parameter and the filter_* parameters of the listing endpoint narrow the stream to the records that match the parameters. ldap-subscriber-api.yaml documents the parameters per endpoint. /realm/example.org/subscriber?subscribe=1&id=alice follows one subscriber.
scope=subtree widens the stream to every record below the collection, at any depth. Each event describes the record's own resource, so a subtree stream on /realm carries realm events and subscriber events. The consumer tells realm events and subscriber events apart by _url.

The server runs one directory search per open stream, and the directory server checks every write against every open search. The number of open searches is the cost of streaming. The number of connections is not a cost. The API serves HTTP/2, so a browser holds every stream on one connection. Open the fewest streams that cover the records you need. The widest streams are the subtree streams of the six top-level collections:

Stream	Records carried
`/realm?subscribe=1&scope=subtree`	realms and subscribers
`/group?subscribe=1&scope=subtree`	subscriber groups
`/service?subscribe=1&scope=subtree`	services, service profiles, and policies
`/client?subscribe=1&scope=subtree`	clients and the ports and taggings of the clients
`/clientGroup?subscribe=1&scope=subtree`	client groups
`/accessDevice?subscribe=1&scope=subtree`	access devices and the ports and taggings of the access devices

The event body

body has the shape of the GET response for the resource that _url names, plus _url. _url therefore selects which schema in ldap-subscriber-api.yaml describes body. Decoding an event does not depend on which stream delivered the event.

Deletes

delete events carry only id and _url in body, not the attributes that the record had before the record was removed. By the time the server sends the event, the record is gone from the directory and the attributes are unrecoverable. A consumer that needs the old state keeps a copy keyed on entryUUID.

Renames

The URL of a resource contains the id of the resource. When an id changes, the change is a rename, and the server sends two events for one rename rather than one modify:

a delete carrying the old id and _url.
an add carrying the new id, _url, and the full body.

Both events share the same entryUUID, so a consumer that treats a rename as one operation correlates the two events by entryUUID. A consumer that only tracks the current URL of a record treats the two events as a removed record followed by a new record.

A move between collections, for example a subscriber re-homed to another realm, produces the same two events. The URL changes, so the server sends delete for the old URL and add for the new URL.

Resuming a dropped stream

The ready event and every entry event carry an SSE id: line holding an opaque resume cookie. EventSource records the most recent cookie and sends the cookie back as Last-Event-ID when EventSource reconnects, so a browser client needs no extra code. A client that does not use EventSource resumes in two steps:

Remember the last id: value received.
On reconnect, send the remembered value in Last-Event-ID on the new request.

The server accepts a cookie of 1024 bytes or fewer with no control characters, and answers any other cookie with 400 Bad Request.

On a reconnect, the server replays only the changes since the cookie, as long as the directory server still holds history back to the cookie. When the cookie is older than the history that the directory server keeps, the server instead sends one present event for every record that still exists and has not changed. Each present event carries id and _url only. The server then sends the changes. A present event means that the record still exists and has not changed since the cookie. Keep the copy you have, and fetch _url if you have none.

Authentication

The stream needs the same bearer token as every other request. Send the token in Authorization: Bearer <token>. A client that cannot set request headers, such as the browser EventSource API, sends the token in the access_token query parameter of a GET. The token then appears in the URL, so the response to a GET with access_token carries Cache-Control: private.

Where the events come from

A syncrepl persistent search (RFC 4533) on the LDAP directory drives the stream. The syncOp values (present, add, modify, delete) and the entryUUID field come from syncrepl. Using the stream requires no knowledge of LDAP. The events describe REST resources, and body contains the JSON shape that the GET endpoint of the resource returns.

Servers

sse://localhost:3000/ssedefault
The host that serves the REST API. A stream is a GET of any listing endpoint of ldap-subscriber-api.yaml with ?subscribe=1 appended, plus the scope, id, and filter_* query parameters that the endpoint accepts.
Security:
HTTP
Scheme: bearer
Bearer format: JWT
Bearer token, sent as Authorization: Bearer <token> or, for a client that cannot set request headers, as the access_token query parameter of a GET.

Operations

RECEIVE {listPath}?subscribe=1

Changes to the records that the listing endpoint at listPath returns. Every listing endpoint of ldap-subscriber-api.yaml provides the stream. The scope, id, and filter_* query parameters select the records. The receive operation documents the parameters.

Open the change stream of a listing endpoint

A GET of the listing endpoint with subscribe=1. The response is a text/event-stream that delivers a ready event once and then an entry event per change. Last-Event-ID resumes a dropped stream. See 'Resuming a dropped stream' in the document description.

Operation IDreceive

Available only on servers:

default

object

Accepts one of the following messages:

#0Initial replay complete
Marks the boundary between the initial replay and the live changes.
- application/json
Message IDready
object
Examples
#1 Example - readyExample
The server sends `ready` once, after the initial replay. The SSE `id:` line carries the resume cookie.
{}

#1Record event

One change to one record. `_url` names the resource, and `body` has the shape of the GET response of the resource.

application/json

Message IDentry

allOf

Examples

#1 Example - realmAdd

A new realm. `body` has the shape of the response to `GET /realm/{realmId}`, plus `_url`.

{
  "entryUUID": "2c8e1f3a-7b46-4d6f-9a4d-92ad1e1f4ab2",
  "syncOp": "add",
  "body": {
    "id": "example.org",
    "description": "Subscribers homed in example.org",
    "serviceProfile": "http://localhost:3000/service/fixedLine/profile/INETG1M",
    "_url": "/realm/example.org"
  }
}

#2 Example - realmModify

A changed realm. `body` is the full new state, not a patch.

{
  "entryUUID": "2c8e1f3a-7b46-4d6f-9a4d-92ad1e1f4ab2",
  "syncOp": "modify",
  "body": {
    "id": "example.org",
    "description": "Subscribers homed in example.org",
    "serviceProfile": "http://localhost:3000/service/fixedLine/profile/INETG10M",
    "_url": "/realm/example.org"
  }
}

#3 Example - realmDelete

A removed realm. `body` is a tombstone with only `id` and `_url`.

{
  "entryUUID": "2c8e1f3a-7b46-4d6f-9a4d-92ad1e1f4ab2",
  "syncOp": "delete",
  "body": {
    "id": "example.org",
    "_url": "/realm/example.org"
  }
}

#4 Example - realmPresent
A realm that still exists and has not changed since the `Last-Event-ID` cookie. The server sends `present` only on a reconnect whose cookie is older than the history that the directory server keeps. `body` has only `id` and `_url`.
```
{
  "entryUUID": "2c8e1f3a-7b46-4d6f-9a4d-92ad1e1f4ab2",
  "syncOp": "present",
  "body": {
    "id": "example.org",
    "_url": "/realm/example.org"
  }
}
```

#5 Example - subscriberAdd

A new subscriber, from `/realm/{realmId}/subscriber`.

{
  "entryUUID": "4f0d2b91-3c5e-4a7b-8d21-6e9f0a1b2c3d",
  "syncOp": "add",
  "body": {
    "id": "foo+bar",
    "subscriberId": 123456789,
    "serviceStatus": "enabled",
    "serviceId": "fixedLine",
    "serviceProfile": "http://localhost:3000/service/fixedLine/profile/INETG1M",
    "_url": "/realm/example.org/subscriber/foo+bar"
  }
}

#6 Example - subscriberGroupAdd

A new subscriber group, from `/group`.

{
  "entryUUID": "7a1c3e5f-9b2d-4f6e-8a0c-1d3e5f7a9b2c",
  "syncOp": "add",
  "body": {
    "id": "business-subs",
    "description": "Business subscribers",
    "serviceProfile": "http://localhost:3000/service/fixedLine/profile/BUSINESS100M",
    "_url": "/group/business-subs"
  }
}

#7 Example - serviceAdd

A new service, from `/service`.

{
  "entryUUID": "8b2d4f60-ac3e-4071-9b1d-2e4f60ac3e40",
  "syncOp": "add",
  "body": {
    "id": "fixedLine",
    "_url": "/service/fixedLine"
  }
}

#8 Example - serviceProfileAdd

A new service profile, from `/service/{serviceId}/profile`.

{
  "entryUUID": "9c3e5071-bd4f-4182-ac2e-3f5071bd4f51",
  "syncOp": "add",
  "body": {
    "id": "INETG1M",
    "_url": "/service/fixedLine/profile/INETG1M"
  }
}

#9 Example - policyAdd

A new policy, from `/service/{serviceId}/profile/{profileId}/policy`.

{
  "entryUUID": "ad4f6182-ce50-4293-bd3f-406182ce5062",
  "syncOp": "add",
  "body": {
    "id": "nokia7750",
    "radiusAttribute": "reply.Vendor-Specific.Nokia-SR.Subsc-ID-Str = \"%{User-Name}@example.org\"",
    "_url": "/service/fixedLine/profile/INETG1M/policy/nokia7750"
  }
}

#10 Example - clientAdd

A new client, from `/client`.

{
  "entryUUID": "be507293-df61-43a4-ce40-517293df6173",
  "syncOp": "add",
  "body": {
    "id": "192.0.2.1",
    "freeradiusClientShortname": "edge-bng-01",
    "freeradiusClientGroup": "BNGs",
    "_url": "/client/192.0.2.1"
  }
}

#11 Example - clientPortAdd

A new physical port, from `/client/{clientId}/port`.

{
  "entryUUID": "cf6183a4-e072-44b5-df51-6283a4e07284",
  "syncOp": "add",
  "body": {
    "id": "1%2F2%2F3",
    "type": "physical",
    "port": "1/2/3",
    "remoteAssociation": "http://localhost:3000/accessDevice/dslam-03",
    "_url": "/client/192.0.2.1/port/1%2F2%2F3"
  }
}

#12 Example - clientPortQinqAdd

A new QinQ tagging, from `/client/{clientId}/port/{portId}/qinq`.

{
  "entryUUID": "d07294b5-f183-45c6-e062-7394b5f18395",
  "syncOp": "add",
  "body": {
    "id": "100+25",
    "type": "qinq",
    "svlan": 100,
    "cvlan": 25,
    "remoteAssociation": "http://localhost:3000/accessDevice/dslam-03",
    "_url": "/client/192.0.2.1/port/1%2F2%2F3/qinq/100+25"
  }
}

#13 Example - clientPortStagAdd

A new S-tag tagging, from `/client/{clientId}/port/{portId}/stag`.

{
  "entryUUID": "e183a5c6-0294-46d7-f173-84a5c60294a6",
  "syncOp": "add",
  "body": {
    "id": "100",
    "type": "physStag",
    "port": "1/2/3",
    "svlan": 100,
    "remoteAssociation": "http://localhost:3000/accessDevice/dslam-03",
    "_url": "/client/192.0.2.1/port/1%2F2%2F3/stag/100"
  }
}

#14 Example - clientGroupAdd

A new client group, from `/clientGroup`.

{
  "entryUUID": "f294b6d7-13a5-47e8-0284-95b6d713a5b7",
  "syncOp": "add",
  "body": {
    "id": "BNGs",
    "description": "Broadband network gateways",
    "_url": "/clientGroup/BNGs"
  }
}

#15 Example - accessDeviceAdd

A new access device, from `/accessDevice`.

{
  "entryUUID": "03a5c7e8-24b6-48f9-1395-a6c7e824b6c8",
  "syncOp": "add",
  "body": {
    "id": "dslam-03",
    "freeradiusClientShortname": "DSLAM03-D6",
    "freeradiusClientType": "dslam",
    "_url": "/accessDevice/dslam-03"
  }
}

#16 Example - accessDevicePortAdd

A new physical port, from `/accessDevice/{accessDeviceId}/port`.

{
  "entryUUID": "14b6d8f9-35c7-490a-24a6-b7d8f935c7d9",
  "syncOp": "add",
  "body": {
    "id": "1%2F2%2F3",
    "type": "physical",
    "port": "1/2/3",
    "remoteAssociation": "http://localhost:3000/client/192.0.2.1",
    "_url": "/accessDevice/dslam-03/port/1%2F2%2F3"
  }
}

#17 Example - accessDevicePortQinqAdd

A new QinQ tagging, from `/accessDevice/{accessDeviceId}/port/{portId}/qinq`.

{
  "entryUUID": "25c7e90a-46d8-4a1b-35b7-c8e90a46d8ea",
  "syncOp": "add",
  "body": {
    "id": "100+25",
    "type": "qinq",
    "svlan": 100,
    "cvlan": 25,
    "remoteAssociation": "http://localhost:3000/client/192.0.2.1",
    "_url": "/accessDevice/dslam-03/port/1%2F2%2F3/qinq/100+25"
  }
}

#18 Example - accessDevicePortStagAdd

A new S-tag tagging, from `/accessDevice/{accessDeviceId}/port/{portId}/stag`.

{
  "entryUUID": "36d8fa1b-57e9-4b2c-46c8-d9fa1b57e9fb",
  "syncOp": "add",
  "body": {
    "id": "100",
    "type": "physStag",
    "port": "1/2/3",
    "svlan": 100,
    "remoteAssociation": "http://localhost:3000/client/192.0.2.1",
    "_url": "/accessDevice/dslam-03/port/1%2F2%2F3/stag/100"
  }
}

Messages

#1Initial replay complete
Marks the boundary between the initial replay and the live changes.
- application/json
Message IDready
object
#2Record event
One change to one record. `_url` names the resource, and `body` has the shape of the GET response of the resource.
- application/json
Message IDentry
allOf

Schemas

object
object
Body of a delete or present event. The tombstone holds only enough to identify the record.
object
allOf
allOf
allOf
allOf
allOf
allOf
allOf
allOf
allOf
allOf
allOf
allOf

SubMan async event streams (SSE) 1.0-166+8578759

Quickstart for an SSE consumer

Choosing what to follow

The event body

Deletes

Renames

Resuming a dropped stream

Authentication

Where the events come from

Servers

Security:

Operations

RECEIVE {listPath}?subscribe=1

Examples

#1 Example - readyExample

Examples

#1 Example - realmAdd

#2 Example - realmModify

#3 Example - realmDelete

#4 Example - realmPresent

#5 Example - subscriberAdd

#6 Example - subscriberGroupAdd

#7 Example - serviceAdd

#8 Example - serviceProfileAdd

#9 Example - policyAdd

#10 Example - clientAdd

#11 Example - clientPortAdd

#12 Example - clientPortQinqAdd

#13 Example - clientPortStagAdd

#14 Example - clientGroupAdd

#15 Example - accessDeviceAdd

#16 Example - accessDevicePortAdd

#17 Example - accessDevicePortQinqAdd

#18 Example - accessDevicePortStagAdd

Messages

Schemas