Skip to content

Collecting data

The test and staging packages of the SDK can send the input they capture to DEA, where it trains the models that score sessions. They do it through the collection transporter, a separate package published in the test channel only. This page describes what its stream delivery sends, when, and where a session ends; each platform's Collect Data page shows how to attach it: Web, Flutter, Android, iOS, React Native.

Warning: Collection sends the raw input of your app to DEA: pointer and touch positions, scrolls, which class of key was pressed, device motion. Collect only in an app DEA asked you to collect with, with your users' agreement, and never add the collection transporter to a build you release.

Two kinds of session

The stream delivery files what it sends under sessions of its own, which are not the session the SDK scores.

SDK session Streamed session
Starts At init At the first object the SDK captures after init and consent, or after the previous streamed session ended
Ends At reset(), or when the SDK is disposed Where the event flow is interrupted — see Where a streamed session ends
Id The SDK's session id, on every captured object Minted by the transporter; the session_id of every request
Used for The classification One row of DEA's training data

One SDK session can stream as several sessions. Your app makes no call for a streamed session to start or end, and the classification is not reset when one ends.

When data is sent

While a streamed session is open, the transporter sends what was captured since its last request as one piece, and the first piece of a session carries its header. When the session ends, the rest goes as a last piece, followed by an end request.

Web Flutter, Android, iOS
A piece goes Every 10 seconds, or as soon as 48 KB is captured Every 30 seconds
A piece with nothing in it Not sent Not sent
The end at the background The rest and the end in one keepalive request, sent at once The rest and the end through the queue below; on iOS inside a background task

Requests go one at a time, in order:

  • Retry: once, half a second later, after a 5xx, a 429 or no answer.
  • A piece that still fails is dropped. Its session ends there, and the session's later pieces are dropped with it; the end is still sent, so DEA keeps what reached it.
  • A full queue (4 pieces, or 1 MB, waiting) drops a new piece, and its session ends.
  • Timeouts: 20 seconds for a piece, 120 seconds for an end.
  • A session that ends before its first piece, shorter than 3 seconds or without a tap, click, scroll or key press, is not sent at all.

Where a streamed session ends

A streamed session ends where the event flow is interrupted, never at the event that might cause one. What the user does, which screen or URL is shown, which game runs: none of it ends a session while the flow continues.

The session ends when Web Flutter, Android, iOS
The app or tab goes to the background Yes Yes
The continuity channel is silent for more than a second Yes Yes
The SDK session changes: reset(), or a disposed and restarted SDK Yes Yes
The document unloads: a reload, a link to another document, the back-forward cache Yes —
The session reaches 5 minutes, or 8 MB of captured input Yes Yes

The continuity channel is the input that arrives without anyone touching the device: the desktop browser's heartbeat, or device motion on a phone or tablet. A session with neither — a phone browser without motion permission, a simulator, an app that switched motion off — never ends by silence, only by the other rows. Where there is a channel, it falls silent, and the session ends, when:

  • the app calls pause(), or the user withdraws consent;
  • a browser dialog (alert(), confirm(), prompt(), print()) stays open for more than a second, or the page's main thread is blocked as long;
  • a phone's browser covers the page with its own interface and stops motion events.

These do not end a streamed session:

  • A game or round that starts or ends; an ad that shows or closes. Mark them with custom events — the marks travel with the stream. An ad that takes the user to another app or tab ends it by the background row.
  • A route change inside a web app (history.pushState, replaceState, a hash change).
  • A pause shorter than a second.
  • Rotation without the screen being recreated, a resize, a zoom, the keyboard.
  • Idle time with the app in front: the heartbeat or the motion keeps arriving, and only the 5-minute limit ends a long one. On Flutter, Android and iOS this needs idleMotionMs set to 0, so that motion is not thinned while nobody touches the screen.

After a trip to the background, nothing is sent until the app or tab comes back, and the return opens a new streamed session.

What is sent

Each request is a gzipped JSON body, { "data": …, "event_type": … }, posted to https://poh-api.dep-protocol.com/api/collect-data/data-training-streaming:

event_type data
session_stream_ingest The session's header and one piece of its captured input
session_stream_end { "session_id": … }, or on the Web at the background the last piece itself

The header names the session (session_id), the label you attached the transporter with, the SDK version, the device (operating system, model, screen density, motion rate and state), the screen or viewport and its changes, and the time the session's clock started. Every piece carries a header, and DEA keeps the one of the session's first piece.

The captured input is the same objects the SDK scores: pointer, touch, wheel and key events, device motion, the desktop heartbeat, the app going to the background and back, and your custom events. A key is sent as its class (letter, digit, space…), never as the character; on a desktop page a key pressed in a password field is not captured at all. DEA computes the training rows from this input on its side, so samples and actions are empty in every request.

Known limitations

  • A crashed or killed app sends no end. DEA closes the session after a while.
  • A piece lost to the network or to a full queue loses the rest of its session.
  • Gaps between 0.1 and 1 second do not end a session.
  • On Flutter, Android and iOS, an app without the collection host or key attaches the transporter anyway and reports every piece as sent, while nothing leaves the device.