Collecting data¶
The test and staging packages of the SDK can send the input they capture to DEA, where it trains the models that score sessions. They do it through the collection transporter, a separate package published in the test channel only. This page describes what its stream delivery sends, when, and where a session ends; each platform's Collect Data page shows how to attach it: Web, Flutter, Android, iOS, React Native.
Warning: Collection sends the raw input of your app to DEA: pointer and touch positions, scrolls, which class of key was pressed, device motion. Collect only in an app DEA asked you to collect with, with your users' agreement, and never add the collection transporter to a build you release.
Two kinds of session¶
The stream delivery files what it sends under sessions of its own, which are not the session the SDK scores.
| SDK session | Streamed session | |
|---|---|---|
| Starts | At init |
At the first object the SDK captures after init and consent, or after the previous streamed session ended |
| Ends | At reset(), or when the SDK is disposed |
Where the event flow is interrupted — see Where a streamed session ends |
| Id | The SDK's session id, on every captured object | Minted by the transporter; the session_id of every request |
| Used for | The classification | One row of DEA's training data |
One SDK session can stream as several sessions. Your app makes no call for a streamed session to start or end, and the classification is not reset when one ends.
When data is sent¶
While a streamed session is open, the transporter sends what was captured since its last request as one piece, and the first piece of a session carries its header. When the session ends, the rest goes as a last piece, followed by an end request.
| Web | Flutter, Android, iOS | |
|---|---|---|
| A piece goes | Every 10 seconds, or as soon as 48 KB is captured | Every 30 seconds |
| A piece with nothing in it | Not sent | Not sent |
| The end at the background | The rest and the end in one keepalive request, sent at once |
The rest and the end through the queue below; on iOS inside a background task |
Requests go one at a time, in order:
- Retry: once, half a second later, after a
5xx, a429or no answer. - A piece that still fails is dropped. Its session ends there, and the session's later pieces are dropped with it; the end is still sent, so DEA keeps what reached it.
- A full queue (4 pieces, or 1 MB, waiting) drops a new piece, and its session ends.
- Timeouts: 20 seconds for a piece, 120 seconds for an end.
- A session that ends before its first piece, shorter than 3 seconds or without a tap, click, scroll or key press, is not sent at all.
Where a streamed session ends¶
A streamed session ends where the event flow is interrupted, never at the event that might cause one. What the user does, which screen or URL is shown, which game runs: none of it ends a session while the flow continues.
| The session ends when | Web | Flutter, Android, iOS |
|---|---|---|
| The app or tab goes to the background | Yes | Yes |
| The continuity channel is silent for more than a second | Yes | Yes |
The SDK session changes: reset(), or a disposed and restarted SDK |
Yes | Yes |
| The document unloads: a reload, a link to another document, the back-forward cache | Yes | — |
| The session reaches 5 minutes, or 8 MB of captured input | Yes | Yes |
The continuity channel is the input that arrives without anyone touching the device: the desktop browser's heartbeat, or device motion on a phone or tablet. A session with neither — a phone browser without motion permission, a simulator, an app that switched motion off — never ends by silence, only by the other rows. Where there is a channel, it falls silent, and the session ends, when:
- the app calls
pause(), or the user withdraws consent; - a browser dialog (
alert(),confirm(),prompt(),print()) stays open for more than a second, or the page's main thread is blocked as long; - a phone's browser covers the page with its own interface and stops motion events.
These do not end a streamed session:
- A game or round that starts or ends; an ad that shows or closes. Mark them with custom events — the marks travel with the stream. An ad that takes the user to another app or tab ends it by the background row.
- A route change inside a web app (
history.pushState,replaceState, a hash change). - A pause shorter than a second.
- Rotation without the screen being recreated, a resize, a zoom, the keyboard.
- Idle time with the app in front: the heartbeat or the motion keeps arriving, and only the
5-minute limit ends a long one. On Flutter, Android and iOS this needs
idleMotionMsset to0, so that motion is not thinned while nobody touches the screen.
After a trip to the background, nothing is sent until the app or tab comes back, and the return opens a new streamed session.
What is sent¶
Each request is a gzipped JSON body, { "data": …, "event_type": … }, posted to
https://poh-api.dep-protocol.com/api/collect-data/data-training-streaming:
event_type |
data |
|---|---|
session_stream_ingest |
The session's header and one piece of its captured input |
session_stream_end |
{ "session_id": … }, or on the Web at the background the last piece itself |
The header names the session (session_id), the label you attached the transporter with, the
SDK version, the device (operating system, model, screen density, motion rate and state), the
screen or viewport and its changes, and the time the session's clock started. Every piece carries
a header, and DEA keeps the one of the session's first piece.
The captured input is the same objects the SDK scores: pointer, touch, wheel and key events, device
motion, the desktop heartbeat, the app going to the background and back, and your custom events.
A key is sent as its class (letter, digit, space…), never as the character; on a desktop page a
key pressed in a password field is not captured at all. DEA computes the training rows from this
input on its side, so samples and actions are empty in every request.
Known limitations¶
- A crashed or killed app sends no end. DEA closes the session after a while.
- A piece lost to the network or to a full queue loses the rest of its session.
- Gaps between 0.1 and 1 second do not end a session.
- On Flutter, Android and iOS, an app without the collection host or key attaches the transporter anyway and reports every piece as sent, while nothing leaves the device.
Related¶
- Accounts and keys — which package collects
- Privacy and network
- Glossary