Collect Data¶
The test and staging packages of the SDK can send what they capture to DEA's collection service, where it trains the models that score sessions. This page shows how a test or staging app attaches the collection transporter's stream delivery: the app's input goes to DEA in pieces while it runs, and the transporter cuts it into sessions on its own, so your app makes no call per screen or per round.
Warning: Collection sends the raw input of your app — touch positions, scrolls, device motion — to DEA. Use it only in an app DEA asked you to collect with, with your users' agreement. On iOS the stream delivery attaches on every package, the production one included: never add the collection transporter to a build you release.
When to use it¶
- A data-collection or QA app that DEA asked you to build
- A staging build of your app whose testers' sessions should reach DEA's training data
Prerequisites¶
- An
appIdandappKeyfor the app — see Accounts and keys - The collection service's API key, which your DEA contact gives you. It is not the
appKey - iOS 13 or later
- Your users agreed to collection — see User Consent
Usage¶
Install the packages¶
The SDK and the collection transporter are Swift packages, published as archives. A staging app takes the SDK from the staging channel and the transporter from the test channel, the only one that publishes it, at the same release. From the folder that keeps your vendored packages:
mkdir -p Vendor/DEA && cd Vendor/DEA
for slot in base automation-detection sdk; do
curl -sS "https://sdk.dep-protocol.com/0.26.0/staging/ios/$slot/dea-$slot-0.26.0.tar.gz" | tar -xz
done
curl -sS "https://sdk.dep-protocol.com/0.26.0/test/ios/ai-backend-transporter/dea-ai-backend-transporter-0.26.0.tar.gz" | tar -xz
This creates four package folders side by side: dea-base-0.26.0,
dea-automation-detection-0.26.0, dea-sdk-0.26.0 and
dea-ai-backend-transporter-0.26.0. Each one names the others by those relative paths,
so keep them together. A test app takes the first three from /test/ios/ instead of
/staging/ios/.
In Xcode, add dea-sdk-0.26.0, dea-base-0.26.0 and
dea-ai-backend-transporter-0.26.0 as local packages (File > Add Package
Dependencies… > Add Local…), and link the DeaSDK, DeaBase and DeaAiBackendTransporter
products to your app target. In a Package.swift:
dependencies: [
.package(path: "Vendor/DEA/dea-sdk-0.26.0"),
.package(path: "Vendor/DEA/dea-base-0.26.0"),
.package(path: "Vendor/DEA/dea-ai-backend-transporter-0.26.0"),
],
targets: [
.target(name: "App", dependencies: [
.product(name: "DeaSDK", package: "dea-sdk-0.26.0"),
.product(name: "DeaBase", package: "dea-base-0.26.0"),
.product(name: "DeaAiBackendTransporter", package: "dea-ai-backend-transporter-0.26.0"),
]),
]
Attach the stream delivery¶
import DeaAiBackendTransporter
import DeaBase
import DeaSDK
import UIKit
enum Dea {
private(set) static var sdk: DeaSdk?
private(set) static var collect: StreamTransportIntegration?
static func start() {
guard sdk == nil else { return }
let collect = StreamTransportIntegration(
config: CollectConfig(
label: CollectLabel(kind: .human, subjectId: "tester-0142", task: "staging-ai-data"),
facts: CollectDevice(
osVersion: UIDevice.current.systemVersion,
dpi: UIDevice.current.userInterfaceIdiom == .pad ? 132 : 163),
baseUrl: "https://poh-api.dep-protocol.com", // the host only; the transporter adds the path
apiKey: "YOUR_COLLECT_API_KEY"))
let dea = DeaSdk.initialize(options: DeaSdkOptions(
appId: "YOUR_APP_ID",
integrations: [collect],
appKey: "YOUR_APP_KEY",
// Keep motion at full rate while nobody touches the screen: motion is what tells the
// transporter the app is still running, and thinned motion ends the session.
idleMotionMs: 0))
dea.start()
self.collect = collect
sdk = dea
}
}
Then ask for consent, as in User Consent: nothing is captured or sent until the user agrees.
Dea.sdk?.requestConsent(from: self) { agreed in
// The answer is recorded; capture and collection follow it.
}
Warning: With no
baseUrlor noapiKey, the transporter still attaches and reports every piece as sent, but nothing leaves the device. Pass both.baseUrlis the host without a path or a trailing slash.
AiBackendTransport.integration(config:delivery: .stream) builds the same integration from a
BackendTransportConfig, typed as Integration.
Label the sessions¶
Every streamed session is filed under the CollectLabel in the CollectConfig.
| Field | Required | Description |
|---|---|---|
kind |
Yes | .human for a person, .bot for an automated client |
subjectId |
No | Who produced the input: a pseudonymous id of the person or the bot, never personal data |
task |
No | What the sessions are for. Pass "staging-ai-data": there is no default, and a label without it is sent with task null |
botFamily |
When kind is .bot |
Which bot produced the input |
difficulty |
No | "easy", "normal" or "hard" |
taskId, seed |
No | Your own task number and seed |
CollectConfig.facts is a CollectDevice with the OS version, the hardware model, the screen
density in points per inch and whether the app runs in the simulator, which the transporter writes
into each session's header.
An invalid label — a bot without botFamily, an unknown difficulty — does not throw. The
transporter refuses to attach and prints dea: collection label refused, and nothing streams:
Dea.collect?.transporter stays nil.
The label is read once, when the SDK starts. To file what follows under another person, dispose of
the SDK and start it again with a new StreamTransportIntegration:
The running streamed session then ends when its motion stops, within a second. Without motion — the simulator, or motion switched off — it is not ended, and DEA closes it after a while.
Keep the event flow going¶
The transporter ends a streamed session where the app's event flow is interrupted — see Where a streamed session ends. Your app does not need to do anything for a session to end, and it should avoid ending one by accident:
- Set
idleMotionMs: 0. By default the SDK keeps one motion sample every 3 seconds while nobody touches the screen, and a second of silence ends the session. - Do not call
reset()orpause()for a screen, a game or a round.reset()starts a new session, and a pause longer than a second stops the event flow. Callreset()only where a session really ends, such as a logout — see Sessions. - Do not block the main thread for more than a second. Input reaches the transporter on the main thread, so a long stall reads as silence and ends the session.
When the app goes to the background, the transporter sends the rest of the session and its end inside a background task, so the requests can finish after the app leaves the screen.
Examples¶
Show what was sent in a debug screen¶
import DeaAiBackendTransporter
import SwiftUI
final class StreamLog: ObservableObject {
@Published private(set) var reports: [StreamReport] = []
/// Pass as `onSend:` when building the `StreamTransportIntegration`. It is called on a
/// background queue, after each answer and for each piece the queue dropped.
func record(_ report: StreamReport) {
DispatchQueue.main.async { self.reports.insert(report, at: 0) }
}
}
let log = StreamLog()
let collect = StreamTransportIntegration(
config: CollectConfig(
label: CollectLabel(kind: .human, subjectId: "tester-0142", task: "staging-ai-data"),
baseUrl: "https://poh-api.dep-protocol.com",
apiKey: "YOUR_COLLECT_API_KEY"),
onSend: { log.record($0) })
Check that it works¶
- Run the app on a device — the simulator has no motion — and use it for 30 seconds after agreeing to consent.
onSendreports asession_stream_ingestwithoktrueevery 30 seconds while the app is in front, and asession_stream_endwhen the session ends, for example when the app goes to the background.sessionIdnames the streamed session, not the SDK's session id.Dea.collect?.transporter?.statscounts what wassent,failedanddropped, and names the opensession.
If nothing is sent:
| Check | Why |
|---|---|
baseUrl and apiKey are both set |
Without them nothing leaves the device, although every piece reports ok |
Dea.collect?.transporter is not nil after start() |
nil means the label was refused |
| Consent was given | Nothing reaches the transporter before it |
| The session lasted 3 seconds or more, with a touch | A session that ends before its first piece is sent, shorter than that or without input, is not sent |
API¶
| Member | Returns | Description |
|---|---|---|
StreamTransportIntegration(config:onSend:) |
StreamTransportIntegration |
The collection transporter's stream delivery, for DeaSdkOptions(integrations:) |
StreamTransportIntegration.transporter |
StreamTransporter? |
The running transporter, nil before start() or when the label was refused |
StreamTransporter.stats |
StreamStats |
session (the open streamed session, or nil), buffered, sent, failed, dropped, lastError |
StreamReport |
— | What one request came to: at, sessionId, eventType, objects, bytes (gzipped), elapsed, ok, error |
Related¶
- Collecting data — what is sent, when, and where a session ends
- User Consent
- Sessions
- Accounts and keys