Skip to content

Collect Data

The test and staging packages of the SDK can send what they capture to DEA's collection service, where it trains the models that score sessions. This page shows how a test or staging app attaches the collection transporter's stream delivery: the app's input goes to DEA in pieces while it runs, and the transporter cuts it into sessions on its own, so your app makes no call per screen or per round.

Warning: Collection sends the raw input of your app — touch positions, scrolls, which class of key was pressed, device motion — to DEA. Use it only in an app DEA asked you to collect with, with your users' agreement. On Android the stream delivery attaches on every package, the production one included: never add the collection transporter to a build you release.

When to use it

  • A data-collection or QA app that DEA asked you to build
  • A staging build of your app whose testers' sessions should reach DEA's training data

Prerequisites

  • An appId and appKey for the app — see Accounts and keys
  • The collection service's API key, which your DEA contact gives you. It is not the appKey
  • minSdk 21 or later, compileSdk 35, Java 17
  • Your users agreed to collection — see User Consent

Usage

Install the packages

Each channel of a release is its own Maven repository. A staging app takes the SDK from the staging repository and the transporter from the test repository, the only one that publishes it:

// settings.gradle.kts
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven {
            url = uri("https://sdk.dep-protocol.com/0.26.0/staging/android")
            content { includeGroup("ai.dea") }
        }
        maven {
            url = uri("https://sdk.dep-protocol.com/0.26.0/test/android")
            content { includeModule("ai.dea", "dea-ai-backend-transporter") }
        }
    }
}
// app/build.gradle.kts
dependencies {
    implementation("ai.dea:dea-sdk:0.26.0")
    implementation("ai.dea:dea-ai-backend-transporter:0.26.0")
}

Warning: Keep the content filters. The production, staging and test builds of dea-base and dea-sdk share one coordinate, ai.dea:dea-base:0.26.0, and are different files. Without the filters Gradle takes them from whichever repository answers first, and a staging app can end up with the test packages without any error.

A test app declares the test repository alone, with no filter: https://sdk.dep-protocol.com/0.26.0/test/android.

Attach the stream delivery

import ai.dea.sdk.DeaConfig
import ai.dea.sdk.DeaSdk
import ai.dea.transporter.backend.AiBackendTransport
import ai.dea.transporter.backend.shared.CollectConfig
import ai.dea.transporter.backend.shared.CollectDevice
import ai.dea.transporter.backend.shared.CollectLabel
import ai.dea.transporter.backend.shared.LabelKind
import ai.dea.transporter.backend.stream.StreamTransportIntegration
import android.app.Application
import android.os.Build

class MyApp : Application() {
    lateinit var collect: StreamTransportIntegration
        private set

    override fun onCreate() {
        super.onCreate()
        collect = AiBackendTransport.integration(
            config = CollectConfig(
                label = CollectLabel(kind = LabelKind.HUMAN, subjectId = "tester-0142", task = "staging-ai-data"),
                facts = CollectDevice(osVersion = Build.VERSION.RELEASE, model = Build.MODEL, dpi = 160.0),
                baseUrl = "https://poh-api.dep-protocol.com", // the host only; the transporter adds the path
                apiKey = "YOUR_COLLECT_API_KEY",
            ),
        )
        DeaSdk.init(
            this,
            DeaConfig(
                appId = "YOUR_APP_ID",
                appKey = "YOUR_APP_KEY",
                // Keep motion at full rate while nobody touches the screen: motion is what tells the
                // transporter the app is still running, and thinned motion ends the session.
                idleMotionMs = 0,
                integrations = listOf(collect),
            ),
        )
    }
}

Then ask for consent, as in User Consent: nothing is captured or sent until the user agrees.

import ai.dea.sdk.DeaConsentDialog

DeaConsentDialog.show(activity) { granted -> /* the answer is recorded */ }

Warning: With no baseUrl or no apiKey, the transporter still attaches and reports every piece as sent, but nothing leaves the device. Pass both; collect.config.uploads is true only when both are set. baseUrl must be an https host, without a path.

AiBackendTransport.integration throws IllegalArgumentException when the label is not one DEA accepts or baseUrl is not https. It runs before DeaSdk.init, so the exception reaches you there.

Label the sessions

Every streamed session is filed under the CollectLabel in the CollectConfig.

Field Required Description
kind No, LabelKind.HUMAN by default LabelKind.HUMAN for a person, LabelKind.BOT for an automated client
subjectId No Who produced the input: a pseudonymous id of the person or the bot, never personal data
task No What the sessions are for. Pass "staging-ai-data": there is no default, and a label without it is sent with task null
botFamily When kind is LabelKind.BOT Which bot produced the input
botInjection No How a bot's input reached the app: "framework", "os-level" or "human-relay"
difficulty No "easy", "normal" or "hard"
taskId, seed No Your own task number and seed

CollectConfig.facts is a CollectDevice with the OS version, the hardware model, the screen density and whether the app runs in an emulator, which the transporter writes into each session's header. The SDK reports positions in dp, so pass dpi = 160.0.

The label is read once, when the SDK starts. A second DeaSdk.init returns the running instance and ignores its config, so to file what follows under another person, dispose of the SDK and start it again with a new integration:

DeaSdk.instance?.dispose()
// Then call DeaSdk.init again with an integration for the new person's subjectId.

The running streamed session then ends when its motion stops, within a second. Without motion — an emulator, or motion switched off — it is not ended, and DEA closes it after a while.

Keep the event flow going

The transporter ends a streamed session where the app's event flow is interrupted — see Where a streamed session ends. Your app does not need to do anything for a session to end, and it should avoid ending one by accident:

  • Set idleMotionMs = 0. By default the SDK keeps one motion sample every 3 seconds while nobody touches the screen, and a second of silence ends the session.
  • Do not call reset() or pause() for a screen, a game or a round. reset() starts a new session, and a pause longer than a second stops the event flow. Call reset() only where a session really ends, such as a logout — see Sessions.
  • Do not pause while the app is in the background. The SDK then drops the return to the foreground, and the transporter waits for the next one before it streams again.

Note: The SDK reports the app going to the background when its last started activity stops. An activity that is recreated on rotation stops and starts again, so a rotation can end the session. The notification shade and dialogs do not.

Examples

Show what was sent in a debug screen

import ai.dea.transporter.backend.AiBackendTransport
import ai.dea.transporter.backend.shared.CollectConfig
import ai.dea.transporter.backend.shared.CollectLabel
import ai.dea.transporter.backend.stream.StreamReport
import android.os.Handler
import android.os.Looper

class StreamLog {
    private val main = Handler(Looper.getMainLooper())
    val reports = mutableListOf<StreamReport>()

    // Called on a background thread, after each answer and for each piece the queue dropped.
    fun record(report: StreamReport) = main.post { reports.add(0, report) }
}

val log = StreamLog()
val collect = AiBackendTransport.integration(
    config = CollectConfig(
        label = CollectLabel(subjectId = "tester-0142", task = "staging-ai-data"),
        baseUrl = "https://poh-api.dep-protocol.com",
        apiKey = "YOUR_COLLECT_API_KEY",
    ),
    onSend = { log.record(it) },
)

Check that it works

  1. Run the app on a device — an emulator has no real motion — and use it for 30 seconds after agreeing to consent.
  2. onSend reports a session_stream_ingest with ok true every 30 seconds while the app is in front, and a session_stream_end when the session ends, for example when the app goes to the background. sessionId names the streamed session, not the SDK's session id.
  3. collect.transporter?.stats counts what was sent, failed and dropped, and names the open session. A test or staging package also writes a collection.uploaded entry for every request to logcat, under the tag dea.

If nothing is sent:

Check Why
collect.config.uploads is true Without baseUrl and apiKey nothing leaves the device, although every piece reports ok
Consent was given Nothing reaches the transporter before it
The session lasted 3 seconds or more, with a touch or key press A session that ends before its first piece is sent, shorter than that or without input, is not sent
The content filters are in place Without them the app can resolve another channel's SDK

API

Member Returns Description
AiBackendTransport.integration(config: CollectConfig, onSend) StreamTransportIntegration The collection transporter's stream delivery, for DeaConfig(integrations = …)
StreamTransportIntegration.transporter StreamTransporter? The running transporter, null before DeaSdk.init
StreamTransporter.stats StreamStats session (the open streamed session, or null), buffered, sent, failed, dropped, lastError
StreamReport — What one request came to: at, sessionId, eventType, objects, bytes (gzipped), elapsedMs, ok, error