# Telemetry Owns the seam gameplay emits events through, the envelope, the format on disk and the catalogue of events. It does **not** own any backend, storage or analysis; there is none, and the format is chosen so one can be bolted on later without touching the game. Read [Architecture.md](Architecture.md) first. Step 2 in [`../Steps.md`](../Steps.md), before the character exists, on purpose. ## Why this exists before there is anyone to measure Two reasons, and the second is the real one. 1. **Retrofitting emit calls is expensive and lossy.** Instrumenting a finished system means re-reading it and guessing what mattered at the time. Instrumenting as it is written costs a line. 2. **Some questions can only be answered with data that starts early.** Which camera mode people live in, whether the anvil is fun or a chore, whether a crafted sword's damage curve is sane, whether prompts explain themselves. Every one of those is a distribution over sessions, and the sessions that never emitted are gone. So the schema is fixed now, the sink writes a local file, and the catalogue grows one step at a time the way the cheats do. ## Decisions ``` [DECIDED] One subsystem, one sink interface, three sinks on day one: null (the shipping default), log (the editor), JSON Lines to a file (playtests). The emitter never knows where events go. ``` ``` [DECIDED] JSON Lines. One event per line, UTF-8, snake_case keys. Append-only, so a crash mid-write loses one line and not the file; schema-flexible, so a new field breaks no old reader; greppable during development; and the native food of every log and analytics pipeline that might sit downstream. Events, never aggregates: every metric is computed from raw facts later, because aggregation choices change and raw events do not. ``` ``` [DECIDED] No personal data. Player identity on the wire is the server-minted player id, hashed. No names, no platform ids, no chat, no free text. Established now because a field is much harder to remove than to never add. ``` ## Layout ``` Source/Core/Telemetry/ ├── TelemetrySink.h // ITelemetrySink, FNullTelemetrySink, FLogTelemetrySink, FJsonlTelemetrySink ├── TelemetryEvent.h // FTelemetryEvent, FTelemetryEnvelope └── TelemetryEvents.h // the event name constants: TelemetryEvents::PlayerDowned etc. Never a literal at a call site. Source//Core/ └── TelemetrySubsystem.h // UGameInstanceSubsystem: owns the sink, stamps the envelope, exposes Emit ``` ## Types ```cpp // Source/Core/Telemetry/TelemetryEvent.h struct FTelemetryEvent { FName Name; // from TelemetryEvents, never a literal TSharedPtr Payload; // event-specific fields; built with a small fluent helper // The subsystem stamps the envelope; call sites fill Name and Payload only. }; struct FTelemetryEnvelope // stamped by the subsystem onto every event { int32 SchemaVersion = 1; FGuid EventId; // unique per emission, for de-duplication FDateTime TimestampUtc; float GameTime; // seconds since the world began FGuid SessionId; // minted by the server, adopted by clients on join FString PlayerId; // hashed server-minted id; empty on the server bool bIsServer; // authoritative events versus observed ones int32 PartySize; FString Build; // FApp::GetBuildVersion() plus the git hash from Scripts/ FString Map; bool bCheatsUsed; // sticky true after the first cheat this session: tainted sessions are filterable, not deleted }; // Source/Core/Telemetry/TelemetrySink.h class ITelemetrySink { public: virtual ~ITelemetrySink() = default; virtual void Emit(const FTelemetryEnvelope& Envelope, const FTelemetryEvent& Event) = 0; virtual void Flush() {} }; // FNullTelemetrySink: does nothing. The shipping default until there is somewhere to send anything. // FLogTelemetrySink: one line of JSON to the output log under LogTelemetry. Verifies a step emitted what it claims. // FJsonlTelemetrySink: appends to Saved/Telemetry/session__.jsonl through a queue drained by a background // task every two seconds and on Flush; the game thread only builds a small object per event. Flushed on quit and // from the unhandled-exception handler so a crash loses at most two seconds of events. ``` ```cpp // Source//Core/TelemetrySubsystem.h UCLASS() class UTelemetrySubsystem : public UGameInstanceSubsystem { GENERATED_BODY() public: void Emit(FName Name, TSharedPtr Payload = nullptr); // stamps the envelope, hands to the sink void BeginSession(FGuid SessionId); // server mints; a client adopts the replicated id void SetSink(TUniquePtr Sink); // Log in the editor, Jsonl with -telemetry, Null otherwise void MarkCheatUsed(); // called by every bs.* console command }; ``` **There is no static `Telemetry::Log(...)` and there will not be one.** Gameplay code resolves the subsystem from its game instance once and keeps the pointer. A static helper is a global with gameplay state in it, which [Architecture.md](Architecture.md) bans, and it makes every emitting class untestable. ## Where the sink lives on each peer Every peer emits its own events to its own file. The server's file is the authoritative record of what happened; a client's file is the record of what that player saw and did. `session_id` is the same across all of them and `is_server` tells them apart. Nothing is forwarded, because the interesting questions are per-peer by nature. Which sink is installed: the log sink in the editor, the file sink when the executable is launched with `-telemetry` or the console variable `telemetry.File 1` is set, the null sink otherwise. Playtests run with the file on and the folder is collected by hand afterwards; that is the whole backend for now. ## The catalogue Each feature spec's Telemetry section lists what that feature emits; this table is the rollup and the two must agree. A row is added to a spec and here in the same pull request as the emit call. | Group | Events | | --- | --- | | Session | `app_started`, `session_started`, `session_ended`, `player_joined`, `player_left`, `settings_changed`, `cheat_used` | | Movement | `movement_sample`, `jump`, `land`, `camera_mode_changed` | | Interaction | `prop_interacted`, `interaction_refused`, `object_picked_up`, `object_dropped`, `object_thrown`, `object_handed_over` | | Stats | `status_applied`, `status_blocked`, `status_removed` | | Combat | `enemy_spawned`, `enemy_killed`, `ability_used`, `player_damaged`, `player_downed`, `player_revived`, `player_died`, `party_wiped`, `grief_action` | | Crafting | `substance_processed`, `piece_shaped`, `piece_enchanted`, `item_assembled`, `assembly_rejected`, `activity_completed`, `characteristic_discovered`, `weapon_equipped` | | Technical | `perf_sample`, `error_logged`, `net_correction` (count of visible movement corrections per 30 s, from `p.NetShowCorrections`) | ### The rows that carry weight ``` item_assembled family, piece_ids[], name, quality, synergy, substance_value, time_at_bench_s weapon_equipped family, crafted, damage, quality enemy_killed enemy, killer_class, weapon, ability, time_to_kill_s ``` Together these are the crafting-to-combat loop as data: does a better-made sword kill faster, and by how much. That is the one question the whole prototype exists to answer, and it cannot be answered from memory. ``` activity_completed activity, operators, quality, auto_play, faults, contribution_by_player ``` `quality` against `auto_play` says whether the anvil is fun or a chore: if the hold-to-work floor is where most sessions live, the minigame is not earning its place. ``` interaction_refused prop, reason assembly_rejected family, reason, part ``` The two cheapest possible measures of whether the game explains itself. A reason that dominates is a prompt that does not. ## Tests - Automation: the JSONL sink writes one valid line per event and never blocks the calling thread longer than building the object; the envelope carries every field; an event with no payload serialises as an empty object. - Functional: launching the gym with the log sink installed emits `app_started` and `session_started` in that order; every event name a feature test triggers is a constant in `TelemetryEvents`. ## Open questions - **Q1. Sampling.** `movement_sample` and `player_damaged` are the high-frequency rows. Every 5 s and every hit are fine at prototype scale; revisit when a file exceeds a few megabytes per session. - **Q2. An analytics provider.** The engine has an `IAnalyticsProvider` interface that several backends implement. When there is a backend, a fourth sink adapts to it; the emitter does not change. - **Q3. Opt-out.** A playtest build needs a telemetry toggle in settings before strangers play it. With the settings pass, not before. - **Q4. Typed payloads.** A JSON object per event is convenient and unsafe. A typed struct per event name is the later answer if typos in payload keys start costing analysis time.