From synthetic players to Unity Sentis: tracing Retentia AI end to end

Retentia AI connects four parts that are often discussed separately: a Python data generator, a small machine-learning pipeline, inference inside a Unity client, and a React dashboard. I built it as an end-to-end prototype to explore how those pieces can fit together. The important qualifier is that the player profiles are synthetic; this project does not establish that its churn predictions work for real players.

The repository contains the training scripts, generated CSV, ONNX model, Unity code, and dashboard. This article follows that actual path, then marks the demo values and local file bridge that should not be mistaken for a production analytics service.

Generate examples before training

The Python script creates 2,000 sample profiles from five hand-authored personas. It assigns values such as sessions, time since last login, levels completed, and support tickets, then derives an IsChurned label using rules in the same generator. The random seed makes generation repeatable; it does not turn the profiles into observations of actual players.

The training script selects ten profile fields, normalizes them to fixed ranges, and splits the generated set into training and test portions. It fits a 100-tree Random Forest with a maximum depth of five, then exports the model to ONNX. The script prints an accuracy value for the held-out synthetic rows, but I do not report that as a real-world model score: both sides of the split come from the same hand-written simulation.

That distinction changes what a score can mean. It can help check that the export pipeline runs on examples produced by this generator. It cannot show that the model predicts churn in a live game, that the labels match a useful business definition, or that an intervention improves retention.

Carry the feature contract into Unity

Unity loads the ONNX asset through Sentis and passes a player profile through the inference service. The feature order and the normalization ranges are repeated in Python and C#. That makes the boundary easy to understand, but it also creates a drift risk: a future change to one side can silently alter what a feature vector means.

The inference service returns 0.5 when the model is unavailable or a prediction path fails. That keeps the prototype from crashing at the missing-model boundary, but the fallback is not a measured probability. A consumer should distinguish “model could not answer” from “the model estimates fifty percent risk.”

For the current contract, I would add a small set of shared golden examples: one named profile, its ordered input vector, the normalized values, and the expected exported-model output within a tolerance. Running the same examples through Python and Unity would catch changes to field order and scaling before they appear in the dashboard.

The relevant implementation is visible in train_model.py, ChurnPredictionEngine.cs, and the committed ONNX model.

Move data through a local bridge

The Unity writer serializes its current demo state to _WebDashboard/public/data.json once per second. The React app reads that file on the same cadence. In the other direction, a dashboard action posts to a small Express endpoint; the server writes a command to commands.json, which the Unity command reader polls and clears after processing.

This file-based exchange makes the prototype easy to run as a local demonstration. It also couples the Unity project to a particular dashboard directory. The Express server has no authentication and enables unrestricted CORS, so this is not a remotely exposed command service. A deployed version would need an authenticated transport, validated commands, durable delivery and a clear player-identity and consent model.

The file bridge is implemented in WebDataWriter.cs, WebCommandReader.cs, and the Express server.

Read the dashboard as a demo

Some dashboard panels are illustrative rather than live measurements. The engagement chart is filled with random values in the frontend; the active-user count is fixed in the component; and the recommendation text is static. The app also reads the Unity-written JSON file for other fields. Without making that distinction, a polished screen can make sample output look like real telemetry.

I keep those claims separate here: the repository demonstrates a synthetic-data-to-Unity inference path and a local dashboard command loop. It does not document real player records, a production backend, an experiment with actual users, or a validated retention outcome.

What evidence would make the next version stronger?

I would start by defining churn as a time-bounded outcome, then evaluate the model on a properly separated dataset that reflects the intended use. Before making a quality claim, I would compare it with a simple baseline, inspect class balance and calibration, and test whether the label leaks information from the prediction window. Any real telemetry would also need an explicit collection and retention policy.

On the integration side, I would replace the shared files with a versioned, authenticated interface and add tests for stale data, duplicate commands, process restarts, and rejected payloads. On the dashboard, I would remove the random chart and fixed sample count or label them visibly as demo data.

Retentia AI is most useful today as an architecture and integration prototype: it makes the path from generated input to Unity inference visible, along with the limits of each signal. The public repository contains its source code.

← All notes