Predicting process time without leaking the future in FlowTwin AI

An offline process model can look impressive while answering a question nobody can ask in production: what did it know at the moment it made the forecast? If a training row quietly includes the eventual case length, a later offer value or the terminal status, the model has already seen part of its answer.

I built FlowTwin AI to explore predictive process monitoring on the public BPI Challenge 2017 loan-application event log. The central design rule is simple: at forecast time t, features may use only the events and values observed by t. The eventual ending is a label for training and evaluation, never an input to that forecast.

Turn an event log into a point-in-time case

FlowTwin downloads the compressed XES event log, checks the published checksum, then streams traces into SQLite. The parser retains the activity, timestamp and selected case attributes it needs, normalizes event time, and records data-quality issues such as missing timestamps, duplicate case IDs and events arriving out of order. A trace is treated as one application case; that unit-of-analysis assumption belongs in the interpretation, not hidden in the implementation.

For the regression target, the prototype considers a trace complete when its last observed activity matches one of six configured terminal activities: A_Pending, A_Denied, A_Cancelled, O_Accepted, O_Refused or O_Cancelled. It excludes traces that do not match this list instead of pretending that the last event in a truncated history is a known completion time.

That is a deliberate, visible heuristic. The source log’s actual process semantics still need to confirm that these endings mean what the model assumes. Training only on cases that pass the rule also selects a population: this is not yet an estimate of time-to-resolution for every application that was still open.

Draw a hard line at the forecast moment

At each checkpoint, FlowTwin builds a snapshot after 1, 3, 5, 10, 20 or 40 observed events, if that many events are available and at least one later event remains. The feature builder receives an explicit prefix length and slices the case history before calculating activity counts, elapsed calendar hours, time since the prior event and the current activity. Selected attributes are carried forward only once they appear in an application-start field or a timestamped event.

This is the important distinction between a feature and a target. To train a supervised model, the code calculates the calendar hours from the end of the observed prefix to the trace’s eventual terminal event. That future timestamp is allowed on the target side. It is not passed into build_features, which reads only events[:prefix_count]. Tests check that a requested amount or loan goal first seen later in a trace stays unavailable to an earlier snapshot.

The model is a scikit-learn HistGradientBoostingRegressor trained with absolute-error loss. FlowTwin also keeps a plain baseline: the training-set median remaining time for the current activity, with a global median as fallback. That gives the model a useful hurdle. A complex estimator has not shown value just because it can produce a number; it should improve on a simple rule on cases it never trained on.

Let time and cases shape the evaluation

Randomly splitting individual snapshots would scatter related moments from the same application across training and test data. FlowTwin instead orders cases by their start time, creates training, validation and later test windows, keeps each case’s prefixes in one window, and excludes completed traces that cross a training or validation boundary.

One case can contribute several checkpoints, so each checkpoint receives only a share of that case’s weight. The case’s total influence stays equal to one, whether it has two available snapshots or six. Overall errors, the baseline comparison and interval summaries use those weights; stage summaries and the bootstrap interval work at case level rather than treating correlated checkpoints as independent applications.

Validation residuals set an empirical 90th-percentile error radius. The final test window then reports whether that interval covered its observed targets and how wide it was. This is a useful uncertainty check, not a guarantee: a historical process can change, and coverage can fall when later cases differ from the validation period.

What the evidence supports today

The latest recorded GitHub Actions run for the reviewed FlowTwin source revision passed compilation and unit plus synthetic integration tests on Python 3.11 and 3.12. Those checks cover such things as prefix isolation, checkpoint selection, XES handling and a small training smoke test on generated cases.

The original BPI event log was not available to the authoring environment when the repository was audited. The project therefore has no verified real-data training run, test MAE, baseline improvement or interval-coverage result to publish yet. The synthetic smoke test proves that the pipeline can exercise its code path on a controlled fixture; it does not establish predictive performance on the historical log.

The data is also from 2016–2017, not a live process feed. Calendar duration includes nights, weekends, queue time and customer waiting; it is not staff work time or an SLA. The model is an offline research and portfolio prototype, not a credit decision, customer-contact or staffing system. An activity associated with a longer estimate is not evidence that the activity caused the delay.

The useful result is the evaluation contract

For me, FlowTwin’s strongest engineering idea is not the choice of gradient boosting. It is making the forecast moment reviewable: define what has been observed, separate future labels from available inputs, keep related checkpoints together, compare with a baseline, and leave the score unpublished until the original-data run exists.

My next validation step is to run the checksum-verified source log end to end, inspect the terminal-activity and attribute-timing assumptions against its event semantics, and publish the generated metrics with their date windows and case counts. If the model does not beat the case-balanced baseline on the untouched test period, that is the result to report too.

FlowTwin AI is public on GitHub, and the BPI Challenge 2017 dataset record is the source to cite and review before reusing the event log. For related data work, read how I quarantine and replay synthetic telemetry in Aurora and how ProcureScope preserves source pages before normalization.

← All notes