Is operator training worth the investment?
Rework spikes on nights. Is that the shift itself, the higher line speed the night crew runs, or the training mix on that shift — and would training more operators pay for itself?
The decision
Whether to train operators on the line, and on which shift. The action is the training standard; the outcome is a defect; the unit is a batch.
- Action
- operator_training
- Operator training
- Outcome
- defect
- Unit
- batches
Why it resists a spreadsheet
Shift is upstream of both the training mix and the line speed, so "nights are worse" is an observation about the rota rather than a cause anyone can act on. The actionable variables are the ones downstream of it, and the report that reaches the plant manager is usually the rota observation.
That inversion is the whole difficulty. The most visible difference between good and bad weeks is the one variable nobody can change, and the variables that can be changed are quieter.
The worked example
Run against the shipped plant-rework scenario: five binary variables, 1,500 rows, screened at an uncorrected threshold of 0.01 over 40 tests.
Same declared-truth scoring. All 10 of 10 observable rows fitted from the counts in the table.
What the screen found
- Precision
- 1.000
- of the edges found, the share that are true
- Recall
- 0.667
- of the true edges, the share that were found
Edges the screen missed
- shift → defect
- shift → line_speed
Four of six true edges recovered and nothing spurious. This is the weakest of the four scenarios, and it is the most instructive, because the two missing edges are both from the same variable and that variable is the one the narrative is about.
The effect
Effect
−16.90 percentage points
risk difference, on the probability that a batch has a defect
risk ratio 0.343
Band
−16.97 to −16.83 percentage points
sensitivity_envelope, basis table_sensitivity@0.05
Enhanced training moves the defect rate from 25.72% to 8.81%. The band is suspiciously narrow — sixteen-hundredths of a percentage point — and a reader should treat a band that tight as a property of a small synthetic table rather than as precision about a production line.
What to adjust for, and what not to
Three of the five variables carry advice, and the most interesting row is the one that says not required about the variable the story blames.
- operator_trainingn/a
treatment
The variable being intervened on.
- defectn/a
outcome
The variable whose response is being measured.
- shiftoptional
cause of treatment
Upstream of the treatment, and it reaches the outcome only through it: there is no backdoor path to block, so omitting it does not confound the estimate. It is an ancestor of both, which is the textbook definition of a confounder — and the definition is wrong here, which is exactly why the advice is computed from the graph rather than from the definition.
- line_speedoptional
cause of outcome
It cannot confound the effect, so adjusting for it changes precision rather than the answer.
- rework_hoursdo not control
collider
Conditioning on it can make the treatment and the outcome appear related when they are not. It is downstream of the outcome, so it is the variable most likely to be reached for as a proxy and the one that does the most damage.
The minimal adjustment set is empty, and this is the row that was wrong until it was fixed. shift was labeled a confounder requiring adjustment while the same response returned an empty minimal set — one field saying to adjust for it and another saying there was nothing to adjust for. The engine was corrected, not the page.
One question this scenario cannot answer
If something unmeasured — a maintenance regime, an operator cohort — drives both training assignment and defects, what is the effect then?
Adjustment cannot help against an unmeasured common cause, and there is no front-door route because the treatment has no child leading to the outcome. The engine returns identification not established and names both failed routes.
This matters more here than elsewhere, because on this scenario we already know the graph is incomplete in a way that touches the answer — see the caveat below. A refusal on top of an incomplete graph is the honest position, not a technicality.
What this does not establish: The important caveat on this page. The screen missed shift → defect. In the graph we recovered, shift therefore reaches the outcome only through the treatment, which is why the advice above is optional rather than required. In the scenario's declared truth the edge exists, which makes shift a genuine confounder and means the effect above was computed without adjusting for a variable that should have been adjusted for. The advice is correct about the graph it was given and the graph is incomplete. A better screenshot, and a worse product, would have shown the other three scenarios only.
What the record keeps
The training standard as framed, who approved it against which stored analysis, which batches were assigned versus actually run on the trained standard, and the measured defect rate reconciled against the design-time claim.
- The analysis as run, with its engine version
- The roles table, including the recovery figures, so the incomplete graph is on the record rather than in someone's memory
- Assigned versus reached, stored separately
- The measured defect rate against the design-time claim
Book a walkthrough
Bring the line decision. The most useful part of the session is usually the variable the plant blames and cannot change.