Which clients should the desk call?
Contact frequency correlates with retention. But high-value clients get more contact and stay anyway. Does more contact cause clients to stay, and does it depend on tolerance for volatility?
The decision
Which clients to call, and with what offer. The capacity is small and the value per client is large, so the ranking matters more than the average — and the ranking is a causal quantity per subgroup, not a propensity score.
- Action
- advisor_contact
- Advisor contact
- Outcome
- attrition
- Unit
- households
Why it resists a spreadsheet
The clients most likely to leave and the clients most worth keeping are different populations, and a model that ranks by churn probability alone will spend the whole call list on the cheapest relationships. What is wanted is the effect of the intervention multiplied by the value at stake.
There is a fairness problem hiding in it too. Targeting by predicted behavior concentrates effort on groups the model already believes behave a certain way, and that is a decision someone will have to defend. Making the structure and the assumptions visible is what makes that defence possible rather than retrospective.
The worked example
Run against the shipped wealth-retention scenario: five binary variables, 1,500 rows, screened at an uncorrected threshold of 0.01 over 40 tests.
Same declared-truth scoring as the other three. All 11 of 11 observable rows fitted from counts, and the network was built without a model call — the counts are the evidence, so no probability in it was elicited.
What the screen found
- Precision
- 1.000
- of the edges found, the share that are true
- Recall
- 0.833
- of the true edges, the share that were found
Edges the screen missed
- volatility_tolerance → attrition
Five of six true edges recovered and nothing spurious. This is the 0.833, and it propagates. The missed edge is exactly the variable the scenario's own question asks about, which is why this page is the one that shows what a missing edge does to the advice.
The effect
Effect
−16.83 percentage points
risk difference, on the probability that a client leaves
risk ratio 0.364
Band
−17.34 to −16.32 percentage points
sensitivity_envelope, basis table_sensitivity@0.05
More contact moves attrition from 26.48% to 9.65%, adjusting for client segment. The band is narrow because the table is small and its counts are sharp; that is a property of this synthetic table rather than a statement about your book.
What to adjust for, and what not to
Four of the five variables carry advice. One is a confounder that must be adjusted for, one must not be touched, and one carries a label that is only correct because we missed an edge.
- advisor_contactn/a
treatment
The variable being intervened on.
- attritionn/a
outcome
The variable whose response is being measured.
- client_segmentrequired
confounder
Leaving it out leaves a backdoor path open, and the estimate absorbs the difference. It is an ancestor of both, and it is the whole reason the naive contact-versus-retention comparison overstates the case.
- aum_changedo not control
collider
Conditioning on it can make the treatment and the outcome appear related when they are not. It is a plausible-looking control and including it would manufacture an association between contact and attrition.
- volatility_tolerancen/a
neither
Reported as neither an ancestor of the treatment nor of the outcome. That is true of the graph we recovered — and the graph we recovered missed the volatility_tolerance → attrition edge. This row is the recall figure arriving in the advice rather than in a footnote.
The minimal adjustment set is client_segment alone. The full backdoor set also contains volatility_tolerance, which adds nothing once the segment is controlled — and which the page's own question suggests should matter. It would, if we had found its edge.
One question this scenario cannot answer
If an unmeasured client attribute drives both contact and attrition, what is the effect then?
Adjustment cannot help against an unmeasured common cause, and there is no front-door route because the treatment has no child leading to the outcome. Identification is not established, and the engine says which two routes failed and why.
The supporting number is the E-value: an unmeasured confounder would have to be associated with both contact and attrition by a factor of 4.93 to explain the effect away. That is a statement about how robust the answer is to a confounder of a stated strength, not a claim that none exists.
What this does not establish: The scenario's headline question mentions volatility tolerance, and our screen did not find the edge that would let us answer it. The effect above is computed on the graph we recovered, adjusting for what that graph says to adjust for. Adjusting for a variable we cannot see is not something a bigger model fixes.
What the record keeps
The outreach as framed, who approved it against which stored analysis, which households were assigned versus actually reached, and the measured attrition reconciled against the design-time claim.
- The analysis as run, with its engine version, so a replay can say whether the answer still holds
- The roles table, so the segment control and the deliberate non-control of aum_change are both on the record
- Assigned versus reached, stored separately
- The fairness question, as the subgroup breakdown that was actually used for the ranking
Book a walkthrough
Bring the retention decision. The useful part is usually the variable everyone assumed was a confounder and was not.