Where should documentation effort go?
A specialty practice sees a denial rate three times the network average. Is that incomplete prior authorization, or poor documentation — and which one is worth fixing first?
The decision
Where to spend clinical-documentation effort: which conditions, which populations, which review process. The options are two levers on the same claim, and the question is which one moves denial.
- Action
- prior_auth_complete
- Prior authorization complete
- Outcome
- denial
- Unit
- claims
Why it resists a spreadsheet
The effort is spent on one thing and the saving lands on another, through a chain of steps no single team owns. Documentation quality affects denial rates, which affect write-offs, which affect the reported cost of care — and by the time it reaches a spreadsheet the chain has been flattened into a correlation between two columns that move together.
Specialty is the trap. It looks like it drives denials and it does correlate with them, but it is a proxy for both documentation quality and case mix. It is also not an action anyone can take: you cannot move a practice from specialty to primary care. A model that ranks by specialty produces a finding nobody can act on.
The worked example
Run against the shipped payer scenario: five binary variables, 1,500 rows, screened at an uncorrected threshold of 0.01 over 40 tests.
The scenario is synthetic and its true structure is declared in code, so the screen can be scored rather than eyeballed. All 14 of 14 observable rows were fitted from the table's counts; no probability here was elicited from a model.
What the screen found
- Precision
- 1.000
- of the edges found, the share that are true
- Recall
- 1.000
- of the true edges, the share that were found
All five true edges recovered and nothing spurious. This is the best of the four scenarios, and it is worth saying that the other three are not as good — a site that showed only this one would be choosing its evidence.
The effect
Effect
−35.61 percentage points
risk difference, on the probability that a claim is denied
risk ratio 0.393
Band
−38.76 to −32.45 percentage points
sensitivity_envelope, basis table_sensitivity@0.05
Completing prior authorization moves the denial rate from 58.71% to 23.10%. The band is a sensitivity envelope — how far the answer moves when each table is shifted by 0.05 — and it names that basis rather than presenting itself as a confidence interval, because it is not one.
What to adjust for, and what not to
The part a downstream estimator actually consumes. Three of the five variables carry advice, and one of them must not be adjusted for.
- prior_auth_completen/a
treatment
The variable being intervened on.
- denialn/a
outcome
The variable whose response is being measured.
- documentation_qualityoptional
cause of outcome
It cannot confound the effect, so adjusting for it changes precision rather than the answer.
- specialtyoptional
cause of outcome
It cannot confound the effect, so adjusting for it changes precision rather than the answer. It is upstream of the outcome but not of the treatment, which is why it cannot be the confounder it looks like.
- rework_costdo not control
collider
Conditioning on it can make the treatment and the outcome appear related when they are not. This is the one a reviewer will ask about, because it looks like a useful control and is the most damaging of the three to include.
The minimal adjustment set for this pair is empty. In the recovered graph the treatment has no parents, so there is no backdoor path to block and the full backdoor set of documentation_quality and specialty buys precision rather than validity. A caller who adjusts for both is not wrong; a caller who adjusts for neither is not wrong either, and fewer variables is less variance.
One question this scenario cannot answer
If something we did not measure drives both prior authorization and denial, what is the effect then?
With an unmeasured common cause of the treatment and the outcome, adjustment cannot help — and there is no front-door route either, because the treatment has no child that leads to the outcome. The engine returns identification not established, and names both failed routes rather than reporting the failure.
This is the query that matters most on the page and it has no number. Anything that produced one here would be producing a number shaped exactly like the identified one, which is the failure this product exists to avoid.
What this does not establish: Five binary variables from a synthetic table, screened at 1,500 rows. It is an illustration of the method on a structure whose truth is known, not a payer's claims warehouse, and a real one would have hundreds of columns, missing values and a case-mix argument this scenario does not have.
What the record keeps
The intervention as framed, who approved it against which stored analysis, what was assigned versus which claims were actually reached, and the measured denial rate reconciled against the design-time claim.
- The analysis as run, including its engine version, so a replay months later can say whether the answer still holds
- The adjustment set and the roles table, so a reviewer can see what was controlled for and what deliberately was not
- Assigned versus reached, which diverge and are stored separately rather than reconciled away
- The measured outcome against the design-time claim, with both numbers kept
Book a walkthrough
Bring your own documentation question. Where the structure is not elicitable we will say so in the session rather than at the end of a procurement cycle.