Ask a data science team to make a credit model explainable and you will get something honest (and useless): a score, maybe a threshold, and the features that pushed the score, each with a weight. It is accurate, but it is also not what the regulator asks for, and it is not information the declined customer can do anything with.
Three regimes we have written about recently are asking for explanations. MAS’s FEAT principles want a customer given, on request, a clear account of what data was used in a decision about them and how it affected the outcome, along with the version that produced it now that the AI risk guidelines are landing as supervisory expectations. Australia’s automated decision-making disclosure from 10 December is policy-level, the kinds of information and the kinds of decisions, but the moment a customer asks about their own decision the question becomes specific. New Zealand’s Privacy Act gives a customer the right to see what you held on them and to have it corrected, which only means something if you can say what the decision used. Worth noting, none of the three asks for feature weights.
Three explanations of the same decline
The first column is what most ‘explainable AI’ work produces. The second is what most customers actually receive when they ask for more information, it’s a reason code that was written for the credit bureau, not for them. The third is the only one that answers the regulator’s question and the customer’s at the same time, and it is not a description of the model at all. It names the policy rule that fired, and it says what would change the result.
In a nutshell: the model produced a number. The policy made the decision. The explanation is in that policy.
Why the counterfactual matters more than the reason
A reason on its own is a verdict. “Serviceability” tells a customer they were refused; it does not tell them whether a smaller amount, a longer term or a second income would have been approved, and those are the questions they will ring up and ask your team. A counterfactual answers them before they ask, and it is also the best fairness check a firm has. If the honest counterfactual for a decline is something the customer cannot change, the decision needs a second look before it goes out, not after a complaint.
This is where the record matters, the one we described in the decision record piece. To say “policy v8, verified income against the threshold for this amount” six months later, the decision has to have been stamped with that policy version and those inputs when it was made. A model score alone cannot be turned into that sentence afterwards, because the score does not know which rule consumed it.
What this means for how you build
Our view is that explanations should be written at the level where the decision is actually made, which is policy, not model. A rule that declines when verified income falls below a serviceability threshold can carry its own customer-language template and its own counterfactual, and both coexist with every decision it makes. The model stays as a signal into the rule, free to be as complex as it needs to be, because nobody is asking it to explain itself to a customer.
That is how CxOS is built: reasons are attached to policy, generated at decision time in customer language, stamped with the version to enable traceability, and the counterfactual is part of the reason. You don’t need to wait for a follow-up call to figure it out.