2021i27, Wednesday: "The vital importance of audit".

Short-ish thought/someone is right on the internet: An all-in-one today - mostly about thinking tools, but with a legal sting in the tail. (Promise!) When I’m talking to my daughter about the welter of information (and mis- and dis-information) that floods across her perception each day, I struggle - as, I imagine, do most parents - to boil the problem down into strategies anyone can actually make work.

“Check the source” is great - but mostly it’s so far removed it’s not going to be evident. “Check the intention” is better; if you can at least make an educated guess about someone’s motives, it tells you a lot. But it’s still too hard for the everyday.

So I’ve settled on a couple of things. In a way, they mean much the same; but there are subtle and I believe useful distinctions. And I think they work just as well for adults:

• First, one about delivery: beware of certainty. Certainty usually implies an unwillingness to learn, or a refusal to accept nuance. The HL Mencken line - that “for every complex problem there’s a solution that’s clear, simple and wrong” - is only too true. As is the rueful joke that there ought to be a “You know, it might be a bit more complicated than that” party out there somewhere. Put all that together, and being very careful of stuff told you by people whose presentation of it implies that they KNOW they’re right, they KNOW it’s true, becomes a sound strategy.
• Second, one about people. A core test for me, and one I try to persuade my daughter to adopt, is to look at the person I’m talking to and try to imagine them saying, “I could be wrong", and meaning it. If I can, I’ll listen. If I can’t, there’s a problem. The other value of this one, of course, is one can apply it to oneself. Am I being sufficiently humble about my state of knowledge? Or am I trapping myself or misleading others about the risk of inaccuracy? (I used to use this one as a reporter sometimes when interviewing someone about some plan their organisation had. I’d ask them: what could go wrong with this? If I got a sensible, thoughtful answer, I’d tend to feel a lot better about the plan; it seemed, in modern parlance, like someone might have run a pre-mortem. If I didn’t - and goodness knows I very rarely did - the temptation to do a Paxman and ask myself “why is this lying liar lying to me” got an awful lot stronger.)

The other critical one, of course, is an awareness of confirmation bias. There are a million cognitive biases, but this one’s the killer - because it means we test information which confirms our core beliefs with far less care than stuff that doesn’t.

God knows I fall down on all three of these, every day. But they’re vital tools; and if I can help my daughter adopt them, I’ll have done at least one thing right as a parent.

Hold on a minute, you may say: I thought this was a SIROTI. Where’s the link?

In fact, it’s to something I’ve linked to before, but it’s more important than ever. Dan Davies of D-Squared Digest notoriety once coined three rules which he said he’d learned in business school and dubbed the “One Minute MBA”. He promulgated them after the Iraq War in relation to (as it turned out) the abject absence of the WMDs which were the ostensible reason for the 2003 Iraq invasion. In short:

• Good ideas do not need lots of lies told about them in order to gain public acceptance. When anyone handwaves a bunch of deliberate untruths and says that the end justifies the means, walk away. Fast.
• Fibbers’ forecasts are worthless. Simply put: you can’t fudge or mitigate or moderate forecasts made by a liar. They’re worthless. Ignore them altogether.
• The vital importance of audit. You need to set the success boundaries for a project before you start - and then you need to check your working afterwards. Anyone who won’t do the first, or seems to fail to want to learn from the second? Again, ignore them if you can. Vote them out if you can’t. They’re dangerous.

I’ve found them enormously effective as a test for all kinds of other things, both political and otherwise. I’ll leave their application to our various current travails as an exercise for the reader.

The last of these three, the audit bit, is to my mind the really important one - and it chimes back to the original tools. Acknowledging how things actually went - asking the “so how did we do?” question and wanting to know the answer - isn’t just basic intellectual honesty; it’s the most fundamental requirement for doing things better in the future. And checking your working is at the heart of that.

Own your errors. Learn from them. Do better. Anything else, from anyone with any kind of responsibility to others, is a betrayal of that responsibility.

Don’t get me wrong. I’m writing badly if for a second I seem to be falling into the trap of putting all public figures into a big box marked “liar”. Humans lie. It’s what we do. The important thing is how much, when and why. It’s too easy, and ultimately incredibly self-defeating and damaging, to play that political game.

But the next time some grand plan is espoused, by anyone, listen for the lies told to sell it, the forecasts made, the success metrics. And then watch out later on to see if those metrics are taken seriously - or are handwaved. It’ll tell you a lot about whether you can trust those making the play.

(And now for the legal bit. I’m a great believer, in court, in crafting a narrative. Starting with one’s case theory, it’s vitally important to end up with a story that makes sense. A lot of things can happen during a case, of course, but the team with the most Occam’s Razor-friendly story to tell starts comfortably ahead.

This is where the One Minute MBA can be really useful. On the one hand, if you can show exaggeration or false forecasts from the other side, that’s a great way to undermine credibility; and if they’ve skirted around anything auditable (or have tried to handwave their promised outcomes later) that, too, is a fantastic lever on which to push.

On the other, of course, if it’s your side that’s got the outlandish predictions and the dodged promises, don’t under any circumstances hope no-one will notice. Your story will have to explain them, rationalise them - if at all possible, find a way of making them sound sensible rather than left-field. Otherwise no-one’s going to live happily ever after.)

Combine a suspicion of certainty, “I could be wrong”, an awareness of confirmation bias, and the One Minute MBA, and that’s a powerful toolbox for dealing with the blizzard of BS in which we all find ourselves. (An appreciation of the difference between lies and BS is useful too.)

Might be a bit much for a 14 year old, not least because I (like all adults) struggle to put it into effect myself. But I think it’s a start.