Saturday, April 18, 2020

A Lead Metric for COVID-19

A significant difficulty with managing complex problems is trying to focus on the right things.  Leaders, the old adage goes, focus on right things, and managers / admin types focus on doing things rightly.  With marketing enrollment for Christian schools, I can boil down the whole process to five lead metrics.  And your average Admissions Director would only be in charge of two of these five.  

Most people I know would be thrilled if you could boil their entire job description down to two points.  In the case of beating COVID-19, let's boil it down to one. 

In contrast to a lag metric – which tells you if you are successful -  (e.g. the number of students the first day of school) a lead metric (# of students represented by new families who visited with the Principal) – tells you if you are likely to be successful.  A lag metric measures success at the end, whereas a lead metric is something you measure as you go.  A lead metric predicts success and is something you can work to improve as you go. 

Robinson Meyer and Alexis Madrigal actually came up with a lead metric for our progress in defeating COVID-19, but apparently they do not know about the Four Disciplines of Execution (Covey et. al.), or the power of focusing on the right lead metric.  I write with the hope that someone picks it up. 

The metric they propose is the percent of positive COVID-19 test results for a period of time (probably a day, or maybe every three days.) 

Imagine a grain elevator with 330 million kernels of corn.  In a routine inspection, an employee notices a unique fungus growing on some of the corn.   What’s clear is that the fungus-infected corn is infecting nearby corn, and the only way to stop it is to move the infected corn away from the non-infected corn.  So the now the race is on to save as much of the corn as possible, and everyone gets out their shovels to take out the infected corn.

Immediately the workers start shoveling, and they send their corn through the size sorter, which traps the swollen, fungus-laden kernels, and allows the others to pass.  The size sorter also does a fairly accurate count of which kernels do and do not have the fungus.   Shoveling out the most obviously infected corn, the grain elevator employees are perturbed to learn that 20% of their corn is infected and must be discarded.

It becomes clear that the fungus is rapidly spreading, soon they bring in the pay loaders.  Since the fungus was in the top of the grain silo, they had assumed that the deeper they went, the less likely they would find fungus-infected corn.  To their horror, even though the pay loaders were sending 100 times the amount of corn through the size sorter than  they could do with their shovels, still 20% of their corn was infected.  So clearly the fungus infection was much, much worse than they thought.

Ultimately, at considerable cost, they rented a special high volume auger and sorter, and after two days, they managed to remove all their swollen fungus corn from the good corn.  All 330 million kernels went through "the beast" as they called it, at considerable expense.  To be extra cautious, they moved all the good corn to another silo altogether.

Only with the high speed auger did they notice that the percent of fungus-laden corn going through the sorter actually decreased.  In fact, towards the bottom of the silo, only 2% of the corn was swollen with the fungus. 

With all their corn in a new, well cleaned silo, everyone breathed a sign of relief.  But a few days later, to their chagrin, the fungus was back!  Apparently the fungus could spread microscopically from corn that had the fungus but had not swollen, or at least not very much.  Or small kernels with the fungus also got through the size sorter.  So they couldn't completely eliminate all of the bad corn from the good corn, no matter how many times they ran it through.  

With no time to lose, the elevator now bought that high speed auger and size sorter, and ran all the remaining good corn through "the beast" one more time. This time, to their relief, they found that only 10% of their corn had gone bad.  Even buying the equipment, the process of running all the corn through the process was expensive, just for the massive amount of electricity the process required.  

After this 2nd go-around, the manager of the grain elevator lit a cigar and had one of his legendary “big-thinks.”  As pondered the situation, it occurred to him that one number could tell him whether he was testing his corn too often or two little.  And that number was the percent of the corn that was swollen with fungus every time it went through the sorter. 

Some portion of all his corn had to go through "the beast" every day.  Now, with the right lead metric, he could try different strategies.  Instead of blowing his budget running all the corn through the high speed auger, he ran 1/14th  of it everyday, because the fungus seemed to fully develop from microscopic to ruined in 14 days.  Then he tried cleaning his grain in sections of the elevator, based on the initial percentages of bad corn in that area. 

He realized that to beat the fungus, his goal was to do auger / sorter testing strategy that reduced the percent of corn with the fungus, every time he did in fact test.  The lower the percent, the lower his ongoing costs, and the more grain he saved.

(Note: every good lead metric results in a clear good question, which is this case is:  How can I test the corn most cost effectively to reduce the number of infected kernels every time I test?) 

The corn testing process was unavoidable, because our cigar-loving elevator manager soon learned that the USDA would not be able to come up with any sort of  cure for at least 18 months.  Testing was required, but how much?

Over time, it became widely accepted by grain elevator managers that the amount of testing they needed depended on the percent of infected corn they found.  That was the best they could do financially to minimize their ongoing costs of managing the fungus through the sorting process, as well as saving as much of their corn as possible.  

Rather wasting money on over-testing, or losing too much corn by under-testing, the elevator managers focused on the percent of positive, infected corn they were getting.  If the percentage was increasing, they had to test more to save their corn.  If  the infected percentage was less, they didn’t have to run their corn through the high speed sorter as much. 

Now back to the real COVID-19 world.  As Meyer and Madrigal document, the US and South Korea diagnosed their first case the same day, and both have tested about 1% of their population.  They tested early, we tested later.  In the case of South Korea, which has relaxed the economy-killing social distancing that the US has, about 2% of the tests were positive.  

In the United States, even as we increased our daily testing over a hundred-fold, the positive rates has remained steady at 20%. 

In other words, the more we tested, the more we found.  Our grain silo had to have been pretty contaminated by the time we started testing.  Asymptotic and pre-symptomatic carriers gave the virus to many, many others. South Korea was much more successful in identifying their contagious people early on, and many less people got it.  Their grain silo was much less contaminated. 

The amount of testing we will need to do in America should be determined by the percent of people who test positive.  The higher that percent (New York City = 41%) the more mass testing is needed.  The lower the percent, the less mass testing is needed.  This sort of lead metric would be helpful in determine how much to test over a region, or for a specific time period.

Clearly massive testing is cheaper than bailing out a comatose economy.  But how much testing?  The rate of positive tests for COVID-19 is the correct lead metric to know how much testing we need, on an ongoing basis, to beat this scourge on our nation.  This is a much more elegant and economically friendly way to quarantine the sick, and protect the vulnerable, then massive lock-downs.