I am attending a workshop on Adversarial Risk Analysis and Critical Infrastructure at the Lorentz Center in Leiden. David Banks, a statistics professor at Duke and one of the workshop organizers, proposed* three levels for modeling that applies to research in statistics, operations research, and optimization:
- LEVEL 1: You solve the problem.
- LEVEL 2: You solve the problem in a cost-effective manner (e.g., using heuristics to get a quick solution that is “good enough”)
- LEVEL 3: You solve the problem in a cost-effective manner that a decision-maker will implement
The other conference attendees (mostly academics) seemed to like these three levels as much as I did. Academics like David and I spend most of our time on Level 1 even though we recognize that the point of doing all this academic work is to inform decision-making. We try to reduce the gap between Level 1 and Level 3 through our work and we occasionally make that leap. We have clearly seen that happen in optimization modeling, where academic work on cutting planes (once Level 1) first developed decades ago is now standard in off-the-shelf optimization software (definitely Level 2 without a loss in quality and maybe Level 3). It might take a few years to get to Level 3, but that is how progress and innovation work.
But that is not why David brought this up. He mentioned informing decisions during a discussion about homeland security and terrorism, and making the jump from Level 2 to Level 3 is tough because the decision-makers–who are often politicians–are not often receptive to math modeling. That’s not to say we should give up, but rather, we should sometimes start with Level 3 as the goal and work backward by rethinking what the problem is and how we solve it.
What do you think?
* David said he “made up” these levels last week.
Punk Rock Operations Research 
May 23rd, 2016 at 6:56 pm
hi there! i work for a data science unit for the singapore government, so have a little experience in working with decision-makers.
it does seem that decision-makers are usually neither interested in nor have the patience to understand the tools that academics develop. Rather, they often want to know (i) do you understand the problem I face, and (ii) what benefits can your solution bring (e.g. $X in savings, X man-hours saved). Only then are they ready to listen to the details (just the broad overview, not the gory technical stuff).
on the other hand, results alone may not be sufficient to convince a decision-maker to implement. decision-makers can be very uncomfortable with “black boxes”: why does the model work and why should i trust it? to get around this, it helps to:
– show that your model is consistent with obvious cases
– show how your model delivers surprising insight in some cases that are not as clear-cut, with some sort of explanation that makes it plausible (“why didn’t i think of this before?”)
May 24th, 2016 at 1:45 am
The late Gene Woolsey either originated or popularised the maxim: “A manager would rather live with a problem he cannot solve than accept a solution he cannot understand.”
May 25th, 2016 at 11:41 am
[…] post by Laura McLay mentions three levels for decision-making […]
May 30th, 2016 at 8:14 am
moving t level 3 is indeed the hardest. This is well recognized by practitioners in industry. They developed interesting strategies such as using a game where decision makers compete against the optimization algorithm. A team won the Franz Edelmann Award for that: https://www.ibm.com/developerworks/community/blogs/jfp/entry/should_or_can_that_is_the_prescriptive_analytics_question?lang=en