If a mathematical model is solved in a forest and a decision-maker is not around to see it, does it have any impact?

I am attending a workshop on Adversarial Risk Analysis and Critical Infrastructure at the Lorentz Center in Leiden. David Banks, a statistics professor at Duke and one of the workshop organizers, proposed* three levels for modeling that applies to research in statistics, operations research, and optimization:

  • LEVEL 1: You solve the problem.
  • LEVEL 2: You solve the problem in a cost-effective manner (e.g., using heuristics to get a quick solution that is “good enough”)
  • LEVEL 3: You solve the problem in a cost-effective manner that a decision-maker will implement

The other conference attendees (mostly academics) seemed to like these three levels as much as I did. Academics like David and I spend most of our time on Level 1 even though we recognize that the point of doing all this academic work is to inform decision-making. We try to reduce the gap between Level 1 and Level 3 through our work and we occasionally make that leap. We have clearly seen that happen in optimization modeling, where academic work on cutting planes (once Level 1) first developed decades ago is now standard in off-the-shelf optimization software (definitely Level 2 without a loss in quality and maybe Level 3). It might take a few years to get to Level 3, but that is how progress and innovation work.

But that is not why David brought this up. He mentioned informing decisions during a discussion about homeland security and terrorism, and making the jump from Level 2 to Level 3 is tough because the decision-makers–who are often politicians–are not often receptive to math modeling. That’s not to say we should give up, but rather, we should sometimes start with Level 3 as the goal and work backward by rethinking what the problem is and how we solve it.

What do you think?

* David said he “made up” these levels last week.

4 responses to “If a mathematical model is solved in a forest and a decision-maker is not around to see it, does it have any impact?

  • kjytay

    hi there! i work for a data science unit for the singapore government, so have a little experience in working with decision-makers.

    it does seem that decision-makers are usually neither interested in nor have the patience to understand the tools that academics develop. Rather, they often want to know (i) do you understand the problem I face, and (ii) what benefits can your solution bring (e.g. $X in savings, X man-hours saved). Only then are they ready to listen to the details (just the broad overview, not the gory technical stuff).

    on the other hand, results alone may not be sufficient to convince a decision-maker to implement. decision-makers can be very uncomfortable with “black boxes”: why does the model work and why should i trust it? to get around this, it helps to:

    – show that your model is consistent with obvious cases
    – show how your model delivers surprising insight in some cases that are not as clear-cut, with some sort of explanation that makes it plausible (“why didn’t i think of this before?”)

  • David Smith

    The late Gene Woolsey either originated or popularised the maxim: “A manager would rather live with a problem he cannot solve than accept a solution he cannot understand.”

  • Three Levels of Problem Solving |

    […] post by Laura McLay mentions three levels for decision-making […]

  • jfpugetFPuget

    moving t level 3 is indeed the hardest. This is well recognized by practitioners in industry. They developed interesting strategies such as using a game where decision makers compete against the optimization algorithm. A team won the Franz Edelmann Award for that: https://www.ibm.com/developerworks/community/blogs/jfp/entry/should_or_can_that_is_the_prescriptive_analytics_question?lang=en

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: