Tag Archives: decision analysis

risk analysis and extreme uncertainty in Beijing

I attended the International Conference on Risk Analysis, Decision Analysis, and Security at Tsinghua University in Beijing on July 21-23, 2017.
The conference was organized by Mavis Chen and Jun Zhuang in honor of my UW-Madison Industrial and Systems Engineering colleague Vicki Bier. The conference was attended by Vicki Bier’s collaborators and former students.

I enjoyed listening to Vicki’s keynote talk about her career in risk analysis and extreme uncertainty. Vicki talked about drawing conclusions with a sample size of one (or fewer!). In her career, Vicki has studied a variety of applications in low-probability, high consequence events such as nuclear power and security, terrorism, natural disasters, and pandemic preparedness. She stressed the importance of math modeling in applications in which the phenomenon you are studying hasn’t happened yet. In fact, you never want these phenomena to happen. Vicki told us, “I am a decision analyst by religion,” meaning that decision analysis is the lens through which she views the world and how she first starts thinking about problems, always recognizing that other methods may in the end provide the right tools for the problem.

Vicki has collaborated with many people of the years, and she shared several stories about her collaborations with her former students. I enjoyed hearing the stories about how her students challenged her and pushed her research in new directions. For example, Vicki told us, “Jun Zhuang made me confront my lifelong distaste for dynamic programming.” Vicki ended her keynote outlining her future work, noting that is not yet ready to retire.

Several conference attendees took a field trip to the Great Wall of China and to visit the tomb of the thirteenth emperor of the Ming dynasty, the only tomb that has been excavated underground, and the Ming dynasty’s summer palace. Many thanks to Mavis Chen and Jun Zhuang for their outstanding organization of the conference!

Pictures from the conference and its side trips are below.

At the #greatwall outside of #Beijing #china

A post shared by Laura Albert (@punkrockanalytics) on

At #Tsinghua university in #beijing #China

A post shared by Laura Albert (@punkrockanalytics) on

Outside of #Tiananmen Square in #Beijing #China from Madison #OnWisconsin

A post shared by Laura Albert (@punkrockanalytics) on

More #gorgeous #lotus flowers in #Tsinghua university in #Beijing #China

A post shared by Laura Albert (@punkrockanalytics) on

Among the #lotus flowers in #Tsinghua university in #Beijing #China

A post shared by Laura Albert (@punkrockanalytics) on

At the #SummerPalace in #beijing #China enjoying #lotus flowers 🌺

A post shared by Laura Albert (@punkrockanalytics) on

#peace signs at #Tiananmen #Square in #Beijing #China

A post shared by Laura Albert (@punkrockanalytics) on

Outside the 13th #ming #dynasty #tomb in #beijing #china

A post shared by Laura Albert (@punkrockanalytics) on

I ate a #popsicle made out of #greenbeans in #Beijing #China and it was #delicious

A post shared by Laura Albert (@punkrockanalytics) on

Advertisements

a multiobjective decision analysis model to find the best restaurant in Richmond

I taught multiobjective decision analysis (MODA) this semester. It is a lot of fun to teach. I always learn a lot when I teach it. One of the most enjoyable parts of the class (for me at least!) is to run a class project that we chip away at during class over the course of the semester. Our project is to find the best restaurant for us to celebrate at the end of the semester. “Best” here is relative to the people in the class and the .

The project is a great way to teach about the MODA process. The process not only includes the modeling, but also the craft of working with decision makers and iteratively improving the model. It’s useful for students to be exposed to the entire analysis process. I don’t do this in my other classes.

On the first day of class, we came up with our objectives hierarchy. I did this by passing out about five Post It notes to each student. They each wrote one criteria for selecting a restaurant on each Post It note. They stuck their Post It notes to the wall. Together, we regrouped and organized our criteria into an objectives hierarchy.  Some of the objectives because “weed out criteria,” such as making sure that the restaurant could accommodate all of us and comply with dietary restrictions.

Our initial criteria were:

  1. Distance
  2. Quality of food
  3. Variety of food
  4. Service: Fast service
  5. Service: Waiting time for a table
  6. Service: Friendly service
  7. Atmosphere: Noise level
  8. Atmosphere: Cleanliness
  9. Cost

Our final criteria were as follows (from most to least important):

  1. Quality of food
  2. Cost (tie with #3)
  3. Distance
  4. Fast service (tie with #5)
  5. Noise level
  6. Cleanliness

We removed variety of food, waiting time, and friendly service because classroom discussions indicated that they weren’t important compared to the other criteria. Variety, for example, was less important if we were eating delicious food at an ethnic restaurant that had less “variety” (variety in quotes here, because it depends on you you measure it).

In the next few weeks, we worked on identifying how we would actually measure our criteria. Then, we came up with a list of our favorite restaurants. During this process, we removed objectives that no longer made sense.

We collaboratively scored each of the restaurants in each of the six categories by using a google docs spreadsheet.

  1. Quality of food = average score (1-5 scale)
  2. Cost (tie with #3) = cost of an entree, drink, tax, and tip
  3. Distance = distance from the class (in minutes walk/drive)
  4. Fast service (tie with #5) = three point scale based on fast service, OK service, or very slow service
  5. Noise level = four point scale based on yelp.com ratings
  6. Cleanliness: based on the last inspection. Score = # minor violations + 4*# major violations.

A real challenge was to come up with:

  • the single dimensional value functions that translated each restaurant score for an objective into a value between 0 and 1.
  • the weights that balanced our preferences across objectives using swing weight thinking. FYI, we used an additive model.

I won’t elaborate on these parts of the process further. Ask me about these if you are interested.

When we finished our model, the “best” decision was to forego a restaurant and do a potluck instead. No one was happy with this. We examined why this happened. This was great: ending up with a bad solution was a great opportunity for learning. We concluded that we didn’t account for the hidden costs associated with a potluck. Namely, it would entail either making a trip to the grocery store or cooking, approximately a 30 minute penalty. We decided that this was equivalent to driving to a distant restaurant, a 26 minute drive in our model.  It was also hard to evaluate cleanliness since the state do not inspect classrooms like they do restaurants. But since cleanliness didn’t account for much of our decision, we decided not to make adjustments there.

The final model is in a google docs spreadsheet.

We performed a sensitivity analysis on all of the weights. Regardless of what they were, most of the restaurants were dominated, meaning that they would not be optimal no matter what the weights were. The sensitivity was not in google docs, since we downloaded the document and performed sensitivity on our own. I show the sensitivity wrt to the weight for quality below. The base weight for quality is 0.3617. When the weight is zero and quality is not important, Chipotle would have been our most preferred restaurant. The Local would be preferred only across a tiny range.

We celebrated in Ipanema, a semi-vegetarian restaurant in Richmond. I think our model came up with a great restaurant. We all enjoyed a nice meal together. Interestingly, Mamma Zu scored almost identically to Ipanema (see the figure below).

I cannot claim credit for this fun class project. I shamelessly stole this idea from Dr. Don Buckshaw, who uses it in MODA short courses.  We use the Craig Kirkwood’s Strategic Decision Making as the textbook for the course. I also recommend Ralph Keeney’s Value Focused Thinking and John Hammond’s Smart Choices.

How do you choose a restaurant?

Sensitivity with respect to the weight for quality (0.3617 in the base case).

are squirrels optimizers or satisficers?

Last month, I had the pleasure of meeting Yakov Ben-Haim and talking with him at length about info-gap decision theory. He used an example of squirrels foraging for nuts to illustrate the types of problems for which info-gap decision theory models are useful.

A squirrel needs calories to survive, and nuts provide the perfect source of calories. The squirrel has a decision to make: where should the squirrel go to forage for nuts? Different foraging locations have different potentials for nut payoffs. They also have risks (not enough food). Foraging in a new location may carry highly uncertain risks that are impossible for the squirrel to estimate (being hit by a car, eaten by a wolf, etc.)

The squirrel has two options: the squirrel can hunt in the usual area where he can obtain n nuts with certainty or he can try a new location where he has a probability P of obtaining N nuts (with N > n) and a probability (1-P) of obtaining zero nuts. Let’s say that N and P are wild guesses.

Let’s say that the squirrel is an optimizer and decides to build a decision tree to maximize the number of nuts he can collect. Using basic decision analysis, he devices that he should choose the new location if PN>n.

The squirrel's decision tree. Squirrels don't really make decision trees, do they?

If the squirrel needs to collect n nuts to survive, then maximizing is nuts (pun intended. Sorry!) Staying with the status quo guarantees survival, even if P and N are large. The payoff for the new location may be greater, but there is a 1-P chance that the squirrel would starve.  The traditional decision tree is not robust to the squirrel’s desire to survive (neither is darting in front of cars on the highway, but I digress).

On the other hand, if the squirrel needs to collect N nuts to survive, then staying with the status quo guarantees the squirrel’s demise.  The new location is worth a look no matter how risky.

In both of these scenarios, the squirrel isn’t really maximizing the subjective expected nuts that he can collect–he really wants to maximize the probability of meeting his nut threshold (the one that guarantees survival). This is a satisficing strategy (although not dissimilar from an optimizing strategy with a moving threshold). The satisficing strategy is a better bet for the squirrel than the optimization strategy in this decision context. The squirrel doesn’t always need to know the exact probabilistic information to make a good decision, as illustrated above.  In fact, he can have absolutely no idea what N and P would be to find an effective nut foraging strategy–even when there is severe uncertainty.

The idea of a squirrel building a decision tree is, of course, ludicrous. But it makes the point that what we should rethink our traditional optimization models so make sure they fit the real decision criteria on hand.  Info-gap decision theory thus focuses on satisfying a given acceptable level of what is traditionally considered the objective function value and instead optimizing robustness.  It also has philosophical implications for how one views certainty.

I’ve been looking more closely at robustness lately.  I won’t abandon my optimization models, but I will acknowledge that including robustness in certain scenarios leads to decisions that more accurately reflect the criteria at hand and decisions that could be counter-intuitive.

Yakov Ben-Haim can explain this much better than I can, so I’ll refer you to his blog about info-gap decision theory and his article about foragers in the American Naturalist if you want to learn more.