Last week, I had the pleasure of attending the Annual Natural Hazards Research and Applications Workshop in Colorado. This workshop is an interdiscplinary meeting attended by academics and practitioners who are interested in natural disasters and catastrophes. If you’re interested in humanitarian logistics, I recommend that you attend this workshop at least once.
Although the workshop is interdisciplinary, it is dominated by social scientists. Many of the discussions, particularly those about resilience, are qualitative. During one panel session, the speakers and attendees grappled with the issue of how to improve resilience when it is virtually impossible to measure.
The perspectives of social scientists are critical for building meaningful operations research models of natural disasters. Having said that, I found myself craving numbers and quantitative numbers after a few sessions. Couldn’t we talk about something that you can measure at least once in a ninety minute session? I am not criticizing the workshop sessions or the social scientists here. I am commenting about my innate need for measuring things. It’s so innate that I involuntarily started to come up with ways to quantify everything qualitative mentioned in the sessions. I realize that there are situations when it’s not good for people like me to jump in and come up with some metrics, simplifying a complex problem. I try to stay away from those situations because, darn it, I like numbers. This is why I do things such as come up with a metric for determining which breakfast cereals to buy. I am happy to acquiesce to the social scientists the rest of the time.
As operations researchers, we know that the objective function does not capture all of the real objectives. We know that the model is an abstraction of reality, not reality itself. Modeling do not need to perfectly reflect reality, they need to be useful and actionable. Interfaces, the operations research journal that highlights the benefits of using operations research in practice, asks authors to “List the resulting benefits [of implementing their model], both quantitative and qualitative” as a way to recognize the fact that we cannot–and should not–measure all of the benefits. I think we strike a good balance.
I like measuring things. I like doing operations research. I would not be happy being a social scientist, but I’m glad they are around.
What do you insist on measuring and modeling?
Punk Rock Operations Research 
July 22nd, 2011 at 12:17 pm
I track my gas mileage. Partly it’s to make sure I’m not getting loose in my driving habits, and partly to make sure my engine isn’t getting funky. Mostly, though, it’s because it is a somewhat pleasing example of a “naturally occurring” sinusoid.
July 29th, 2011 at 5:52 pm
Great post! I am in the unenviable (and challenging!) position of having to measure inherently qualitative outcomes for my organization. I am responsible for measuring (or demonstrating value in) the USCG’s operational performance across our entire mission set. To touch on a few, our diverse actions include saving lives (relatively easy to measure), stemming flow of illegal drugs (somewhat challenging), responding to oil spills or disasters (a little more challenging), ensuring a safe waterway (tough), and deterring terrorist activities (really tough). Part of the challenge is data–you measure what you can vice what you want or should. Part of the challenge is balancing theory with reality. I, like you, crave to measure things but the measurements must be meaningful. At my level budgets and resources depend upon our ability to quantify to qualitative.
And, yes, resiliency to disasters is part of my world and I would be curious to see your list of metrics you came up with. I’d especially be interested in how you (or anyone out there) would tackle measuring any of the other aforementioned activities.