Assumptions Behind the Assumptions in the War on Terror: 'Event-Specific Catastrophism', Cost-Benefit Analysis and Its Limitations

I have just posted a new article up to SSRN: The Assumptions Behind the Assumptions in the War on Terror: Risk Assessment as an Example of Foundational Disagreement in Counterterrorism Policy, 54 Wayne Law Review 5505-535 (2008).

Here is the abstract posted to SSRN:

This 2007 article (based around an invited conference talk at Wayne State in early 2007) addresses risk assessment and cost benefit analysis as mechanisms in counterterrorism policy. It argues that although policy is often best pursued by agreeing to set aside deep foundational differences, in order to obtain a strategic plan for an activity such as counterterrorism, foundational differences must be addressed in order that policy not merely devolve into a policy minimalism that is always and damagingly tactical, never strategic, in order to avoid domestic democratic political conflict.

The article takes risk assessment in counterterrorism, using cost benefit analysis, as an example of a foundational disagreement that cannot easily be elided. Examining an extreme, indeed crude, recent example of cost benefit analysis applied to the risks of terror and the costs of counterterrorism - John Mueller's widely noticed Overblown - the article suggests that cost benefit analysis, at least applied in this way, runs roughshod over other important values in counterterrorism policy, such as justice, but in addition, makes radical yet unstated assumptions about what cost benefit analysis seeks to compare in establishing counterterrorism policy or estimating the risks and costs of terrorism - unstated assumptions that, in fact, assume the conclusion.

The article notes that cost benefit analysis tends to promote a policy-minimalizing "event specific catastrophism" - seeking above all to prevent simply the next, serial terrorist attack, with however no greater strategic vision. Indeed, the article says in conclusion (as Philip Bobbitt has noted) cost benefit analysis is "relentlessly tactical," not strategic; it also tends toward serial 'event specific catastrophism' as its analytic frame; and it is a method of evaluating proposed courses of action, not generating them, and hence promotes a strategically questionable tendency to reaction as a response to terrorism.

This article presents these ideas in brief fashion, however, as the first draft in a larger project on cost benefit analysis and counterterrorism, and it does so by reference to a book that is unabashedly crude in its approach to both cost benefit analysis and terrorism/counterterrorism. The critical project will extend beyond this particular article, which is in effective a a first pass at developing a critique. It is also an article that does not extend beyond events of early 2007 (when the original address was given) and should be read in that light.

I don’t think this is the smartest piece I’ve ever written, alas. I’m treating it as the first draft of a larger, but important, project on the limits of cost benefit analysis. Mueller is popular, in all the vulgar bad senses of the word, and in that sense an easy target. But the vulgarity of Mueller’s analysis has certain advantages, in that it makes every conceivable mistake, openly and notoriously. But I think much of the critique also applies even to much more cautious formulations of CBA as the driver of policy in responding to terrorism. Here is the conclusion, by the way, slightly rewritten to stand alone, as something I shared with the Hoover Task Force last week:

Kenneth Anderson, The Assumptions Behind the Assumptions in the War on Terror: Risk Assessment as an Example of Foundational Disagreement in Counterterrorism Policy
(Wayne Law Review, 2008)

(Slightly modified conclusion from the not-final draft galleys.)

V. CONCLUSION: OF STRATEGY AND TACTICS

This essay has made five over-arching points.

First, US responses to terrorism—whether one calls it generically
counterterrorism policy or a war on terror or anything else—depend on
certain underlying assumptions, what this essay has called the
‘assumptions behind the assumptions’ in counterterrorism. Cost benefit
analysis is a core ‘assumption behind the assumptions’ lying below the
surface of operational counterterrorism policy.

Second, cost benefit analysis itself depends upon further
assumptions. These further assumptions have a large impact on the
otherwise apparently straight-forward comparative approach to weighing
up policy options in the face of risk and uncertainty. These further
assumptions embedded, but not necessarily transparent, within cost
benefit analysis include, among others, the difficulties in ensuring that
the analysis compares apples-to-apples or oranges-to-oranges. In a social
and political world of multiple and plural values, this is far more difficult
than it would be, for example, in the case of financial analysis in the
private marketplace, where, at least in principle, comparisons can be
reduced to the common denominator of money. Not all values in our
social world can be reduced to a common denominator.

Third, although cost benefit analysis can provide important data
for making moral judgments about such fraught matters as how to
respond to terrorism, it does not finish the moral discussion—at least not
for most people in American society. Beyond whatever advice cost
benefit analysis might give, most people are ‘permissive deontologists’
when it comes to matters of how to respond to purposive and intentional
actions such as murder and terrorism. For the same reason—justice—we
devote far greater resources to the pursuit of criminals than cost benefit
analysis might plausibly justify, we are also inclined to devote more
resources to responding to and preventing terrorism. Arguments from a
cost benefit analysis that suggest that we devote too many resources to
counterterrorism would also apply with equal force to the argument that
we allocate too many resources to the criminal justice system for the
pursuit of ordinary criminals.

Fourth, similar observations about the overreaching tendency of cost benefit
analysis, under an apparently simple exterior, can be made with respect
to ‘commensurability.’ As noted earlier in the essay, this is a point
closely related to, but still different from, the observation that we, as a
society, embrace plural values that are not reducible to one common
denominator. cost benefit analysis relies upon the comparison of
‘opportunity costs,’ but the comparison of opportunity costs depends
upon them being genuinely available ‘opportunities’—social choices that
might genuinely be made. Whether an opportunity is genuinely an
opportunity in our existing social world or not is a question of social fact
about the world. Arguments from cost benefit analysis that rely upon
opportunity cost comparisons involving socially or politically
implausible opportunities—opportunities from another, alternative
world, so to speak, not our real one—are of much less importance than
their conclusions might seem. Again, such arguments overreach.

This essay has focused on the writings of one particular analyst, John
Mueller, and his book Overblown, as an example of the deeply flawed
use of cost benefit analysis. It is a more than fair point to respond that
using Overblown as the case study in the ills of cost benefit analysis is
the worst kind of strawman argument. On the one hand, the book has
been widely noticed, cited, and relied upon for argument by important
journalists and policy analysts, such as journalist James Fallows.62 On
the other hand, serious academic students of cost benefit analysis would
recoil from the sweeping, breezy assertions and conclusions made by the
book, on all the objections raised above and perhaps more. The reason
for making it the target in this essay is not in order to suggest that it
stands in for much more serious cost benefit analysis. It does not.

But, Overblown is illustrative of the basic errors that can and might
arise from failing to take into account the underlying assumptions of cost
benefit analysis—and the illustration is far easier to see in a crude form
of cost benefit analysis, rather than a more careful and hedged version of
it. The point of this essay is not to undermine the case for cost benefit
analysis in responding to terrorism—far from it—but instead to help
define the subtle limits upon the method and the matters that must be
drawn out carefully and explicitly in order to ensure that comparisons are
indeed comparable, particularly with respect to counterterrorism policy.

It is therefore useful to start with a view, claimed on its own terms to be
generated by cost benefit analysis, that America’s approach to terrorism,
far from trying to wipe out its perpetrators or even devote much in the
way of resources to prevent it from taking place, might instead merely
“center around creating the potential to absorb its direct effects,” and
“mitigate its longer range consequences.”63 The very boldness of the
claim, and the fact that the claim reached such radical conclusions
through the application of cost benefit analysis, puts squarely on the table
what the method can do, cannot do, and what assumptions it relies upon.
There are sophisticated and defensible applications of the method to
counterterrorism,64 but as a starting point for considering the method’s
assumptions, sometimes the crudest, least methodologically protected
example provides the most illustrative value (provided that the critic
understands that the baby of cost benefit analysis cannot, therefore, be
thrown out with the bathwater, as it were).

Fifth, one final observation can be made about cost benefit analysis and its
underlying assumptions. It is an observation particularly pertinent to its
application to terrorism and counterterrorism. The nature of cost benefit
analysis is essentially reactive. It is—and this point deserves an essay all
its own with respect to national security and terrorism policy—a method
of evaluation, a mechanism for evaluating proposed courses of action,
not for generating them.65 As a method, it is, in Philip Bobbitt’s phrase,
“relentlessly tactical.”66 Cost benefit analysis does not propose solutions;
it evaluates solutions offered by other processes. It is not a strategic form
of thinking.

The fundamental limitation of cost benefit analysis, in other words,
lies not so much in its own assumptions, but in the limits of what it does and does not do.
The long-term US response to terrorism—counterterrorism policy, the
war on terror, however one wants to frame it—requires a strategic form
of thinking. We will not agree on what the strategy should be, which is
why, as a democracy, we have majoritarian processes to sort out the
agreements and disagreements, and come to a form of action. Cost
benefit analysis can provide indispensable information for arguing over,
and finally formulating, strategic approaches. But it will not come up
with those strategies in the first place. And that, in the end, is its true
limitation.