Tweenbots

I, for one, welcome our Tweenbot overlords! 

CTLab symposium on PW Singer Wired for War

Complex Terrain Laboratory is hosting an online discussion next week on PW Singer’s new book on robotics and war, Wired for War (starting Monday, March 30).  Singer is participating and, having read his opening post, it looks to be fascinating.  It is a terrific lineup of participants, including yours truly. Check it out!

Why targeted killing? And why is robotics so crucial an issue in targeted killing?

(Given the recent Obama administration review of the Predator campaign in Pakistan, I thought it would be a useful thing to put this discussion on the table.  Welcome Instapunditeers, and thanks Glenn for the Instalanche!  You might also want to check out the ComplexTerrainLab's discussion among academics of PW Singer's Wired for War, here.)

Why targeted killing? And why the insistence that it will increase in utility as it is partnered with high-technology, stand-off platforms such as Predator drone aircraft? Why the emphasis on targeted killings and robotics? There is a fundamental strategic rationale lying behind the policy trend.

The United States has found the limits of how extensively it can wage full-scale wars with its military; even if wanted to take on more wars, it has logistical and political limits. In addition, the United States has discovered that full-on war is useful principally against regimes. Full scale, large scale war of the kind waged in Afghanistan and Iraq is useful primarily for bringing down a regime that, for example, might harbor or support terrorists, or which might be believed to be willing to supply terrorists with materials for weapons of mass destruction (WMD). Full-scale war has a crucial strategic place in national counterterrorism policy, but by its nature that role is about states and regimes fundamentally.

Large-scale military operations are less useful directly against transnational terrorists, however, who are few in number, dispersed across populations and often borders, disinclined to fight direct battles, and more efficiently targeted through narrower means. The fundamental role of war in counterterrorism is to eliminate the regimes that provide safe haven to terrorist groups; terrorist groups can be strategically understood as an extreme version of a guerrilla organization engaged in a strategy of logistical raiding – in which civilian morale and resulting manipulation of political will is the logistical target. Logistical raiders typically need a safe base to which to retreat, and full-scale war is most useful in eliminating such safe bases and convincing other regimes not to provide them. But it is not usually an efficient way of going directly after transnational terrorist groups themselves.

Law enforcement utilized outside the United States, on the other hand, has also discovered its outer limits. Many debates are still to be had over the rights of alleged terrorists once in U.S. custody. Even so, whatever they are, few would argue that going out to ‘arrest’ terrorists in, for example, Pakistan’s tribal zones is a winning policy or a serious option. The same is true in Somalia and other places, and it will be true in other places in the world in the future.

Moreover, the political costs for any U.S. administration taking and holding detainees are now enormous. Once you hold them, over time they will likely be accorded quasi-Constitutional protections by the courts, at least some version of habeas corpus. Politically, the most powerful institutional incentive today is to kill rather than capture them. The intelligence losses of killing rather than capturing in order to interrogate them are great. But since the U.S. political and legal situation has made interrogation a questionable activity anyway, there is little reason to seek to capture rather than kill. And if one intends to kill, the incentive is to do so from a standoff position, because it removes messy questions of surrender.

All this speaks to the advantages to the U.S. government of targeted killing of terrorists or persons seriously believed to be terrorists, and it also speaks to the advantages to the US government from using stand-off robotics technology to perform these attacks. But the humanitarian advantages of ‘targeted’ killing are enormously important as well, and ought to be on the table. This is particularly so given that targeted killing has come in for a barrage of criticism, legal and ethical, much of which seems motivated by the fact that it can be more discriminate than full scale military assault; the fear seems to be that it makes violence too easy to undertake. The same criticism is offered of the evolution of robotic technology that increasingly allows targeted uses of force without having to risk one’s own personnel. Not using one’s own personnel allows a party to attack without the fear of counterassault that might increase the need to use greater amounts of force and cause greater collateral damage – but it also, so it is sometimes argued, thereby reduces the inhibitions on the decision to use force.

Why this should be a downside for US strategic counterterrorism policy is not entirely evident, but clearly some critics are disturbed by it. Much of the criticism amounts to a very contemporary restatement, aimed against the targeted killing that evolving robotic and surveillance technology might permit, of a very old argument against the idea itself of the introduction of humanitarian standards in conflict (one that stretches back at least to the founding of the International Committee of the Red Cross (ICRC)), that humanitarian standards by their promulgation would reduce the disincentives to war.

Whatever the critics say, however, is unlikely to sway US strategic policy, under the Obama administration or anyone else subsequently. The humanitarian benefits of precision targeting are far more obvious than the more remote and abstract suppositions of their humanitarian costs. Their direct policy consequence is to introduce greater discrimination in targeting than full-scale military assault and large-scale war permit, through targeted killing using high technology. There is a clear humanitarian advantage favoring the use of targeted killing over full-scale war. Advancing technology allows for more discrete surveillance and therefore more precise targeting that is finally better able to minimize collateral civilian damage.

The result is a strategic incentive for targeted killing, for Predator strikes, and for increasing the quality of technology to make targeted killings both more precision targeted and more standoff. Precision targeting and standoff delivery are each independently desirable and, in combination, considerably increase the incentive. The Obama policy team did not quite run on a policy of targeted killing – but it did run on a policy of taking the fight to Al-Qaeda in Pakistan in a targeted way.
The Obama administration is right about this – right about the logic that presses toward targeting standoff killing as a necessary and available and technologically advancing part of counterterrorism. It is also right about it as a moral and humanitarian proposition in the law and policy of the use of force. It is a conclusion that is correct as well as for foreseeable future administrations, even if administrations naturally prefer to couch it in softer terms.

None of this alters the equally impeccable strategic logic underlying the use of law enforcement mechanisms in some circumstances. Nor does it alter the logic behind other forms of intelligence activities such as surveillance or financial interdiction, or even the use of open, full-on war. The strategic logic for toppling a regime in pursuit of counterterrorism during the next ten or twelve years can by no means be ruled out. But these are not disjunctive policies. They all can and should work together. But targeted killing is likely to increase as a policy preference as full-scale wars decreases in number and intensity.

The paradox, however, is that although the strategic logic for targeted killing increases in persuasiveness, the legal space for it and the legal rationales on which it has been traditionally justified are shrinking. It has been shrinking in ways that might surprise members of Congress and the Obama administration. And it is at risk of shrinking still further through seemingly innocuous, unrelated legal policy actions that the Obama administration or Congress might be inclined to take in support of various political constituencies, usually related to broadly admirable goals of human rights and international law.

U.S. law, in domestic law since the original Cold War legislation establishing the CIA in 1947 at least, and in the US view of international law, accepts a legal, political, and policy space for the use of violence by political decision not in the course of large scale, open armed conflict governed by international humanitarian law, and not in the course of judicially supervised law enforcement operations, either. ‘Violence by political decision’, in peacetime outside of open armed conflict under international humanitarian law, was a space of activity accepted and considered vital to national security throughout the long decades of the Cold War. Only in certain narrow times and places was the decades-long conflict with the Soviet Union and its allies a ‘hot’ war, open and large-scale armed conflict, clashes of armies. Political violence in the Cold War was often covert, often denied, but it was authorized and endorsed by US domestic law, although it was frequently a violation of the law of states where such activity took place and unsurprisingly was sometimes, too, a source of grave diplomatic and other friction.

This category of force is an obvious means by which to confront non-state transnational terrorists outside the territorial United States. It is especially true outside the territory of states where effective mechanisms exist for arrest, detention, investigation, trial, and punishment, or alternatively extradition, of suspected terrorists. Regimes that have allied themselves to terrorist organizations – the Taliban in Afghanistan – might be toppled. Failed states might require large-scale military action in order to block the use of territory as a safe haven by terrorist groups. But as a strategic matter, actual attack on a physically small number of terrorists embedded among civilians is often best served by attacks made as physically precise and discrete as surveillance and targeting technology allow.

Comments on Bosquet and Clausewitz at Complex Terrain Lab

I’m guest blogging at Complex Terrain Lab over the weekend in a symposium on Antoine Bosquet’s The Scientific Way of War. I just put up something about Clausewitz and friction, and noting the relationship to LIncoln and the Second Inaugural Address. It’s a terrifically fun symposium, check it out. Here’s what I posted:

My congratulations to Dr. Bousquet for a highly insightful and readable book that engaged me with a historically shifting body of metaphors for war and conflict. I want to focus briefly on one subchapter, the end of chapter three, devoted to Clausewitz and the metaphors of thermodynamics. I was much taken, and am generally strongly in agreement with, both your intellectual history of Clausewitz's thought and its rootedness in a certain scientific world view as well as your reading of the famous "friction" metaphor. I wonder if the the friction metaphor might not be broken out still further, in four ways.

First, the famous Clausewitzian undermining of the army-as-clockwork mechanism, undermined by the friction of the clash of two armies. Second, the concept of friction as expressed in the technologies of thermodynamic weapons and war - explosions and counter-explosions. Third, Clausewitz's also famous dictum of friction as created by the accumulation of errors in the system of war, located in failures of communication, delivery, and execution that accumulate, again to undermine the army-as-clock from within.

Fourth, what I suppose we might call the friction created by the 'ghost in the machine of war': friction that arises not from a clash of two armies, nor from errors internally accumulating, nor from explosions meeting explosions, but instead from the clash of two fundamentally different conceptions of conflict, the inherent clash between, on the one hand, the mechanism that enables a vast array of people and things to act with a single will, deterministic and mechanistic and, on the other, the animal passions that are both unleashed but relied upon particularly in battle. It is not precisely that the mechanism tames the beast; it is, rather, a dialectic in which the machine needs the animal spirits and the animal spirits need the discipline of the machine.

Anyway, for what it's worth, it seems to me that those are all separable as readings of 'friction' in Clausewitz and beyond, and that some parts of those distinctions are picked up later in your discussion.

But I was also struck by your reading of Clausewitz in this way - correctly focused on friction as the central concept rather than the 'other means' trope - in part because of work currently on my own desk on a reading of the moral psychology of Lincoln's Second Inaugural. One or two phrases in that address express a profoundly Clausewitzian sentiment: "Neither party expected for the war the magnitude or the duration which it has already attained. Neither anticipated that the cause of the conflict might cease with or even before the conflict itself should cease. Each looked for an easier triumph , and a result less fundamental and astounding."

This passage is sometimes read as a sort of Clausewitzian sentiment about war as politics by other means. I have not thought that the best reading, and your chapter in this regard gives me stronger reasons for thinking so: it seems more emphatically, on the contrary, an expression that war, as its own social life, and driven by its own frictional forces, frictional forces that are internal to war itself, can independently lead politics rather than necessarily the other way around.

Blogging on battlefield robots at Opinio Juris

I put up a post introducing the topic of battlefield robots over at Opinio Juris today. And Glenn Reynolds gave it an Instalanche - thanks Glenn!

Robot war spiders

My research assistant Marc found this mock-up of a robotic intelligence gathering spider.  (Not yet real.)

British defence giant BAE Systems is creating a series of tiny
electronic spiders, insects and snakes that could become the eyes and
ears of soldiers on the battlefield, helping to save thousands of
lives.

Prototypes could be on the front line by the end of the year,
scuttling into potential danger areas such as booby-trapped buildings
or enemy hideouts to relay images back to troops safely positioned
nearby.

http://www.dailymail.co.uk/pages/live/articles/technology/technology.html?in_article_id=563786&in_page_id=1965

Armed robot photo

Borrowing from the Danger Room article, here is a photo of a SWORDS with machine gun atop. You can watch a YouTube video of it, here: http://www.youtube.com/watch?v=e26Gnc3E0yg.

Armed robots deployed in Iraq

(Welcome, Instapunditeers, and thanks Glenn for the link!)

I blogged earlier, here, about important efforts to conceptualize the ethics of robot soldiers. For the first time, armed recon robots have been deployed in a warzone - Iraq. HT thanks Instapundit; from Danger Room/Wired Blogs, here.

This is not really about the topic of my ethics of robot soldiers post. That post was about machines with the capability to act independently, independent of human control, and it pointed to very important discussions trying to anticipate how and what the ethical decision making of such independent robots should be. The machines being deployed now to Iraq are not that, and are a long, long way from that - the Iraq deployed machines are a new version of the already widely deployed SWORDS reconnaisance robot vehicle - new because the new machines have a weapon, a machine gun, added to it. They are remote controlled in real time by humans, not machines making programmed decisions about weapons use. (Also, if you are thinking of robots in the I, Robot mode - well, these are more like mobile sleds with a machine gun on top - they don't anthropomorphize.) The ethical and legal questions raised are not about independent machine decisionmaking and action, but instead about the use of a human controlled and operated but remote platform. Still, this is a step closer to what seems to me not just a natural, but an inevitable step forward in warfare for the world's most highly technological army.

Robots as a response to asymmetric, law of war violating warfare:

It is important to understand that the inevitable move toward robots on the battlefield is not merely driven, as in past times, past conflicts and wars, by material considerations of conservation of personnel, force to space ratios, and so on. It is driven as much or more today by moral, legal, and ideological considerations - part of an effort to limit the exposure of one's soldiers when dealing with enemies who will not follow the laws of war with respect to our soldiers. Part of that is obviously the attempt to not get your soldiers killed - but another important of it is to avoid having your soldiers captured by an enemy that does not pay attention to the laws of war except when, by loudly appealing to it, it can benefit from it.

The US, for good moral reasons, has given up the possibility of reprisals against civilians or other people hors de combat, such as captured enemy fighters. It has also shown itself unwilling, for not such good reasons, however, to enforce certain important remaining laws of war with regards to abuses by the enemy (such as the US refusing, in its internal rules of engagement, to fire on a mosque being used as an enemy emplacement, despite being allowed to do so under the laws of war). The US therefore finds that it has few or no behavioral levers with respect to the behavior of an enemy fighting using illegal methods. In such a case, one response is the attempt to compensate through technology - by limiting the exposure of one's soldiers in particular to death, injury, or capture and replacing them with machines.

Will robot soldiers eventually lead to a more "policing" attitude on the battlefield? Might roboticized war be a factor leading, perhaps inadvertently, to fewer decisive engagements and more protracted warfare?

One question we might have is what happens over time if fewer American soldiers were to appear at all on asymmetric battlefields, and when they did and fell into hostile hands, we gradually came to assume, on the basis of experience, that they would be held hostage under terms hardly meeting the Geneva Conventions or else beheaded on internet video. It is unlikely that we would respond by war without quarter of our own. On the contrary, part of our technological drive to create and deploy remote fighting machines is in order to get away from having to enforce a barbarous reciprocity that has always been thought otherwise necessary (the ICRC and HRW and the ICC and all the rest of the modern day "heralds" of war notwithstanding) in order to deter such actions by the other side and so ensure adherence to the laws of war.

We might conceivably move, in such circumstances, to treat those we captured more as criminal detainees than as something closer to POWs - and to reconceptualize, over the long term, the general categories of detainees in asymmetric. We would after all not incline to treat them like POWs because we would have already long since determined that their behavior was that of an unprivileged belligerent. The category of actual legal POW might even conceivably wither (away?) from disuse. We would assume our people would be abused and/or killed, or else held as hostages or for ransom - much as Israel's soldiers held by Hizbollah, for example. It wouldn't make sense to us to treat unprivileged belligerents as POWs, especially given that our people - who would indeed be entitled to such treatment - would not be so treated. I would guess that we would evolve to treat them as some form of quasi-criminal detainee - I say 'quasi' because we would not be be able typically to prove criminality except on the basis of participation in an armed enterprise that as an enterprise systematically violated the laws of war, and often not on an individual basis. And 'quasi' also because it would likely have important elements of administrative preventive detention. Of course, we face exactly such issues now, but we have not really resolved them; the widespread deployment of armed robots on the battlefield, however, might constitute one pressure in that direction.

Curiously, however, it wouldn't surprise me, on the current evolution of things, if "battle" turned gradually into some form of particularly violent and contested attempt at "arrest" after a demand for surrender. Warmaking might evolve, at least in the asymmetric urban setting, to battle as a form of "policing." As soldiers were less physically present on the actual battlefield, and if you had armed machines dominating the battlefield, manned remotely, at least on one side, might there be greater pressure on your military to call for the fighters on the other side to surrender, for example - issuing a call to surrender, rather than simply attacking or undertaking ambush or surprise? Whether that would facilitate winning a conflict, as opposed to merely managing it over the long term is not clear. It might inadvertently create conditions for systematically less decisive engagements - tactical engagements with possibly less collateral damage, but also no victory - which is, of course, the definition of victory for guerrillas in a guerrilla struggle, never win, but also never lose and finally just outlast the enemy. Whether we would care, if such long-term "managed," never-decisive warfare cost us in treasure, but not especially in blood, is also not clear.

Can robot technology overcome behavioral shifts toward illegal warfare by irregular forces?

The development of remote and robot technologies is driven by a parallel consideration that also arises from moral, legal, and ideological consideration. It is the attempt to create machines that will follow determinate legal rules of engagement, particularly with respect to the combatant-noncombatant distinction - in consideration of an enemy, however, that deliberately violates that distinction in its own combat operations. Again, the effort is to find a way to overcome the inability through our battlefield behavior (such as the reprisals we deliberately and properly don't take) to affect deliberately planned, illegal enemy behavior - through a technological fix.

The move to robots is all but inevitable and, in fact, particularly but not just under these circumstances, desirable. I have my doubts, however, that any technological fix can permanently compensate for behavior on the other side. If the nature of arms races is competitive - either a "counter" or a "defensive" move to respond to changes in the conduct of war - then we are in a peculiar historical moment in which one side attempts to respond with equipment changes to changes in behavior on the other side. Is it possible for technological ingenuity to beat out determined and evolving bad behavior? I don't know.

Legal liability and robots on the battlefield:

Those deploying armed robots to Iraq for use in the field, remotely controlled, had probably better be prepared for a much greater willingness on the part of the outside monitors, the human rights organizations, outside critics, etc., to charge illegality, criminal behavior, war crimes, violations of the law of war, etc., in any collateral damage created by these weapons than currently exists - with charges and accusations against operators as well as commanders. And against the companies that design and build and sell such weapons.

Why more than in the case of soldiers present on the battlefield? Well, it doesn't necessarily make much sense - the rules of engagement, after all, are presumably exactly the same - but I would bet with pretty high confidence that the deep and not necessarily articulated premise will be that you are more liable for damage caused if you caused it remotely and were not yourself at risk, not being present on the battlefield, operating the robot remotely.

The idea that you yourself are in some fashion at risk - even if not very much, as in the case of a standoff aircraft or tank or what have you - on the battlefield, hence giving some compensatory justification to your collateral damage, makes a difference, or anyway will likely make a difference, I would bet, in how these weapons are seen by outside critics. It will seem weird to the military - it will seem to it as very close to claiming that remote operators have an "unfair" advantage and hence are entitled to no otherwise legal collateral damage - and it will not, to the military, seem any different from any other standoff platform such as aircraft or remote artillery. Why should it be?

But I would be willing to bet that it will seem quite different to outside monitors and critics. The two core criticisms will be: (a) you are not putting yourself at risk and hence are not entitled to collateral damage because, notwithstanding that the criterion of collateral damage is "military necessity," not "did I risk myself?" it will somehow seem "unfair" - despite the fact that you are battling an enemy for whom asymmetric warfare via violations of the laws of war is de rigeuer.

And (b), the fact that you risk only a robot but risk causing collateral risk in human life means that you should not do anything that risks collateral damage at all. Civilians and even civilian objects, in the lingo of Protocol I, trump any kind of claimed military necessity. This is especially so, it will likely be said, under the ICRC's interpretation of the language of Protocol I referring very narrowly to "concrete military advantage" in the immediate circumstances as the measure of military necessity. That the US has never accepted Protocol I as a treaty and has never accepted that particular interpretation of the customary law rule regarding military necessity - and that many other countries offered reservations and interpretations on that very point when they did join Protocol I is not likely to be seen by the critics as of any account.

If your definition is military advantage is sufficiently narrow, in other words, then no collateral damage is justifiable if all you risk is some equipment, not lives, on the battlefield - if your definition of military advantage is so narrow and immediate that it cannot include the necessity of winning this battle, or any particular battle, as part of a larger plan to win a war.

As I say, this will possibly seem puzzling and quite wrong to the military itself, which operates all kinds of remote platforms for launching weapons - and as armies have done, at least since the advent of the long range bow, the catapult, and artillery. But I would urge it to prepare for precisely such criticisms. I would guess this is how the public argument will go, and it might even culminate in someone or some organization calling for indictments against US soldiers for civilian deaths resulting from the use of remotely controlled robots in combat. Or civil law suits via the Alien Tort Statute against the companies creating this equipment.

Yet this would be disastrous if it led to the curtailment of these weapons, their development and deplyment - disastrous from the standpoint of the long term integrity of the laws of war in a period in which asymmetric warfare is tending to undermine their very foundations, because reciprocity has been largely lost - and disastrous to the effort to find ways through technology of combating an enemy that does not fight by the rules. Unfortunately, that has never been a concern of those who propose to make the rules of war, but do not have anything at stake in actually having to fight using them.

(Note on the first two comments. I emphatically do not think that the JAG and those formulating the US position on the laws of war would take the view that I have here attributed as being likely to come from outside critics in the human rights or perhaps academic communities. Or from countries that, not having any pressing wars to fight, are overly willing to opine on the content of laws in which they have no stake in the outcome. On the contrary, I think that the JAG and the US military laws of war lawyers would see this more or less as I suggest above: that these armed battlefield robots are remote platforms like any other, and that in any case military necessity is, at the end of the day, about winning wars. Military necessity does not justify anything and everything, of course, and it rules out many, many things; but it does not mean that a military has any obligation to risk itself or its personnel as a condition of being able to risk otherwise legal collateral damage. But I would be interested in comments from JAG, from current or past serving laws of war lawyers, and others interested in commenting.)

(Update, 9 August 2007, check out this link HT Instapundit from Popular Mechanics. Here.)

The ethics of robot soldiers?

Increased roboticization of US military operations is both inevitable and, on balance, a very good idea. (See this general article on robots on the battlefield.) Along with the things on the immediate horizon such as robot surveillance and remote sensing and all that has been the exploration, in the long term, of robot fighters. In that process, the more sci-fi inclined among us - that includes me - have been thinking about the issues of ethics and robot fighters, if they were made to include independent decisionmaking in at least some circumstances.

There have been some discussions in the academy, and some references to those in the press. The most easily accessible is this short piece in the Economist, "Robot Wars," June 7, 2007, here:

But whereas UAVs and their ground-based equivalents, such as the machinegun-toting Sword robots, are usually controlled by distant human operators, the Pentagon would like to give these robots increasing amounts of autonomy, including the ability to decide when to use lethal force.
To achieve this, Ronald Arkin of the Georgia Institute of Technology, in Atlanta, is developing a set of rules of engagement for battlefield robots to ensure that their use of lethal force follows the rules of ethics. In other words, he is trying to create an artificial conscience. Dr Arkin believes that there is another reason for putting robots into battle, which is that they have the potential to act more humanely than people. Stress does not affect a robot's judgment in the way it affects a soldier's.

His approach is to create what he calls a “multidimensional mathematical decision-space of possible behaviour actions”. Based on inputs ranging from radar data and current position to mission status and intelligence feeds, the system would divide the set of all possible actions into those that are ethical and those that are not. If, for example, the drone from which the fatal attack on Atef was launched had sensed that his car was overtaking a school bus, it might then have held fire.

There are comparisons to be drawn between Dr Arkin's work and the famous Three Laws of Robotics drawn up in the 1950s by Isaac Asimov, a science-fiction writer, to govern robot behaviour. But whereas Asimov's laws were intended to prevent robots from harming people in any circumstances, Dr Arkin's are supposed to ensure only that they are not unethically killed.

I have been working on preliminary notes for an essay on this topic, but it is all very preliminary. The most striking part of the project is that I do not see that the attempt to translate ethical decisionmaking into machine terms involves genuinely novel questions of ethics as such. On the contrary, what we seek to do is not to establish novel ethical principles, but rather to create, or re-create, hypothetically ideal or perfect ethical decisionmaking and conduct as we would imagine it for a hypothetically ideal or perfect human soldier but do so within a machine, a robot. The problems are in translation, not the creation of new problems or new solutions. In that sense, one could say that however interesting or important a task of ethical translation, it poses no new tasks in fundamental ethical theory.

And yet, accepting that, there nonetheless remains an area of grave difficulty - not because it represents a new problem of ethical decisionmaking different from humans, but because we do not have an adequately theorized approach to dealing with it. I refer to the question of proportionality jus in bello - the balancing of military advantage and damage to noncombatants that is (one of, if not) the core judgment of military ethics and indeed the laws of war. I can say with a fair amount of authority, having been working on this problem very quietly in my study for the last couple of years, that we have no method of weighing these two that is very defensible as a matter of ethical theory. It may be that the very idea of a "theory" to explain the weighing of what might well be understood as incommensurables is itself the problem, and yet in practice we do it and accept that we must do it. The problem, in other words, is not simply how one comes up with a theoretically defensible moral calculus for partly subjective judgments about how to weigh things that have enough similar properties to count as weighing oranges against oranges. That would be a difficult enough calculus to adapt to a machine but at least it would be about weighing similar things.

The much more difficult problem occurs when the things being weighed are, arguably, apples and oranges - both values, in the Isaiah Berlin plurality-of-values sense, but about very different things that seemingly cannot be weighed against each other, even though, as with many things of value in liberal theory, we must. One might think of Berlin's plurality of values as both a glory of liberalism and the tragedy of liberalism. Arguably, such incommensurability is what takes place in attempting to make moral judgments of proportionality jus in bello. Military advantage is a shorthand for describing not merely winning in a narrow military sense, but instead the values for which winning is morally, and not just prudentially, important - the moral value of a political community, its survival and interior values, stability in the external and internal political order, the assertion of moral values such as counter-genocide, etc. Damage to civilians, on the other hand, while referring in part to more remote and abstract values such as political community, is much more about immediate death and destruction. Although we immediately realize, in cases where the disproportion is great enough, when one or the other trumps, it is not very easy to elaborate a set of decisional rules about how to value these against each other. We can, to be sure, develop a certain practice, in a Witttgensteinean sense, or for that matter, a common law lawyer's precendential sense - but that is not really the same as a set of decision rules.

The point about robot soldiers is that this problem reduplicates itself when trying to reproduce a moral calculus at the machine level. It presents a problem, of course - but exactly, in principle at least, the same problem that we as humans have in conceptualizing the process of weighing and decision. But it also presents, perhaps, an opportunity - a kind of thought experiment, sci-fi made real, opportunity to think about how one would seek to operationalize, to make explicit, make external, what are otherwise highly intuitive and internal moral evaluations. And it is in this that I find the ethical issue of robot soldiers particularly interesting.

(Notes from a slowly developing draft paper, "Robot Soldiers and the Ethics of Proportionality Jus in Bello." Forthcoming ... someday.)