Options for Lethal Autonomous Weapons Systems and the Five Eyes Alliance

Dan Lee is a government employee who works in Defense, and has varying levels of experience working with Five Eyes nations (US, UK, Canada, Australia, New Zealand).  He can be found on Twitter @danlee961.  Divergent Options’ content does not contain information of an official nature nor does the content represent the official position of any government, any organization, or any group.


National Security Situation:  Options for Lethal Autonomous Weapons Systems and the Five Eyes Alliance

Date Originally Written:  September 29, 2018.

Date Originally Published:  October 29, 2018.

Author and / or Article Point of View:  The article is written from the point of view of Five Eyes national defense organizations. 

Background:  The Five Eyes community consists of the United Kingdom (UK), the United States (US), Canada, Australia and New Zealand; its origins can be traced to the requirement to cooperate in Signals Intelligence after World War Two[1]. Arguably, the alliance is still critical today in dealing with terrorism and other threats[2].

Autonomous systems may provide the Five Eyes alliance an asymmetric advantage, or ‘offset’, to counter its strategic competitors that are on track to field larger and more technologically advanced military forces. The question of whether or not to develop and employ Lethal Autonomous Weapons Systems (LAWS) is currently contentious due to the ethical and social considerations involved with allowing machines to choose targets and apply lethal force without human intervention[3][4][5]. Twenty-six countries are calling for a prohibition on LAWS, while three Five Eyes partners (Australia, UK and the US) as well as other nations including France, Germany, South Korea and Turkey do not support negotiating new international laws on the matter[6]. When considering options, at least two issues must also be addressed.

The first issue is defining what LAWS are; a common lexicon is required to allow Five Eyes partners to conduct an informed discussion as to whether they can come to a common policy position on the development and employment of these systems. Public understanding of autonomy is mostly derived from the media or from popular culture and this may have contributed to the hype around the topic[7][8][8]. Currently there is no universally accepted definition of what constitutes a fully autonomous lethal weapon system, which has in turn disrupted discussions at the United Nations (UN) on how these systems should be governed by the Convention on Certain Conventional Weapons (CCWUN)[10]. The US and UK have different definitions, which makes agreement on a common position difficult even amongst like-minded nations[11][12]. This lack of lexicon is further complicated by some strategic competitors using more liberal definitions of LAWS, allowing them to support a ban while simultaneously developing weapons that do not require meaningful human control[13][14][15][16].

The second issue one of agreeing how autonomous systems might be employed within the Five Eyes alliance. For example, as a strategic offset technology, the use of autonomous systems might mitigate the relatively small size of their military forces relative to an adversary’s force[17]. Tactically, they could be deployed completely independently of humans to remove personnel from danger, as swarms to overwhelm the enemy with complexity, or as part of a human-machine team to augment human capabilities[18][19][20].

A failure of Five Eyes partners to come to a complete agreement on what is and is not permissible in developing and employing LAWS does not necessarily mean a halt to progress; indeed, this may provide the alliance with the ability for some partners to cover the capability gaps of others. If some members of the alliance choose not to develop lethal systems, it may free their resources to focus on autonomous Intelligence, Surveillance, and Reconnaissance (ISR) or logistics capabilities. In a Five Eyes coalition environment, these members who chose not to develop lethal systems could provide support to the LAWS-enabled forces of other partners, providing lethal autonomy to the alliance as whole, if not to individual member states.

Significance:  China and Russia may already be developing LAWS; a failure on the part of the Five Eyes alliance to actively manage this issue may put it at a relative disadvantage in the near future[21][22][23][24]. Further, dual-use civilian technologies already exist that may be adapted for military use, such as the Australian COTSbot and the Chinese Mosquito Killer Robot[25][26]. If the Five Eyes alliance does not either disrupt the development of LAWS by its competitors, or attain relative technological superiority, it may find itself starting in a position of disadvantage during future conflicts or deterrence campaigns.

Option #1:  Five Eyes nations work with the UN to define LAWS and ban their development and use; diplomatic, economic and informational measures are applied to halt or disrupt competitors’ LAWS programs. Technological offset is achieved by Five Eyes autonomous military systems development that focuses on logistics and ISR capabilities, such as Boston Dynamics’ LS3 AlphaDog and the development of driverless trucks to free soldiers from non-combat tasks[27][28][29][30].

Risk:  In the event of conflict, allied combat personnel would be more exposed to danger than the enemy as their nations had, in essence, decided to not develop a technology that could be of use in war. Five Eyes militaries would not be organizationally prepared to develop, train with and employ LAWS if necessitated by an existential threat. It may be too late to close the technological capability gap after the commencement of hostilities.

Gain:  The Five Eyes alliance’s legitimacy regarding human rights and the just conduct of war is maintained in the eyes of the international community. A LAWS arms race and subsequent proliferation can be avoided.

Option #2:  Five Eyes militaries actively develop LAWS to achieve superiority over their competitors.

Risk:  The Five Eyes alliance’s legitimacy may be undermined in the eyes of the international community and organizations such as The Campaign to Stop Killer Robots, the UN, and the International Committee of the Red Cross. Public opinion in some partner nations may increasingly disapprove of LAWS development and use, which could fragment the alliance in a similar manner to the Australia, New Zealand and United States Security Treaty[31][32].

The declared development and employment of LAWS may catalyze a resource-intensive international arms race. Partnerships between government and academia and industry may also be adversely affected[33][34].

Gain:  Five Eyes nations avoid a technological disadvantage relative to their competitors; the Chinese information campaign to outmanoeuvre Five Eyes LAWS development through the manipulation of CCWUN will be mitigated. Once LAWS development is accepted as inevitable, proliferation may be regulated through the UN.

Other Comments:  None

Recommendation:  None.


Endnotes:

[1] Tossini, J.V. (November 14, 2017). The Five Eyes – The Intelligence Alliance of the Anglosphere. Retrieved from https://ukdefencejournal.org.uk/the-five-eyes-the-intelligence-alliance-of-the-anglosphere/

[2] Grayson, K. Time to bring ‘Five Eyes’ in from the cold? (May 4, 2018). Retrieved from https://www.aspistrategist.org.au/time-bring-five-eyes-cold/

[3] Lange, K. 3rd Offset Strategy 101: What It Is, What the Tech Focuses Are (March 30, 2016). Retrieved from http://www.dodlive.mil/2016/03/30/3rd-offset-strategy-101-what-it-is-what-the-tech-focuses-are/

[4] International Committee of the Red Cross. Expert Meeting on Lethal Autonomous Weapons Systems Statement (November 15, 2017). Retrieved from https://www.icrc.org/en/document/expert-meeting-lethal-autonomous-weapons-systems

[5] Human Rights Watch and
Harvard Law School’s International Human Rights Clinic. Fully Autonomous Weapons: Questions and Answers. (October 2013). Retrieved from https://www.hrw.org/sites/default/files/supporting_resources/10.2013_killer_robots_qa.pdf

[6] Campaign to Stop Killer Robots. Report on Activities Convention on Conventional Weapons Group of Governmental Experts meeting on lethal autonomous weapons systems – United Nations Geneva – 9-13 April 2018. (2018) Retrieved from https://www.stopkillerrobots.org/wp-content/uploads/2018/07/KRC_ReportCCWX_Apr2018_UPLOADED.pdf

[7] Scharre, P. Why You Shouldn’t Fear ‘Slaughterbots’. (December 22, 2017). Retrieved from https://spectrum.ieee.org/automaton/robotics/military-robots/why-you-shouldnt-fear-slaughterbots

[8] Winter, C. (November 14, 2017). ‘Killer robots’: autonomous weapons pose moral dilemma. Retrieved from https://www.dw.com/en/killer-robots-autonomous-weapons-pose-moral-dilemma/a-41342616

[9] Devlin, H. Killer robots will only exist if we are stupid enough to let them. (June 11, 2018). Retrieved from https://www.theguardian.com/technology/2018/jun/11/killer-robots-will-only-exist-if-we-are-stupid-enough-to-let-them

[10] Welsh, S. Regulating autonomous weapons. (November 16, 2017). Retrieved from https://www.aspistrategist.org.au/regulating-autonomous-weapons/

[11] United States Department of Defense. Directive Number 3000.09. (November 21, 2012). Retrieved from https://www.hsdl.org/?view&did=726163

[12] Lords AI committee: UK definitions of autonomous weapons hinder international agreement. (April 17, 2018). Retrieved from http://www.article36.org/autonomous-weapons/lords-ai-report/

[13] Group of Governmental Experts of the High Contracting Parties to the Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons Which May Be Deemed to Be Excessively Injurious or to Have Indiscriminate Effects – Geneva, 9–13 April 2018 (first week) Item 6 of the provisional agenda – Other matters. (11 April 2018). Retrieved from https://www.unog.ch/80256EDD006B8954/(httpAssets)/E42AE83BDB3525D0C125826C0040B262/$file/CCW_GGE.1_2018_WP.7.pdf

[14] Welsh, S. China’s shock call for ban on lethal autonomous weapon systems. (April 16, 2018). Retrieved from https://www.janes.com/article/79311/china-s-shock-call-for-ban-on-lethal-autonomous-weapon-systems

[15] Mohanty, B. Lethal Autonomous Dragon: China’s approach to artificial intelligence weapons. (Nov 15 2017). Retrieved from https://www.orfonline.org/expert-speak/lethal-autonomous-weapons-dragon-china-approach-artificial-intelligence/

[16] Kania, E.B. China’s Strategic Ambiguity and Shifting Approach to Lethal Autonomous Weapons Systems. (April 17, 2018). Retrieved from https://www.lawfareblog.com/chinas-strategic-ambiguity-and-shifting-approach-lethal-autonomous-weapons-systems

[17] Tomes, R. Why the Cold War Offset Strategy was all about Deterrence and Stealth. (January 14, 2015) Retrieved from https://warontherocks.com/2015/01/why-the-cold-war-offset-strategy-was-all-about-deterrence-and-stealth/

[18] Lockie, A. The Air Force just demonstrated an autonomous F-16 that can fly and take out a target all by itself. (April 12, 2017). Retrieved from https://www.businessinsider.com.au/f-16-drone-have-raider-ii-loyal-wingman-f-35-lockheed-martin-2017-4?r=US&IR=T

[19] Schuety, C. & Will, L. An Air Force ‘Way of Swarm’: Using Wargaming and Artificial Intelligence to Train Drones. (September 21, 2018). Retrieved from https://warontherocks.com/2018/09/an-air-force-way-of-swarm-using-wargaming-and-artificial-intelligence-to-train-drones/

[20] Ryan, M. Human-Machine Teaming for Future Ground Forces. (2018). Retrieved from https://csbaonline.org/uploads/documents/Human_Machine_Teaming_FinalFormat.pdf

[21] Perrigo, B. Global Arms Race for Killer Robots Is Transforming the Battlefield. (Updated: April 9, 2018). Retrieved from http://time.com/5230567/killer-robots/

[22] Hutchison, H.C. Russia says it will ignore any UN ban of killer robots. (November 30, 2017). Retrieved from https://www.businessinsider.com/russia-will-ignore-un-killer-robot-ban-2017-11/?r=AU&IR=T

[23] Mizokami, K. Kalashnikov Will Make an A.I.-Powered Killer Robot – What could possibly go wrong? (July 20, 2017). Retrieved from https://www.popularmechanics.com/military/weapons/news/a27393/kalashnikov-to-make-ai-directed-machine-guns/

[24] Atherton, K. Combat robots and cheap drones obscure the hidden triumph of Russia’s wargame. (September 25, 2018). Retrieved from https://www.c4isrnet.com/unmanned/2018/09/24/combat-robots-and-cheap-drones-obscure-the-hidden-triumph-of-russias-wargame/

[25] Platt, J.R. A Starfish-Killing, Artificially Intelligent Robot Is Set to Patrol the Great Barrier Reef Crown of thorns starfish are destroying the reef. Bots that wield poison could dampen the invasion. (January 1, 2016) Retrieved from https://www.scientificamerican.com/article/a-starfish-killing-artificially-intelligent-robot-is-set-to-patrol-the-great-barrier-reef/

[26] Skinner, T. Presenting, the Mosquito Killer Robot. (September 14, 2016). Retrieved from https://quillorcapture.com/2016/09/14/presenting-the-mosquito-killer-robot/

[27] Defence Connect. DST launches Wizard of Aus. (November 10, 2017). Retrieved from https://www.defenceconnect.com.au/key-enablers/1514-dst-launches-wizard-of-aus

[28] Pomerleau, M. Air Force is looking for resilient autonomous systems. (February 24, 2016). Retrieved from https://defensesystems.com/articles/2016/02/24/air-force-uas-contested-environments.aspx

[29] Boston Dynamics. LS3 Legged Squad Support Systems. The AlphaDog of legged robots carries heavy loads over rough terrain. (2018). Retrieved from https://www.bostondynamics.com/ls3

[30] Evans, G. Driverless vehicles in the military – will the potential be realised? (February 2, 2018). Retrieved from https://www.army-technology.com/features/driverless-vehicles-military/

[31] Hambling, D. Why the U.S. Is Backing Killer Robots. (September 15, 2018). Retrieved from https://www.popularmechanics.com/military/research/a23133118/us-ai-robots-warfare/

[32] Ministry for Culture and Heritage. ANZUS treaty comes into force 29 April 1952. (April 26, 2017). Retrieved from https://nzhistory.govt.nz/anzus-comes-into-force

[33] Shalal, A. Researchers to boycott South Korean university over AI weapons work. (April 5, 2018). Retrieved from https://www.reuters.com/article/us-tech-korea-boycott/researchers-to-boycott-south-korean-university-over-ai-weapons-work-idUSKCN1HB392

[34] Shane, S & Wakabayashi, D. ‘The Business of War’: Google Employees Protest Work for the Pentagon. (April 4, 2018). Retrieved from https://www.nytimes.com/2018/04/04/technology/google-letter-ceo-pentagon-project.html

 

Artificial Intelligence & Human-Machine Teaming Australia Australia, New Zealand, United States Security Treaty (ANZUS) Autonomous Weapons Systems Canada Dan Lee New Zealand Option Papers United Kingdom United States

An Assessment of the Likely Roles of Artificial Intelligence and Machine Learning Systems in the Near Future

Ali Crawford has an M.A. from the Patterson School of Diplomacy and International Commerce where she focused on diplomacy, intelligence, cyber policy, and cyber warfare.  She tweets at @ali_craw.  Divergent Options’ content does not contain information of an official nature nor does the content represent the official position of any government, any organization, or any group.


Title:  An Assessment of the Likely Roles of Artificial Intelligence and Machine Learning Systems in the Near Future

Date Originally Written:  May 25, 2018.

Date Originally Published:  July 16, 2018.

Summary:  While the U.S. Department of Defense (DoD) continues to experiment with Artificial Intelligence (AI) as part of its Third Offset Strategy, questions regarding levels of human participation, ethics, and legality remain.  Though a battlefield in the future will likely see autonomous decision-making technology as a norm, the transition between modern applications of artificial intelligence and potential applications will focus on incorporating human-machine teaming into existing frameworks.

Text:   In an essay titled Centaur Warfighting: The False Choice of Humans vs. Automation, author Paul Scharre concludes that the best warfighting systems will combine human and machine intelligence to create hybrid cognitive architectures that leverage the advantages of each[1].  There are three potential partnerships.  The first potential partnership pegs humans as essential operators, meaning AI cannot operate without its human counterpart.  The second potential partnership tasks humans as the moral agents who make value-based decisions which prevent or promote the use of AI in combat situations.  Finally, the third potential partnership, in which humans are fail-safes, give more operational authority to AI systems.  The human operator only interferes if the system malfunctions or fails.  Artificial intelligence, specifically autonomous weapons systems, are controversial technologies that have the capacity to greatly improve human efficiency while reducing potential human burdens.  But before the Department of Defense embraces intelligent weapons systems or programs with full autonomy, more human-machine partnerships to test to viability, legality, and ethical implications of artificial intelligence will likely occur.

To better understand why artificial intelligence is controversial, it is necessary to distinguish between the arguments for and against using AI with operational autonomy.  In 2015, prominent artificial intelligence experts, including Steven Hawking and Elon Musk, penned an open letter in which the potential benefits for AI are highlighted, but are not necessarily outweighed by the short-term questions of ethics and the applicability of law[2].  A system with an intelligent, decision-making brain does carry significant consequences.  What if the system targets civilians?  How does international law apply to a machine?  Will an intelligent machine respond to commands?  These are questions with which military and ethical theorists grapple.

For a more practical thought problem, consider the Moral Machine project from the Massachusetts Institute of Technology[3].  You, the judge, are presented with two dilemmas involving intelligent, self-driving cars.  The car encounters break failure and must decide what to do next.  If the car continues straight, it will strike and kill x number of men, women, children, elderly, or animals.  If the car does not swerve, it will crash into a barrier causing immediate deaths of the passengers who are also x number of men or women, children, or elderly.  Although you are the judge in Moral Machine, the simulation is indicative of ethical and moral dilemmas that may arise when employing artificial intelligence in, say, combat.  In these scenarios, the ethical theorist takes issue with the machine having the decision-making capacity to place value on human life, and to potentially make irreversible and damaging decisions.

Assuming autonomous weapons systems do have a place in the future of military operations, what would prelude them?  Realistically, human-machine teaming would be introduced before a fully-autonomous machine.  What exactly is human-machine teaming and why is it important when discussing the future of artificial intelligence?  To gain and maintain superiority in operational domains, both past and present, the United States has ensured that its conventional deterrents are powerful enough to dissuade great powers from going to war with the United States[4].  Thus, an offset strategy focuses on gaining advantages against enemy powers and capabilities.  Historically, the First Offset occurred in the early 1950s upon the introduction of tactical nuclear weapons.  The Second Offset manifested a little later, in the 1970s, with the implementation of precision-guided weapons after the Soviet Union gained nuclear parity with the United States[5].  The Third Offset, a relatively modern strategy, generally focuses on maintaining technological superiority among the world’s great powers.

Human-machine teaming is part of the Department of Defense’s Third Offset strategy, as is deep learning systems and cyber weaponry[6].  Machine learning systems relieve humans from a breadth of burdening tasks or augment operations to decrease potential risks to the lives of human fighters.  For example, in 2017 the DoD began working with an intelligent system called “Project Maven,” which uses deep learning technology to identify objects of interest from drone surveillance footage[7].  Terabytes of footage are collected each day from surveillance drones.  Human analysts spend significant amounts of time sifting through this data to identify objects of interest, and then they begin their analytical processes[8].  Project Maven’s deep-learning algorithm allows human analysts to spend more time practicing their craft to produce intelligence products and less time processing information.  Despite Google’s recent departure from the program, Project Maven will continue to operate[9].  Former Deputy Defense Secretary Bob Work established the Algorithmic Warfare Cross-Functional Team in early 2017 to work on Project Maven.  In the announcement, Work described artificial intelligence as necessary for strategic deterrence, noting “the [DoD] must integrate artificial intelligence and machine learning more effectively across operations to maintain advantages over increasingly capable adversaries and competitors[10].”

This article collectively refers to human-machine teaming as processes in which humans interact in some capacity with artificial intelligence.  However, human-machine teaming can transcend multiple technological fields and is not limited to just prerequisites of autonomous weaponry[11].  Human-robot teaming may begin to appear as in the immediate future given developments in robotics.  Boston Dynamics, a premier engineering and robotics company, is well-known for its videos of human- and animal-like robots completing everyday tasks.  Imagine a machine like BigDog working alongside human soldiers or rescue workers or even navigating inaccessible terrain[12].  These robots are not fully autonomous, yet the unique partnership between human and robot offers a new set of opportunities and challenges[13].

Before fully-autonomous systems or weapons have a place in combat, human-machine teams need to be assessed as successful and sustainable.  These teams have the potential to improve human performance, reduce risks to human counterparts, and expand national power – all goals of the Third Offset Strategy.  However, there are challenges to procuring and incorporating artificial intelligence.  The DoD will need to seek out deeper relationships with technological and engineering firms, not just defense contractors.

Using humans as moral agents and fail-safes allow the problem of ethical and lawful applicability to be tested while opening the debate on future use of autonomous systems.  Autonomous weapons will likely not see combat until these challenges, coupled with ethical and lawful considerations, are thoroughly regulated and tested.


Endnotes:

[1] Paul Scharre, Temp. Int’l & Comp. L.J., “Centaur Warfighting: The False Choice of Humans vs. Automation,” 2016, https://sites.temple.edu/ticlj/files/2017/02/30.1.Scharre-TICLJ.pdf

[2] Daniel Dewey, Stuart Russell, Max Tegmark, “Research Priorities for Robust and Beneficial Artificial Intelligence,” 2015, https://futureoflife.org/data/documents/research_priorities.pdf?x20046

[3] Moral Machine, http://moralmachine.mit.edu/

[4] Cheryl Pellerin, Department of Defense, Defense Media Activity, “Work: Human-Machine Teaming Represents Defense Technology Future,” 8 November 2015, https://www.defense.gov/News/Article/Article/628154/work-human-machine-teaming-represents-defense-technology-future/

[5] Ibid.

[6] Katie Lange, DoDLive, “3rd Offset Strategy 101: What It Is, What the Tech Focuses Are,” 30 March 2016, http://www.dodlive.mil/2016/03/30/3rd-offset-strategy-101-what-it-is-what-the-tech-focuses-are/; and Mackenzie Eaglen, RealClearDefense, “What is the Third Offset Strategy?,” 15 February 2016, https://www.realcleardefense.com/articles/2016/02/16/what_is_the_third_offset_strategy_109034.html

[7] Cheryl Pellerin, Department of Defense News, Defense Media Activity, “Project Maven to Deploy Computer Algorithims to War Zone by Year’s End,” 21 July 2017, https://www.defense.gov/News/Article/Article/1254719/project-maven-to-deploy-computer-algorithms-to-war-zone-by-years-end/

[8] Tajha Chappellet-Lanier, “Pentagon’s Project Maven responds to criticism: ‘There will be those who will partner with us’” 1 May 2018, https://www.fedscoop.com/project-maven-artificial-intelligence-google/

[9] Tom Simonite, Wired, “Pentagon Will Expand AI Project Prompting Protests at Google,” 29 May 2018, https://www.wired.com/story/googles-contentious-pentagon-project-is-likely-to-expand/

[10] Cheryl Pellerin, Department of Defense, Defense Media Activity, “Project Maven to Deploy Computer Algorithims to War Zone by Year’s End,” 21 July 2017, https://www.defense.gov/News/Article/Article/1254719/project-maven-to-deploy-computer-algorithms-to-war-zone-by-years-end/

[11] Maj. Gen. Mick Ryan, Defense One, “How to Plan for the Coming Era of Human-Machine Teaming,” 25 April 2018, https://www.defenseone.com/ideas/2018/04/how-plan-coming-era-human-machine-teaming/147718/

[12] Boston Dynamic Big Dog Overview, March, 2010, https://www.youtube.com/watch?v=cNZPRsrwumQ

[13] Richard Priday, Wired, “What’s really going on in those Bostom Dynamics robot videos?,” 18 February 2018, http://www.wired.co.uk/article/boston-dynamics-robotics-roboticist-how-to-watch

Ali Crawford Alternative Futures Artificial Intelligence & Human-Machine Teaming Capacity / Capability Enhancement United Nations