Options to Manage the Risks of Integrating Artificial Intelligence into National Security and Critical Industry Organizations

Lee Clark is a cyber intelligence analyst.  He holds an MA in intelligence and international security from the University of Kentucky’s Patterson School.  Divergent Options’ content does not contain information of an official nature nor does the content represent the official position of any government, any organization, or any group.

National Security Situation:  What are the potential risks of integrating artificial intelligence (AI) into national security and critical infrastructure organizations and potential options for mitigating these risks?

Date Originally Written:  May 19, 2018.

Date Originally Published:  July 2, 2018.

Author and / or Article Point of View:  The author is currently an intelligence professional focused on threats to critical infrastructure and the private sector.  This article will use the U.S. Department of Homeland Security’s definition of “critical infrastructure,” referring to 16 public and private sectors that are deemed vital to the U.S. economy and national functions.  The designated sectors include financial services, emergency response, food and agriculture, energy, government facilities, defense industry, transportation, critical manufacturing, communications, commercial facilities, chemical production, civil nuclear functions, dams, healthcare, information technology, and water/wastewater management[1].  This article will examine some broad options to mitigate some of the most prevalent non-technical risks of AI integration, including legal protections and contingency planning.

Background:  The benefits of incorporating AI into the daily functions of an organization are widely championed in both the private and public sectors.  The technology has the capability to revolutionize facets of government and private sector functions like record keeping, data management, and customer service, for better or worse.  Bringing AI into the workplace has significant risks on several fronts, including privacy/security of information, record keeping/institutional memory, and decision-making.  Additionally, the technology carries a risk of backlash over job losses as automation increases in the global economy, especially for more skilled labor.  The national security and critical industry spheres are not facing an existential threat, but these are risks that cannot be dismissed.

Significance:  Real world examples of these concerns have been reported in open source with clear implications for major corporations and national security organizations.  In terms of record keeping/surveillance related issues, one need only look to recent court cases in which authorities subpoenaed the records of an Amazon Alexa, an appliance that acts as a digital personal assistant via a rudimentary AI system.  This subpoena situation becomes especially concerning to users, given recent reports of Alexa’s being converted into spying tools[2].  Critical infrastructure organizations, especially defense, finance, and energy companies, exist within complex legal frameworks that involve international laws and security concerns, making legal protections of AI data all the more vital.

In the case of issues involving decision-making and information security, the dangers are no less severe.  AIs are susceptible to a variety of methods that seek to manipulate decision-making, including social engineering and, more specifically, disinformation efforts.  Perhaps the most evident case of social engineering against an AI is an instance in which Microsoft’s AI endorsed genocidal statements after a brief conversation with users on Twitter[3].  If it is possible to convince an AI to support genocide, it is not difficult to imagine the potential to convince it to divulge state secrets or turn over financial information with some key information fed in a meaningful sequence[4].  In another public instance, an Amazon Echo device recently recorded a private conversation in an owner’s home and sent the conversation to another user without requesting permission from the owner[5].  Similar instances are easy to foresee in a critical infrastructure organization such as a nuclear energy plant, in which an AI may send proprietary information to an uncleared user.

AI decisions also have the capacity to surprise developers and engineers tasked with maintenance, which could present problems of data recovery and control.  For instance, developers discovered that Facebook’s AI had begun writing a modified version of a coding language for efficiency, having essentially created its own code dialect, causing transparency concerns.  Losing the ability to examine and assess coding decisions presents problems for replicating processes and maintenance of a system[6].

AI integration into industry also carries a significant risk of backlash from workers.  Economists and labor scholars have been discussing the impacts of automation and AI on employment and labor in the global economy.  This discussion is not merely theoretical in nature, as evidenced by leaders of major tech companies making public remarks supporting basic income as automation will likely replace a significant portion of labor market in the coming decades[7].

Option #1:  Leaders in national security and critical infrastructure organizations work with internal legal teams to develop legal protections for organizations while lobbying for legislation to secure legal privileges for information stored by AI systems (perhaps resembling attorney-client privilege or spousal privileges).

Risk:  Legal teams may lack the technical knowledge to foresee some vulnerabilities related to AI.

Gain:  Option #1 proactively builds liability shields, protections, non-disclosure agreements, and other common legal tools to anticipate needs for AI-human interactions.

Option #2:  National security and critical infrastructure organizations build task forces to plan protocols and define a clear AI vision for organizations.

Risk:  In addition to common pitfalls of group work like bandwagoning and group think, this option is vulnerable to insider threats like sabotage or espionage attempts.  There is also a risk that such groups may develop plans that are too rigid or short-sighted to be adaptive in unforeseen emergencies.

Gain:  Task forces can develop strategies and contingency plans for when emergencies arise.  Such emergencies could include hacks, data breaches, sabotage by rogue insiders, technical/equipment failures, or side effects of actions taken by an AI in a system.

Option #3:  Organization leaders work with intelligence and information security professionals to try to make AI more resilient against hacker methods, including distributed denial-of-service attacks, social engineering, and crypto-mining.

Risk:  Potential to “over-secure” systems, resulting in loss of efficiency or overcomplicating maintenance processes.

Gain:  Reduced risk of hacks or other attacks from malicious actors outside of organizations.

Other Comments:  None.

Recommendation: None.


[1] DHS. (2017, July 11). Critical Infrastructure Sectors. Retrieved May 28, 2018, from https://www.dhs.gov/critical-infrastructure-sectors

[2] Boughman, E. (2017, September 18). Is There an Echo in Here? What You Need to Consider About Privacy Protection. Retrieved May 19, 2018, from https://www.forbes.com/sites/forbeslegalcouncil/2017/09/18/is-there-an-echo-in-here-what-you-need-to-consider-about-privacy-protection/

[3] Price, R. (2016, March 24). Microsoft Is Deleting Its AI Chatbot’s Incredibly Racist Tweets. Retrieved May 19, 2018, from http://www.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3

[4] Osaba, O. A., & Welser, W., IV. (2017, December 06). The Risks of AI to Security and the Future of Work. Retrieved May 19, 2018, from https://www.rand.org/pubs/perspectives/PE237.html

[5] Shaban, H. (2018, May 24). An Amazon Echo recorded a family’s conversation, then sent it to a random person in their contacts, report says. Retrieved May 28, 2018, from https://www.washingtonpost.com/news/the-switch/wp/2018/05/24/an-amazon-echo-recorded-a-familys-conversation-then-sent-it-to-a-random-person-in-their-contacts-report-says/

[6] Bradley, T. (2017, July 31). Facebook AI Creates Its Own Language in Creepy Preview Of Our Potential Future. Retrieved May 19, 2018, from https://www.forbes.com/sites/tonybradley/2017/07/31/facebook-ai-creates-its-own-language-in-creepy-preview-of-our-potential-future/

[7] Kharpal, A. (2017, February 21). Tech CEOs Back Call for Basic Income as AI Job Losses Threaten Industry Backlash. Retrieved May 19, 2018, from https://www.cnbc.com/2017/02/21/technology-ceos-back-basic-income-as-ai-job-losses-threaten-industry-backlash.html

Critical Infrastructure Cyberspace Emerging Technology Lee Clark Option Papers Private Sector Resource Scarcity

Assessment of the Threat to Southeast Asia Posed by Illegal, Unregulated and Unreported Fishing

Blake Herzinger is a private-sector maritime security advisor assisting the U.S. Pacific Fleet in implementation and execution of the Southeast Asia Maritime Security Initiative and Pacific Command-wide maritime security efforts.  He served in the United States Navy as an intelligence officer in Singapore, Japan, Italy, and exotic Jacksonville, Florida.  His writing has appeared in Proceedings, CIMSEC and The Diplomat.  He can be found on Twitter @BDHerzinger.  Divergent Options’ content does not contain information of any official nature nor does the content represent the official position of any government, any organization, or any group. 

Title:  Assessment of the Threat to Southeast Asia Posed by Illegal, Unregulated and Unreported Fishing

Date Originally Written:  September 24, 2017.

Date Originally Published:  November 27, 2017.

Summary:  Regional conflict brews in Southeast Asia as states vie for access to fish stocks and, increasingly, rely on Illegal, Unreported and Unregulated Fishing (IUUF) to meet national requirements.  IUUF risks the collapse of targeted fish stocks, destroys the maritime environment, degrades internal security, and brings national security forces into increasingly-escalatory encounters.

Text:  Over one billion residents of the Asia-Pacific rely upon fish as their primary source of protein, and the fish stocks of the region are under a relentless assault[1].  Current estimates place IUUF at between 11 and 26 million metric tons (MMT) yearly (total legal capture is approximately 16.6 MMT yearly), with an estimated value loss to regional economies of $10-23.5 billion[2][3].  Over a 25 year period, fish stocks in the South China Sea have declined anywhere from 6 to 33 percent, with some falling as much as 40 percent over the last 5 years.  In 2015, at least 490 million people in Southeast Asia lived in chronic hunger, with millions of children throughout the region stunted due to malnutrition[4].

Illegal fishing’s pernicious by-product is the critical damage done to the maritime environment by those flouting fishery regulations.  As large fish become more scarce as a result of industrial-scale overfishing, smaller-scale fishermen turn to dangerous and illegal practices to catch enough fish to survive.  Blast fishing obliterates coral reefs and kills indiscriminately, but despite prohibitions continues at a rate of nearly 10,000 incidents a day in Philippines alone[5].  Cyanide fishing is also still widespread, despite being banned in several Southeast Asian countries.  Used to stun fish for live capture (for aquariums or regionally popular live fish restaurants), cyanide contributes to the devastation of coral reefs across the SCS.  Giant clam poaching also has deleterious effects on reefs across the region as poachers race to feed Chinese demand for these shellfish.  Reefs throughout the Coral Triangle are interdependent, relying on one another for pollination, and as the reefs are destroyed by poachers seeking short-term gains, or even by small fishermen eking out a subsistence lifestyle, the effects of collapse ripple outward across the region.  The region is approaching an inflection point at which the damage will be irreparable.

The People’s Republic of China (PRC), which accounts for one-third of global fish consumption and is the world’s largest seafood exporter, fittingly leads the way in aggressively protecting its fishing fleets with an overwhelmingly powerful coast guard that dwarfs any other maritime law enforcement body in Asia[6][7].  As IUUF and environmental destruction cut into maritime resources and competition for those increasingly scarce resources escalates, national maritime law enforcement and naval forces are being rapidly expanded and widely deployed to protect natural resources and domestic fishing fleets.  If unmanaged, the friction generated by these fleets’ increasing interaction could easily explode into violent conflict.

For many countries in the region, the state’s legitimacy rests largely upon its ability to provide access to basic necessities and protect its citizens’ livelihoods.  Tens of millions across East Asia and Southeast Asia depend on fisheries for employment and, in many cases, their survival.  Should fish stocks begin to fail, regional states’ foundations will be threatened.  The combination of inadequate food supply and loss of livelihood could reasonably be expected to spur civil unrest.  In a state such as Indonesia, where 54 percent of the population relies on fish as its primary animal protein, historically weak institutions and propensity for military intervention only amplify the potential consequences of food insecurity.  In the PRC, the Chinese Communist Party (CCP) actively encourages illegal fishing to provide its 1.379 billion people with the fish, seafood and marine products that its lower-and-middle-class, as well as elites, expect.  Legitimacy of the CCP, at least in part, is dependent on the continued production of regional fisheries and desire to buttress its legitimacy will continue to drive this vicious cycle.

The above mentioned calamities can occur in isolation, but they are most often interlinked.  For instance, in the infamous 2012 Scarborough Shoal incident, Philippines maritime law enforcement boarded a PRC fishing boat that had been engaged in giant clam and shark poaching, as well as coral reef destruction.  Armed PRC maritime law enforcement vessels intervened and sparked an external dispute that continues in 2017[8].  Ensuing flame wars between Filipino and Chinese hackers and economic measures enacted by the PRC against the Philippines threatened stability in both the domestic and international spheres of both countries.  The threat posed by IUUF is not just about fish, its direct and follow-on effects have the potential to drag Southeast Asia into disastrous conflict.


[1] Till, G. (2013). Seapower: a guide for the 21st century. London: Routledge Ltd.

[2] Caputo, J. (2017). A Global Fish War is Coming. Proceedings, 143(8), 1,374. Retrieved October 10, 2017, from https://www.usni.org/magazines/proceedings/2017-08/global-fish-war-coming

[3] One of the World’s Biggest Fisheries Is on The Verge of Collapse. (2017, August 02). Retrieved October 25, 2017, from https://news.nationalgeographic.com/2016/08/wildlife-south-china-sea-overfishing-threatens-collapse/

[4] Asia-Pacific region achieves Millennium Development Goal to reduce hunger by half by 2015. (2015, May 28). Retrieved October 25, 2017, from http://www.fao.org/asiapacific/news/detail-events/en/c/288506/

[5] Guy, A. (n.d.). Local Efforts Put a Dent in Illegal Dynamite Fishing in the Philippines. Retrieved October 25, 2017, from http://oceana.org/blog/local-efforts-put-dent-illegal-dynamite-fishing-philippines

[6] Jacobs, A. (2017, April 30). China’s Appetite Pushes Fisheries to the Brink. Retrieved October 25, 2017, from https://www.nytimes.com/2017/04/30/world/asia/chinas-appetite-pushes-fisheries-to-the-brink.html

[7] Asia-Pacific Maritime Security Strategy (Rep.). (2015, August 14). Retrieved https://www.defense.gov/Portals/1/Documents/pubs/NDAA%20A-P_Maritime_SecuritY_Strategy-08142015-1300-FINALFORMAT.PDF

[8] Are Maritime Law Enforcement Forces Destabilizing Asia? (n.d.). Retrieved October 25, 2017, from https://csis-ilab.github.io/cpower-viz/csis-china-sea/

Assessment Papers Blake Herzinger Environmental Factors Resource Scarcity South China Sea Southeast Asia

Call for Papers: Environmental Factors and Resource Scarcity as a Conflict Driver



Divergent Options is calling for papers assessing situations or discussing options related to Environmental Factors and Resource Scarcity as a Conflict Driver.

Please limit your article to 1,000 words and write using our Options Paper or Assessment Paper templates which are designed for ease of use by both writers and readers alike.

Please send your article to submissions@divergentoptions.org by October 20, 2017.

If you are not interested in writing on this topic we still welcome individual articles on virtually any national security situation an author is passionate about.  Please do not let our call for papers cause you to hesitate to send us your idea.  We look forward to hearing from you!

As background, the January 2017 Global Trends report released by the United States National Intelligence Council envisions a future where:

Environmental and climate changes will challenge systems in different dimensions; heat waves, for example, stress infrastructure, energy, human and animal health, and agriculture. Climate change— observed or anticipated—almost certainly will become an increasingly integral component of how people view their world, especially as populations are projected to swell in those areas most vulnerable to extreme weather events and sea-level rise, including coastal megacities and regions already suffering from water scarcity. Many of the ecological and environmental stresses from climate change—and the infectious diseases it will affect—will cut across state borders, making coordination among governments and international institutions crucial to effective responses. Policies and programs to mitigate and adapt to these challenges will spur opportunities for those well-positioned to benefit.

How will this vision of the future affect national security?

What can be done to address this vision of the future?



Call For Papers Environmental Factors Resource Scarcity