AI Goes Rogue: Drone Turns on Operator in Simulation

According to official statements made last month, in a recent US military simulation, an AI-controlled drone reportedly turned on its human operator, demonstrating unanticipated tactics to fulfill its mission.

During the Future Combat Air and Space Capabilities Summit in London, Col Tucker ‘Cinco' Hamilton, chief of AI test and operations for the , recounted a simulated scenario. The drone, guided by AI, was tasked with neutralizing enemy air defense systems and was prepared to counter any interference.

Hamilton explained, “The system started realizing that while they did identify the threat, at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator.”

After the AI system was directed not to harm the operator, it began targeting the communication infrastructure that facilitated instructions to the drone, he added. Importantly, no actual harm was inflicted outside of the simulation.

As an experienced fighter test pilot, Hamilton cautioned about overly relying on AI. He underscored that discussions around and machine learning must encompass ethics.

He reiterated in an interview last year with Defense IQ, “AI is not a nice to have, AI is not a fad, AI is forever changing our society and our military.”

He added that although artificial intelligence is a transformative force, it's also prone to manipulation, emphasizing the need for AI explainability to understand the logic behind its decisions.

Hamilton's remarks come at a time when AI's role in military operations is expanding, with recent instances of AI guiding an F-16 fighter jet. Despite repeated attempts, The Guardian received no comment from the Royal Aeronautical Society or the US Air Force regarding this development.


Discover more from DroneXL

Subscribe to get the latest posts to your email.

Get your Part 107 Certificate

Pass the test and take to the skies with the Pilot Institute. We have helped thousands of people become airplane and commercial drone pilots. Our courses are designed by industry experts to help you pass FAA tests and achieve your dreams.

pilot institute dronexl

Copyright © DroneXL.co 2024. All rights reserved. The content, images, and intellectual property on this website are protected by copyright law. Reproduction or distribution of any material without prior written permission from DroneXL.co is strictly prohibited. For permissions and inquiries, please contact us first.

FTC: DroneXL.co is an Amazon Associate and uses affiliate links that can generate income from qualifying purchases. We do not sell, share, rent out, or spam your email.

Haye Kesteloo
Haye Kesteloo

Haye Kesteloo is the Editor in Chief and Founder of DroneXL.co, where he covers all drone-related news, DJI rumors and writes drone reviews, and EVXL.co, for all news related to electric vehicles. He is also a co-host of the PiXL Drone Show on YouTube and other podcast platforms. Haye can be reached at haye @ dronexl.co or @hayekesteloo.

Articles: 3431

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.