New Neural Tech Could Power Insect-Sized Intelligent Flying Robots

Researchers at Delft University of Technology have developed a groundbreaking neuromorphic vision-to-control system that enables autonomous drone flights. This new technology, which mimics the brain's biological processing, could pave the way for creating intelligent flying robots as small as insects.

The Breakthrough: Neuromorphic Vision-to-Control Mechanism

The innovative neuromorphic system developed by the research team involves a spiking neural network (SNN) that processes data from event-based cameras. This network, consisting of 28,800 neurons, is designed to map incoming raw data to estimate the 3D motion of the drone within its environment, reports Interesting, Engineering. Unlike traditional vision-based drone flight systems that use artificial neural networks, this new approach is significantly more compact and energy-efficient.

“The neuromorphic processor consumes only 0.94 watts of idle power and an additional 7-12 milliwatts while running the network,” researchers stated. This efficient use of power is a crucial factor for small-scale autonomous drones.

The drone operates autonomously using neuromorphic image processing and control modeled after the functions of animal brains.
The drone operates autonomously using neuromorphic image processing and control modeled after the functions of animal brains.

SNN-Driven Drone Flight

Biological systems are known for their low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware, which includes event-based vision and spiking neural networks, aims to replicate these characteristics. However, current implementations are limited by the small network size of existing neuromorphic processors and the complexities involved in training spiking neural networks.

To showcase the potential of neuromorphic hardware, the researchers utilized a complete neuromorphic vision-to-control pipeline to operate a drone. The SNN, trained to process raw event camera data, enabled the drone to conduct autonomous vision-based ego-motion estimation and control at a frequency of about 200 Hz. This allowed the drone to hover, land, and maneuver sideways autonomously.

Efficient Neuromorphic Flight

The vision-to-control pipeline was implemented on 's Loihi neuromorphic processor and integrated into a small flying robot. This setup included an event camera, a neuromorphic processor, a single-board computer, and a flight controller. Weighing 994 grams and with a diameter of 35 cm, the drone demonstrated efficient and smooth height reduction during landing experiments.

The system assumed a static, texture-rich, flat surface viewed by a DVS 240 camera to meet computational constraints. The vision processing pipeline reduced spatial resolution by focusing on corner regions of interest (ROIs), processing up to 90 events per ROI with a small SNN. This SNN, containing 7,200 neurons and 506,400 synapses across five layers, estimated optical flow for each ROI.

Trained using self-supervised contrast maximization for vision and an evolutionary algorithm for control, the system successfully navigated in various lighting conditions. Its energy consumption was also notably efficient compared to traditional GPU solutions.

Overview of the spiking neural network.
Overview of the spiking neural network.

Future Implications

The research team's success with the neuromorphic vision-to-control pipeline demonstrates the potential for developing intelligent robots on a much smaller scale. These findings, published in the journal Science Robotics, could lead to further advancements in neuromorphic sensing and processing.

The drone's ability to accurately control its ego-motion and perform complex maneuvers autonomously highlights the promise of this technology for future applications.

DroneXL's Take

This breakthrough in neuromorphic technology represents a significant step forward in the field of robotics. By mimicking the brain's processing capabilities, researchers are developing more efficient and compact systems that could revolutionize the use of drones and other autonomous devices. As this technology continues to evolve, we can expect to see increasingly intelligent and capable robots that are both energy-efficient and effective in a variety of environments.


Discover more from DroneXL

Subscribe to get the latest posts to your email.

MAKE YOUR VOICE HEARD

Proposed legislation threatens your ability to use drones for fun, work, and safety. The Drone Advocacy Alliance is fighting to ensure your voice is heard in these critical policy discussions.Join us and tell your elected officials to protect your right to fly.

Drone Advocacy Alliance
TAKE ACTION NOW
Follow us on Google News!

Get your Part 107 Certificate

Pass the test and take to the skies with the Pilot Institute. We have helped thousands of people become airplane and commercial drone pilots. Our courses are designed by industry experts to help you pass FAA tests and achieve your dreams.

pilot institute dronexl

Copyright © DroneXL.co 2024. All rights reserved. The content, images, and intellectual property on this website are protected by copyright law. Reproduction or distribution of any material without prior written permission from DroneXL.co is strictly prohibited. For permissions and inquiries, please contact us first.

FTC: DroneXL.co is an Amazon Associate and uses affiliate links that can generate income from qualifying purchases. We do not sell, share, rent out, or spam your email.

Haye Kesteloo
Haye Kesteloo

Haye Kesteloo is the Editor in Chief and Founder of DroneXL.co, where he covers all drone-related news, DJI rumors and writes drone reviews, and EVXL.co, for all news related to electric vehicles. He is also a co-host of the PiXL Drone Show on YouTube and other podcast platforms. Haye can be reached at haye @ dronexl.co or @hayekesteloo.

Articles: 3477

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.