TUM Professor Uses ChatGPT to Choreograph Drones

Researchers at the Technical University of Munich (TUM) have successfully integrated ChatGPT, a large language model, into robotics for the first time. Professor Angela Schoellig and her team have developed a system that allows ChatGPT to command drones to perform choreographed flights without colliding, opening up new possibilities for human-robot interaction, reports Interesting Engineering.

Drones Dance to Music with ChatGPT

In Professor Schoellig’s lab, a doctoral student demonstrated the capabilities of the system by selecting a music track and entering text, which resulted in six drones performing a choreographed flight. The computer calculates the feasibility of the flight paths before the drones take off, ensuring a smooth performance.

The lab is equipped with six ceiling-mounted cameras in a room measuring approximately 1,076 square feet and 9.8 feet high. The cameras detect the position of the quadrocopters, which are equipped with four propellers and motors, 200 times per second, comparing them with the desired position.

Safety First: Avoiding Mid-Air Collisions

To ensure the safety of the drones during the choreography, the research team had to incorporate a special safety feature. Without this feature, only one in four demonstrations would be accident-free. The safety algorithm maps out precise flight paths for the proposed choreography, completely avoiding mid-air collisions.

SwarmGPT: An Interface Between Robot and Human

Professor Schoellig has coined the term “SwarmGPT” for the adjusted computer software that enables ChatGPT to command the drones. This functionality serves as an interface between the robots and humans, allowing anyone to operate the drones without the need for a remote control or specialized training.

Rapid Advancements in Drone Choreography

Professor Schoellig, who has been working on Drone Research for nearly 15 years, notes the significant progress made in drone choreography thanks to ChatGPT. While it previously took over three years to develop six choreographies for six drones, the team has programmed 30 choreographies for nine drones in just three months since integrating ChatGPT.

Future Applications and Scalability

Although the drones currently operate in a controlled environment without external factors like weather patterns, this demonstration serves as a starting point for exploring the potential of ChatGPT in robotics. Professor Schoellig believes that the concept is scalable and could be applied to command robots to perform other actions, such as opening doors or picking up objects, although these tasks may be more challenging to program.

The successful integration of ChatGPT into robotics by Professor Angela Schoellig and her team at TUM marks a significant milestone in human-robot interaction. By enabling drones to perform choreographed flights based on text input and music, the researchers have demonstrated the potential for large language models to revolutionize the way we command and interact with robots. As this technology continues to develop, it may open up new possibilities for robots to assist humans in various tasks and environments.