Microsoft researchers plan to control robots

Microsoft has expanded its ChatGPT AI system to control robots based on speech. Researchers in the Autonomous Systems and Robotics Research Group want to use human speech as the most intuitive way to control robots and find out if ChatGPT is capable of solving robotics problems in addition to generating text. With this technology, users can intuitively control robots without needing deep knowledge in robotics.

ChatGPT can not only generate text, but also write computer code. Microsoft is now using ChatGPT to control robotic arms, drones and household assistants by voice. Previously, robotics engineers had to develop complex code structures to control robots. In the future, AI control using ChatGPT may make it possible without deep knowledge in robotics to control the robot based on feedback loops with voice AI.

To train ChatGPT, the researchers created a set of design principles to help the speech AI solve robotic tasks. These include specific language constructs, APIs, and feedback capabilities. The researchers believe that speech AI-based robotic controls will play an important role in the future in making robots useful for everyday life and a broad public.

However, the team stresses that precautions should be taken before ChatGPT is used with real robots. ChatGPT-based control can only write the computer code for the robot based on a text input from the human user. It is therefore necessary for a human engineer to explain in detail to ChatGPT how the robot's interface works so that the AI program can create the appropriate computer code.

The Microsoft researchers have therefore created some guidelines to write effective text input for ChatGPT when it comes to controlling robots. The team has also created an open-source platform on GitHub where anyone can share examples of text input for controlling different categories of robots.

The researchers emphasize that ChatGPT-based controls will play an important role in robotics in the future, but safety precautions must be taken before they can be used. ChatGPT must not take full control of the robotics pipeline, especially for safety-critical applications. It is important to ensure that the quality and safety of the code is assured with human supervision before it is run on the robot.