The True Driver of Today's Robotics

Software is becoming the soul of the machine

While robotics is typically associated with hardware, which enables interaction with reality, it is the software that defines a robot's capabilities, behaviour, and value. Advanced algorithms and data processing embedded within lines of code allow robots to learn, adapt, and perform autonomously.

Complex robot made of simple parts. Image created by OpenAI's DALL·E

Lego robots serve as an excellent educational tool, demonstrating how sophisticated software can create complex behaviors using simple parts. In contrast, even hardware as advanced as a spaceship can fail in unpredictable environments without the appropriate intelligence. Russia’s Luna 25 mission ended in failure on August 19, 2023 because the engine, which was supposed to correct the course and lead the spacecraft to pre-landing orbit, did not shut down normally. The control unit failed to turn the engines off because it wasn’t receiving the necessary data from one of Luna-25’s accelerometers which was not turned on, “due to the possible entry into one data array of commands with different priorities for their execution by the device (sic).” As a result, instead of the planned 84 seconds, the engine worked for 127 seconds.

This approach to space missions reminds the first landings on the Moon which were automated but relied on “hard-coded” algorithms, where decision-making by the spaceship is limited by a short list of options. To our estimates, only 34% of Luna Program missions were successful. Human-caused code errors, along with inadequate initial data and limited flexibility, lead to significant risks due to unforeseen and overlooked scenarios.

Luna 25 crash. Image created by OpenAI's DALL·E

But another, more flexible approach is possible, where Artificial Intelligence (AI) can help to land on the Moon. India’s Chandrayaan-3 mission made a historic landing on the Lunar South Pole on August 23, 2023. This achievement not only made India the first country to reach the lunar south pole but also the fourth nation in the world to successfully perform a controlled landing on the Moon.

AI played a significant role in facilitating this mission. The AI-powered sensors were securing the touchdown of the lander on the lunar surface. These sensors assisted the Lander Hazard Detection and Avoidance Camera to anticipate the lunar topography for any kind of hurdles or obstacles that could pose a risk during the soft landing. 

Unlike hard-coded algorithms, the AI systems were pre-trained with the ability to adjust to unexpected conditions in real-time, drawing on past data to guide their decisions. Also, their proficiency in detecting intricate patterns in images of the landing area, was key to steering the lander's descent, minimizing dangers, and securing a safe landing. The AI algorithms are also used during the lunar exploration phase of the rover, Pragyan.

Can a computer running AI software be classified as a robot?

As we consider the incredible success of India's AI-assisted moon landing, it raises an intriguing question: Can we consider a computer running AI software a robot? The difference comes down to interaction. PCs might be smart - they can run complex programs and have some basic senses like microphones and webcams -  yet they lack the ability to interact with or alter their surroundings through the dynamic process of feedback loops. Robots, on the other hand, are all about action. They combine the brains of advanced software with the 'muscles' and 'senses' to interact with the world around them.

But when we talk about the software that brings robots to life, it's also not just your everyday app or program enabled to interact with the real world. Robots require embedded real-time software. Embedded software operates with constrained processing power and memory. It operates behind the scenes, without direct user interaction. Moreover, real-time software is bound by stringent temporal constraints—it must ensure results are not only logically accurate (logical correctness) but also timely, maintaining temporal determinism (temporal correctness). 

It's the difference between just being fast and being punctual—robot software has to get things done at exactly the right moment. In the transition from embedded to real-time software, as Luna 25’s failure demonstrates, there are many topics to pay attention to, including Worst-Case Execution Time (WCET, as opposed to more common outside robotics average execution time), stochastic hardware performance, schedulers, inter-process communication, task priorities, interrupts, timers, and preemption.

Three waves

The first wave of robotics was characterized by structured manipulation in industrial settings, where robots were hardwired and purpose-built. Back then, the focus was on building better and smarter parts—think robot arms that could move in lots of ways, batteries that lasted days, and all kinds of sensors, cameras, and GPS to see and feel their surroundings. And, of course, powerful processors. We've come a long way with this hardware, making some amazing tools. These robots are built for precision and equipped with embedded real-time software but throw a curveball at them, and they'd be stuck. Now, we're pretty much on a plateau in that space. These days, the real action isn't in the hardware — it's not a bottleneck anymore.

The evolution of robotics into the second wave transcends mere mechanical enhancements. It's the dawn of ushering robots into the realms of uncertainty, an era where AI is indispensable. Interestingly, AI's necessity in robotics isn't for traditionally 'intelligent' work like chess or medical diagnoses, but rather for seemingly trivial tasks like walking, seeing, or even making a pizza — activities that are surprisingly complex to automate. This second wave of robotics harnesses machine learning and reasoning under uncertainty, challenging our preconceived notions of automation and intelligence, sometimes with the cheapest hardware.

Consider Abagy, whose AI-driven robots exemplify this shift. With advanced image recognition and AI algorithms, they make on-the-fly decisions. Abagy's approach makes it possible to transition from one product to another in manufacturing, which is especially useful in non-serial, high-mix production environments. This flexibility is a significant advantage because it eliminates the need for extensive reprogramming when switching between products, which is often a time-consuming and technical challenge in traditional manufacturing settings.

In this context, the robots of the second wave with their advanced, adaptable AI, claim the true title of 'robots', leaving the machines of the first wave as sophisticated, yet limited automata. It is within this paradigm that we see the full expression of robots as entities that not only perform tasks but adapt, learn, and navigate the unpredictable real world.

Think also about a pizza-topping robot as a fun example. Imagine you come up with a zesty new pizza topping combo. You won't need to bother reprogramming the robot for each new ingredient variation. It's smart enough to work out the right portions of sauce and the perfect timing to distribute ingredients evenly. Such robots already exist! Take XPizza Cube as a good example. It appears that using computer vision & AI in such robots enables the use of cheaper mechanics, as it provides stable control with fewer degrees of freedom.

This pizza bot is pretty much like a dramatically simplified version of a self-driving Tesla. It takes in the sensory data and adjusts controls on the fly to successfully achieve the stated objective. Coders don't need to map out every turn, road junction and every possible combination of cars on the road.

As we get further, we see that advances in fields that are unrelated at first glance become the real game-changers in robotics. Then, what's next on the horizon? We see great potential for Large Language Models (LLMs). Mostly harnessed for tasks like language comprehension and image recognition, LLMs could step into a more central role, becoming akin to the operating system of a robot's brain. Their ability to handle tasks and motion planning is becoming increasingly useful. With each new situation or environment a robot encounters, LLMs could reprogram the robot's actions, ensuring precise temporal correctness. We're gearing up for what you might call the third wave, where it's not just about handling unknown situations in a familiar environment. It's about robots facing new concepts and new environments head-on, equipped with something like reasoning and common sense.

Right now, it's tough to use LLMs and other AI models in real-world systems because of latency, context size limitations, and compute costs. But using them for low-level control could give us a cool preview of how we might teach machines to turn language patterns into real actions. This isn't just your regular AI; it's about giving robots a touch of real-world savvy. It's a bit hyperbolic to say gears and hardware will become secondary. They're still the backbone of any robot. Application-Specific Integrated Circuit (ASIC) like neuromorphic chips, could really speed up advancements in robotics. However, it's clear that the rise of AI is what's driving the need for these chips, not the other way around.