FRIDA robotic arm brings DALL-E-style AI art to real canvases

FRIDA (Framework and Robotics Initiative for Developing Arts) is not so much a research project as an art experiment. Of course, it faces issues of art and creativity, just like DALL-E and ChatGPT – although, probably, the question is even more acute when the manipulator robot paints on a real painting canvas.

However, the project of the Institute of Robotics of Carnegie Mellon University has certain advantages over software systems of artificial intelligence.

“FRIDA is a project exploring the intersection of human and robotic creativity,” says CMU professor Jim McCann. “FRIDA is using the kind of AI models that have been developed to do things like caption images and understand scene content and applying it to this artistic generative problem.”

The system currently requires some input, including textual descriptions and images, although it can work a little more abstractly. In one case, the team turned on the all-time hit – Dancing Queen by ABBA.

The team is also trying to cut off the issue of job automation. On the one hand, people are needed not only to introduce materials, but also to mix paint (although it is quite realistic to assume that this can be automated). The team believes that this is an ideal option for collaboration.

Perhaps the most interesting part of this is the inaccuracy of the system. The goal of such robotics, as a rule, is to achieve maximum accuracy. Here, however, the system is allowed to make mistakes and adjusts the rest of the picture accordingly, using an overhead camera to monitor its own progress. Speed is also not a priority. Work on each painting lasts for hours.