An old Atari 2600 game console beat ChatGPT at chess

Large AI language models have very little in common with real intelligence. This is proven by an experiment conducted by Citrix engineer Robert J. Caruso. He made ChatGPT compete in chess with an old Atari 2600 game console (via the Stella emulator). The Atari 2600 won. The engineer writes about this in his post on LinkedIn.

"ChatGPT got absolutely wrecked on the beginner level. This was after a conversation we had regarding the history of AI in Chess which led to it volunteering to play Atari Chess. It wanted to find out how quickly it could beat a game that only thinks 1-2 moves ahead on a 1.19 MHz CPU," writes Robert J. Caruso.

"Despite being given a baseline board layout to identify pieces, ChatGPT confused rooks for bishops, missed pawn forks, and repeatedly lost track of where pieces were — first blaming the Atari icons as too abstract to recognize, then faring no better even after switching to standard chess notation. It made enough blunders to get laughed out of a 3rd grade chess club."

"Meanwhile, Atari’s humble 8-bit engine just did its thing. No language model. No flash. Just brute-force board evaluation and 1977 stubbornness."

"For 90 minutes, I had to stop it from making awful moves and correct its board awareness multiple times per turn. It kept promising it would improve "if we just started over." Eventually, even ChatGPT knew it was beat — and conceded with its head hung low," the engineer concludes.

Recall that the Atari 2600 game console was produced from 1977 to 1992. It had a MOS Technology 6507 processor with a frequency of 1.19 MHz, 128 bytes of RAM (just a byte, not even a kilobyte). Games were supplied on cartridges with 4 kB ROM. A total of 30 million Atari 2600 consoles were sold.

Chess for the Atari 2600 – Video Chess was created by programmers Larry Wagner and Bob Whitehead in 1979. Both programmers later worked at Activision.