Meta’s new AI can play Diplomacy better than most humans

Researchers at Meta have created an artificial intelligence capable of playing Diplomacy and beating most human players.

As explained on the Meta AI blog, Diplomacy has been considered a “nearly impossible grand challenge in AI” for decades because of the skill set required to play the game. The goal of Diplomacy is to control most of the board, but for this players negotiate with each other through conversations, form alliances, make deals, and detect when other players are bluffing.

Meta rose to the challenge by building and training an AI called CICERO, which is included in top 10% of Diplomacy players who have played more than one game (on webDiplomacy.net). This was achieved thanks to the combination of two different areas of AI research: strategic thinking and natural language processing.

AI is formed from a guided dialogue model and a strategic thinking engine that work together. This allows CICERO to predict the moves other players might make, as well as the moves those players think CICERO will make (strategic reasoning). CICERO then uses a set of carefully selected plans on which it builds its conversations with other players to “negotiate, offer advice, share information, and make agreements” (natural language processing).

AI capable of beating the best players at games such as chess, poker, and Go are trained using self-play reinforcement learning. As Diplomacy requires cooperation with human players, that method doesn’t work. Typically, supervised learning is used for such games, but Meta says that creates a”relatively weak and highly exploitable” opponent. Instead, Meta used an iterative planning algorithm that “balances dialogue consistency with rationality.”

CICERO is far from perfect and can generate inconsistent dialogue, but it’s good enough to seriously challenge even the best players. However, Meta believes that the technology developed during the creation of this AI can be used in other areas. For example, allowing voice assistants to have long conversations with us and teach us new skills. In addition, such AI can be used in video games to play the role of realistic characters that are difficult to distinguish from human players.