Eight AIs, including OpenAI's o3 and o4-mini, Deepseek R1, Grok 4, Gemini 2.5 Flash and 2.5 Pro, Kimi K2, and Claude 4 Opus, competed in the tournament. Deepseek and Kimi are Chinese products, while the others are American.

The AI "players" competed in a single-elimination format. The quarterfinals began at midnight on Wednesday, August 7th, Hanoi time. Any LLM making four consecutive illegal moves was automatically forfeited. The tournament was held on Google's new online arena, Kaggle.

Before the tournament, the AIs underwent skill assessments to determine their ratings and seedings for match pairings. Kaggle also evaluated the AIs' abilities using an Elo-like rating system, allowing viewers to compare their chess prowess.

8 AIs participate in a chess tournament exclusively for LLMs. Photo: Chess

8 AIs participate in a chess tournament exclusively for LLMs. Photo: Chess

Each match consisted of four games. A win earned 1 point, a draw 0.5 points, and a loss zero points. The AI reaching 2.5 points or more advanced. A 2-2 tie resulted in an armageddon-style tiebreaker, where White had to win to proceed.

World number two Hikaru Nakamura provided live commentary throughout the tournament. World number one Magnus Carlsen also commented on some matches.

The event generated considerable excitement. On the Chess platform, user GavinSuckAtChess commented, "There are definitely going to be a lot of illegal moves".

User real_jaya predicted, "I'm betting on a dominant victory for the Gemini representatives. They performed exceptionally well in previous Pokemon games".

LLMs differ from specialized chess engines like Stockfish or Leela Chess Zero (Lc0). These engines' Elo ratings far surpass those of top human players (over 3,600 compared to 2,800). Nakamura, even with a piece advantage, suffered a significant defeat against Lc0 in an exhibition match. Lc0 can also be considered an AI, having learned chess by playing millions of games against itself.

LLMs are a type of artificial intelligence (AI) primarily designed for language processing, translation, and content creation. Users interact with LLMs through chatbots, such as OpenAI's ChatGPT. Technically, LLMs operate by "predicting the next word".

In chess, LLMs seem to apply a similar approach to their moves. This explains their proficiency in openings and ability to avoid early traps. However, in the middle and endgame, they are more prone to errors and illegal moves. ChatGPT famously lost to Carlsen without capturing a single piece.

Compiled by Xuan Binh