Kimi K3.1 Pro, a model from Beijing-based Moonshot AI, debuted with notable benchmark results, ranking fourth on Artificial Analysis' Intelligence Index v4 and first on its Terminal Bench 4.0 for computer use. On Tuesday, Artificial Analysis announced Intelligence Index v5, which it described as its largest upgrade since v4 launched in January. After the update, Kimi K3.1 Pro fell from fourth to ninth place.
According to Artificial Analysis, 60% of the new test involves private or held-out questions, an approach aimed at limiting benchmark preparation. Artificial Analysis stated that with the new index, the open weights gap widens from 3 points to 12. Moonshot had not released the model's weights at the time of the ranking and was not due to until November 4.
Google's Gemini 4 Carbon became the top-ranked model on Intelligence Index v5, scoring 59. Gemini 4 Carbon was evaluated but not released when it was announced on Wednesday, and Google featured its top ranking in its blog post about the model.
Alternative explanations for the performance drop among Chinese models have been offered. Physicist Stephen Hsu wrote on X that he believed U.S. models were outmatching them in math and science. The article notes that Artificial Analysis is a private firm, not a standardizing body or academic institution, and appears to have been founded in 2023.