The Gemini-powered Google Bard matched ChatGPT's performance in the popular chatbot arena, placing second on the leaderboard behind OpenAI's most advanced model, the GPT-4-Turbo.

Powered by a newly updated version of the new Gemini Pro artificial intelligence model, Bard has seen a marked improvement in performance since the model's release in December; in my tests comparing Bard to the free version of ChatGPT, Bard prevailed.

The Large Model Systems Organization (LMSYS) Chatbot Arena pits key AI models against each other with humans judging and evaluating their output. Closed-source models such as GPT-4 and Gemini Pro versus Meta's open-source AI like Llama 2

This is the first time Bard has beaten the basic version of GPT-4, which drives premium versions of both ChatGPT and Microsoft Copilot. Jeff Dean, chief scientist at Google DeepMind, writes in X that the recent success is due to a new version of Gemini Pro called "scale."

As in any battlefield, the chatbot arena brings two models into the ring and puts them head-to-head.

You write a prompt, it is sent to two anonymous models, who, after their answers are displayed, must decide which of the two has the better answer.

Only the underlying system knows which model won each round, and the collective human judgment is used to create the leaderboard. To date, more than 200,000 votes have been cast.

Gemini Pro is ranked #9 on its own as an API, not as part of the Bardo with live Internet access, and an earlier version of Gemini Pro is ranked #12 on the current leaderboard.

While Bard with Gemini Pro is doing well at #2, all three of the top five are versions of OpenAI's GPT-4 model, with GPT-4-Turbo taking the top spot.

Nine of the top 10 models are proprietary, with French startup Mistral's Mixtra-8x7b being the only open-source large language model in the top table area.

Anthropic's Claude and Mistral's Mistral Medium are proprietary versions of the Mixtral model and the only models outside of Google and OpenAI in the top 10.

According to Jeff Dean, Bard now "carries the Gemini Pro Scale model, which is an updated base version of the Gemini Pro, and this change has allowed it to move into second place. It is still based on a mid-level version of LLM's Gemini family.

Google is said to be releasing its most advanced and native multimodal model, the Gemini Ultra, at some point this year.

This will drive a new premium version of the chatbot called Bard Advanced, and presumably this will significantly surpass the event GPT-4-Turbo.

Dean writes in X: "The race is heating up like never before! I look forward to the next release of the Bard + Gemini Ultra."