Gemini 1.5 Pro is the new leader in the field of generative artificial intelligence benchmarks.

On August 1, Google surreptitiously released an experimental version of its most recent model, surpassing the previous champion, OpenAI's ChatGPT-4o.

The most recent version of Gemini, which is presently classified as experimental, was released quietly. However, as word spread that it was outperforming its competitors on benchmark scores, it soon attracted the interest of the AI community on social media.

Benchmarks for artificial intelligence
Since GPT-3's release, OpenAI's ChatGPT has served as the industry standard for generative AI. For the past year or so, Anthropic's Claude-3 and its newest model, GPT-4o, have dominated the majority of other models in the majority of common benchmarks with minimal opposition.

The LMSYS Chatbot Arena is one of the most well-known benchmarks. After putting models through a battery of tests, it generates an overall competency score. Claude-3 scored a respectable 1,271, while GPT-4o received a score of 1,286.

Gemini 1.5 Pro, an earlier iteration, received 1,261. However, the August 1st release of the experimental version (Gemini 1.5 Pro 0801) earned an astounding 1,300.

This indicates that it’s overall more capable than its competitors, but benchmarks aren’t necessarily an accurate representation of what an AI model can and can’t do.

Excitement in the community
We're entering a time where the AI chatbot market has developed to the point where there are several solutions available, excluding further in-depth comparisons. Which AI model is most effective for a given user is ultimately up to them.

Anecdotally, social media users have been raving about the most recent version of Gemini, calling it "insanely good." It even "blows 4O out of the water," according to one Redditor.

Currently, it's unknown if Gemini 1.5 Pro's experimental version will eventually become the default. Although the model is still widely accessible as of the publication of this article, the fact that it is in the early release or testing phase suggests that it may be modified or withdrawn for alignment or safety concerns.

#Google #Technology #AI #ChatGPT #OpenAI