Google’s Gemini 1.5 Pro Will Have 2 Million Tokens. Here’s What That Means

In the world of large language patterns, the technology behind artificial intelligence, size matters. And Google said it’s letting users feed its Gemini 1.5 Pro model more data than ever before.

During Google I/O developer conference on Tuesday, Alphabet CEO Sundar Pichai said Google is growing The context window of Gemini 1.5 Pro from 1 million to 2 million tokens. Pichai said the update would be released to developers in a “private preview” but did not say when it might be more widely available.

“It’s amazing to look back and see how much progress we’ve made in a few months,” Pichai said after announcing that Google was doubling down Gemini 1.5 Pro context window. “And this represents the next step in our journey towards the ultimate goal of infinite context.”

Large language patternsor LLMs like Gemini 1.5 Pro, are AI models that are trained on massive amounts of data to understand language, so tools like Gemini — a competitor to the search giant ChatGPT — can generate content that people can understand.

Doubling the Gemini 1.5 Pro context window from 1 million to 2 million tokens can significantly improve the results you get from Google’s LLM. But tokens, context windows, and other AI jargon is decidedly nebulous. And without some of that context, which Pichai was so interested in discussing, it can be hard to understand why 2 million tokens is such a big deal.

Read on for an example of tokens and how increasing the number can change the way you use and interact with Gemini going forward. And for more information on Gemini and other AI tools like ChatGPT, Microsoft Copilot, perplexity and Claude as well as news, tips, and explanations on all things AI, check out CNET AI atlas resource.

What are Tokens in AI?

In AI, tokens are parts of words which LLM evaluates to understand the broader context of a request. Each token consists of four English characters. These characters can be letters and numbers, of course, but also spaces, special characters, and more. It is also important to note that the length of an individual token will vary by language.

As AI models add the ability to analyze images, video and audio, they similarly use tokens to get the full picture. If you input an image into a context model, the AI ​​models will split the image into parts, each part representing tokens.

Tokens are used as both inputs and outputs. So when users enter a query into an AI model, the model itself breaks the words into tokens, parses them, and delivers a response in tokens, which are then converted into words that humans understand.

OpenAI, the company that owns ChatGPT, offers convenient example to understand tokens. Have you ever heard Wayne Gretzky’s famous quote, “You miss 100% of the shots you don’t take?” This sentence consists of 11 lexemes. If you replace the percent symbol with the word percent, the number of tokens increases to 13 tokens.

If you’re interested in seeing how many tokens your text consists of, take a look OpenAI’s Tokenizer toolwhich allows you to enter text and see how many tokens it uses.

It is important to understand how many tokens are contained in each word or sentence. The more tokens available in a context window, the more data you can enter in a query, and the more data the AI ​​model will understand and use to provide results.

What does the context window do?

No token conversation is complete without an explanation of the context window. Really, it’s in the context window where tokens are used – and that’s what matters most.

Think of the context window as the length of your memory. The larger the context window, the more memory you have access to understand what someone is saying and respond appropriately. Popups help AI models remember information and reuse it to deliver better results to users. The bigger the context windows (meaning the more tokens it can use in dialog with users), the better the results.

“You may have had an experience where a chatbot ‘forgot’ information after a few spins,” Google wrote blog post earlier this year. “This is where long context windows can help.”

Why would it be better to have more tokens?

So why are more tokens better? It all comes down to simple math.

The more tokens a context window can accept, the more data you can enter into a model. The more data you can input, the more information the AI ​​model can use to provide answers. The better the answers, the more valuable the experience of using an AI model.

Think of it this way: If you want to get a summary of an important moment in world history, just giving an AI model a sentence to digest and provide a summary wouldn’t be all that useful. But imagine giving him a whole book about the event and the superb result you will get. The latter case is only possible with more tokens.

When will the updated Google context window be available?

Google’s updated context window only runs on its Gemini 1.5 Pro model for now. Pichai said it will first be available to developers in a “private preview,” with Google revealing later during the I/O event that it will be released “later this year.” So stay tuned.

What is infinite context and when will we get to it?

Pichai mentioned a future where we will reach “infinite context,” a point where LLMs will be able to ingest and output an infinite amount of data, effectively giving them access to all the world’s knowledge to deliver superior results. But honestly, we’re not close.

One of the problems with token increments is that it takes more computing power with each increment. And while infinite context is indeed something AI proponents are looking forward to, no one can say for sure when, or even if, computing power will reach a level where this is possible.

in blog post in February, Google highlighted how Gemini 1.5 Pro supported 1 million tokens at the time. And while the company admitted it was working on expanding context windows, at the time its research was only able to achieve a context window of 10 million tokens – far from infinite.

However, as you continue to use AI models, expect popups to increase not only from Google, but also from other vendors. And along the way, enjoy the better results that expanded token availability makes possible.

Editor’s note: CNET used an AI engine to help create a handful of stories. Reviews of AI products like this, just like Other CNET hands-on reviews, are written by our human team of in-house experts. For more see CNET’s AI policy and how we test AI.