Every product or service evolves from the conception of an idea to an exclusive offering for a select few, eventually becoming a public utility.

The internet followed a similar trajectory, transitioning from an unstable and chaotic environment to a stable and resilient high-speed broadband ecosystem. AI is no exception. Today, web pages load almost instantly, images appear in moments, and platforms like Netflix can stream entire movies without issue.

Few remember the unstable origins of early internet connections.AI is undergoing a similar phase, akin to the broadband internet era, but progressing at a much faster pace in terms of speed and performance. In the next two years, enterprise AI adoption is expected to reach its maturity phase.

To understand this, let’s consider some basic economic aspects of LLMs. A common performance metric for an LLM is text processing speed, measured in "tokens per second." A token can be a word, part of a word, or even punctuation—it’s the basic unit of data (text) that an LLM processes.

Now, imagine a sentence as a chain of tokens that the LLM must read, comprehend, and respond to.In AI’s current evolutionary phase, processing speeds range from 50 to 100 tokens per second. This is sufficient for basic chatbot conversations or simple questions, but things slow down significantly as tasks become more complex.However, there are early signs that the era of mature AI is not far off.

A few months ago, Groq, a chip startup, announced reported speeds of up to 800 tokens per second.What does this mean for AI? With the ability to process more tokens at faster speeds, language models can handle increasingly complex tasks in real time. They can now analyze large amounts of text instantly, enabling outputs with higher levels of sophistication.

Information sourced from Sangeet Choudary.

4o