Google Deepmind has announced a brand new AI model, Gemma 2 2B. This is a 2 billion-parameter model that offers GPT 3.5-level performance while being small enough to fit on a smartphone.

This announcement follows the Gemma 2 9B and 27B announced in June, both of which represent Google's counterattack against rivals in the AI LLM space, particularly Meta and its Llama 3.1 family.

The new Gemma 2B model is more evidence of a deliberate shift toward smaller, lighter models that can run on a wider range of devices, not just powerful computers.

Although the new LLM is a fraction of the physical size of previous models such as OpenAI's GPT-3.5 and GPT-4, its performance in testing reflects the massive improvements in training and packaging currently offered.

According to Google, the new model "delivers best-in-class performance for its size, outperforming other open models in its category," and the LYMSYS charts certainly show impressive statistics.

The model seems to achieve this performance improvement "by learning from larger models through distillation." This technique, introduced in a 2015 paper by Jeffrey Hinton, employs a sophisticated student-teacher learning method that allows for smaller, compressed models.

The market for mobile-based and edge-based Al used in small portable and static devices such as sensors is expected to grow exponentially in the coming years. This is especially true for multimodal AI applications that use vision and hearing to create a rich interactive world for the user. Hence, there is a race to perfect smaller, more powerful systems as quickly as possible.

However, despite the talk of running this type of AI locally on smartphones (aka "on-device"), the reality is that few portable devices are yet capable of handling the current LLM processing demands.

At least some of the latest iPhones running IOS 14 and later Android phones using Snapdragon processors, such as the Galaxy S23 Ultra, can handle it. But even then, performance is modest compared to computer-based systems, and since Apple's Apple Intelligence is mostly on-device, at least an iPhone 15 is required.

Thus, for most people, cloud-based Al is likely to remain dominant, at least in the short term. However, as the technology matures, on-device AI will inevitably take a larger piece of the chatbot pie. Especially as users begin to value privacy, security, and trust more and more.

Interested readers can play with the new Gemma 2 2B model in the Google AI Studio playground.