If you follow the mainstream AI giants, their development, trajectories, promises and narratives are all about being bigger and better. This linear brute-force concept of AI development is bound to take a hit.
The parallel improvements will come from the fundamental behaviour of how the pre-trained transformers work in terms of the core architecture of generative engines, along with how these engines use the processing and memory units.
The near-future is about efficient distribution of contextual tokens in layers and clusters, so that a few lines of prompt do not need to fetch a demigod for a response. Once these changes take effect sooner, the brute-force necessity of bigger, faster GPUs is bound to come down significantly.
Meanwhile, the balloons of building an AGI should go burst. The reason is very simple, we do not need an AGI or a super-intelligent human-like AI. We need useful tools, contained and embedded solutions in everything we build, at home, at schools, at offices. This requires small, efficient models, focusing on specific work.
We don’t need one elephant to generate our next month’s report, rather we need a fast-moving, highly efficient swarm of house-flies to help us. This also means, the rise of containerised and decentralised small specialised models, not an AGI behemoth. My weekly meeting note-taker does not need to know the equation for the Big Bang, nor does it need to run a big LLM in the background. Which will also reduce the ever-expanding boundaries of where our data travels, and who gets access to it.
If it heads toward the “right” direction, if enough open-source community movements heat the pots, then there is a big possibility that the AI technology will be releasing itself from the grips of western tech giants, and it will increasingly become open and free, transparent yet containerised, governed by local regulations for better privacy and protection.
