I bet ...

Image credit: VentureBeat with ChatGPT

Seems a tiny entity, Sapient, an outfit with connects to China, Singapore and California, has emulated The Mythical Man Month in terms of AI and coding by separating instructions from calc, thus reducing bloat and inefficiencies to the extreme while costing $1500 to build. Better yet, it's open source as they believe in free as in beer. The code is free but you pay for services. :)

Try their HRM code on Github for free.

Training a foundation LLM from scratch costs millions and requires internet-scale data — which is why most enterprises don't bother. Sapient thinks it has a cheaper path.

To overcome this brute-force scaling dogma, researchers at Sapient developed HRM-Text, which replaces standard Transformers with a highly sample-efficient Hierarchical Recurrent Model (HRM), an architecture they first introduced last year.

HRM decouples computation into slow-evolving strategic and fast-evolving execution layers. Instead of brute-force autoregressive prediction on raw text, HRM-Text trains exclusively on instruction-response pairs. This is close to real-world enterprise settings, where users usually expect a targeted answer to a specific task.

The researchers were able to train a 1B-parameter HRM-Text from scratch at a fraction of the cost and tokens of normal LLMs. Their model achieved performance competitive with much larger open models on key industry benchmarks.

For real-world AI applications, this means foundational pretraining is no longer restricted to highly resourced institutions. With HRM-Text, organizations can affordably pretrain their own highly capable reasoning models from scratch and pair them with external knowledge stores.

I bet the bond holders are thinking really hard about all the billions of dollars spent
to build AI and data centers that are eating the world as we speak. I know I would.