This is ironically a pretty solid use case for (ex VLIW research) ILP-optimizing compilers.
Given knowable runtime hardware usage patterns (huge bursts of memory bandwidth saturation) and a single limited core/thread-shared resource (memory bandwidth), one could optimize for the constraint ahead of runtime.
Because most of the performance optimization levers you have available to pull are (a) trade compute for memory bandwidth (e.g. compression), (b) preload when memory bandwidth is available, (c) optimize the choice of what's in cache when, (d) align to cache size / memory boundaries.
Or tl;dr, try to approximate GPU ISAs at the CPU compiler level. (Which why would anyone but hobbyists, because everyone else just buys pallets of Nvidia/AMD or designs their own ML chips?)
Under appreciated requirement for this to work in post-cloud times: open source
If a vendor can SaaS a solution, then enterprise is generally happy (they don't want to have to hire folks for maintenance), and that completely locks out any ability to run locally.
Between enterprise's ambivalence and the obvious financial incentive to vendors, you get SaaS-only products.
The problem is that both camps take their positions as religious righteousness, which lobotomizes their abilities to have productive, pros and cons discussions about matters at hand.
The internet/apps of the last 20 years have not exactly boosted people's ability to think critically and set aside their passions though.
Much easier to keep eyeballs glued and sell them ads if you encourage their baser impulses.
Everyone is susceptible to addictions or psychosis to some degree.
What matters is when the stimulus presented exceeds their resistance.
Extended AI use is a highly attractive stimulus that exceeds most people's resistance, especially when sycophantically interacted with in an echo chamber (human-AI, with no other humans in the room).
So yes, it's dangerous in the same way that cigarettes and social media are.
Just because some people can avoid slipping into it, doesn't mean we should ignore population-as-a-whole outcomes.
> We support the use of AI for lawful foreign intelligence and counterintelligence missions. But using these systems for mass domestic surveillance is incompatible with democratic values.
> Claude Code's biggest advantage isn't Opus, rather it's the shared knowledge the community has been building and sharing around using it effectively.
This. Never underestimate the ability of a large number of power users to substantially improve the actual utility of a complex software product.
They always have more time (and sometimes more skill) than a product's developers.
Sometimes the quantity of monkeys matters more than the quality of the typewriters.
2010s Javascript, putting down the controller: Ha, no one will ever surpass my high score for wasting programmer time with dependency churn...
2026 Open Source ML: Hold my beer.
reply