I truly believe AI content has some kind of 'survivor bias' in the sense when it is detected. When it is sloppy and terrible it's really noticeable, when it's well done or used partially with a human you won't even notice or realize it's AI.
This black/white view on content is for lazy people who do the laziest prompts and send slop without checking, editing, copywriting, improving anything.
One of the biggest disappointments I found on the human race with the release of useful LLMs is the propensity to have the lowest denominator behaviour possible, be as lazy as possible and let themselves be cognitively replaced by AI, if you put in the SAME effort we used to put in before writing content, using ai tools as leverage I have no doubt it would be an improvement, we are just comparing apples to oranges when you consider the effort dispensed to write.
It depends on what the writing actually is, of course, but in general -- I have zero interest in hearing what genAI has to say about anything. I'm very interested in what people have to say about things. The "quality" of the writing doesn't enter into it.
I think it depends on where you work at and your team.
If you come from a good team with high quality engineers around you, I imagine the experience is not as good, or if you work in a big company where on your daily job you just owe a very small slice of the whole pipeline.
I have worked on several companies where my colleagues were very mediocre on software engineering fundamentals (I'm in AI before ChatGPT a lot of Jupyter Notebook warriors [Data Scientists] having to put stuff in production for midsized companies) and very average engineers, who wouldn't really care implementing patterns, tests, linting, documentation etc...
At the risk of sounding arrogant I always wanted to be able to just 'clone' myself and keep my code discipline consistently on a project, I would push a high quality PR with good test coverage and the cleanest code I could muster, while my colleagues would approve between each other PRs with no test coverage, no linting, horrible technical debt or with work arounds around linting, making my effort moot as the slop overwhelmed any quality I tried to implement. Not just one company, 3 companies I have seen the same experience.
Also I was a full stack AI engineer in the fullest sense of the word, I did devops, deployment, proxy set up, ci/cd pipelines, backend, frontend and AI, on local servers, no external team to deploy for me, no external team to qa for me all myself, with great flexibility (my manager was completely non-technical) to try different stacks and use AI assisted coding where I forced i my AGENTS.md for codex to destroy every code, or feature I was writing as a Staff engineer, allowing me to learn a lot and upskill autonomously extremely quickly.
With this knowledge, I set up my side project where I locked in a good stack, applied solid patterns from the beginning, extensive static checks for code (extremely strict linting, testing and coverage checks) and my trust on AI grew with the project and I catered exactly to my way.
The initial commits I used it to write extremely clean code almost by hand, checking every line, locking in the stack, the linters, testing methodology, tdd methodology, etc, and I was slow but writing very clean code and setting up code examples.
Once I noticed my progress was going slow, and was noticing the AI was getting prompted by my clean code as it read the files to suggest the next lines, I started letting it go more autonomously but checking every line of code, I set up a lot of markdown files on how to test, how to code, code styling, the architecture and the patterns and would make it mandatory reading to the ai before starting, with the existing 'clean code' I got really good results, I just had to enforce the discipline that I could not enforce on my colleagues.
With the memory system in claude code and countless iterations of fixing AI mistakes and not doing stuff the way I liked it, I arrived by myself on a very similar position as Steve Yegge on me and Claude converging to a set of laws and strict way of working, and I just have a lead orchestrator agent and subagents, where I pick the issues and check every PR before merging, and merge by hand always.
This includes a specific cycle of each subagent taking a worktree, symlinking to the .env, checking if the subagent will use Playwright (spin up it's own server) and estimate the RAM memory usage (I have allocated 15GB for programming), forcing subagents to run tests with their own Haiku subagents to not pollute their context, the orchestrator scoping the implementer subagent to Sonnet or Opus depending on complexity, every subagent gets a separate reviewer that enforces the ARCHITECTURE.md and ticket goals and runs mutations on the tests done by the implementer subagents (they are forced to implement with TDD).
It is it's own ecosystem coding exactly how I like, without complaining. My initial clean code few shots the AI on keeping high code standards and there is naturally some smell, but my reviewer agents enforce separation of concerns very strictly, I know the smell is contained and if it becomes an issue on my application I can easily refactor it without breaking the rest, which I think is the most important aspect on agentic development, accept the tradeoff of the speed, and keep it contained.
For example on every issue, my orchestrator will assign the implementer subagent exactly which files it can touch, and explicitly which ones it cannot touch. The orchestrator knows what all the other subagents are doing, and this helps prevent conflicts when you have 5+ running in parallel.
It's a lot, but it's wonderful ONLY if you have full control on the whole process. Having to work under someone else process must be a nightmare. So it's a very mixed bag I would say. However for me after so many iterations and convergence I'm extremely pleased, and am a big advocate for it, but you need discipline, and a high skill ceiling on what you are doing, and know how to build things.
Isn't OpenAI burning billions and have billions more spending commitments? If they managed to downsize so much the cost they should have kept the price the same and become profitable, really weird move, unsure what led to this.
Slower grow, or even shrink in usage? Right now, the promise of a future "everyone will use our models and pay whatever we ask" is what keeps $$ flowing towards OpenAI.
More than $650B due 2030. I don’t understand how it makes any sense that they reduce the price so much, unless they expect seriously such a massive saving and increase in demand from their latest improvements?
For me the most pragmatic way to solve this is in the agent harness layer simply prohibiting commands like git force and others in the settings configuration of your harness of choice with pre tool hooks, its quite easy to setup, obvious ones are prohibiting pushes to main, among others.
Unless someone is building their harness from scratch like you this is a 30sec configuration and you could just save your list of blocked commands with wildcards in a file, and copy paste from one harness to the other.
Agreed. I had to solve this at the harness layer myself. It just makes more sense to me this way. We basically created hooks around these operations and you can configure what happens at these calls.
Disclosure: I'm building AQ (aq.dev) a multiplayer coding harness.
Most people don't understand that 'agentic AI' is a completely synthetic, force fed capability by extensive Reinforcement Learning on synthetic domain specific 'agentic' datasets on post training.
If the LLM wasn't post-trained to adhere to specific handbook, it just won't work. If the LLM wasn't trained on an use case the lab decided was worth making a synthetic agentic dataset, it won't work as well as you want.
There's a reason the main agentic task LLMs excel at are coding tasks, it's the way of working of the creators, and they understand intimately the flow and can train for it.
I believe the true way will be able to easily fine tune models on your agentic use cases, but it would require a big company to compile a huge dataset on it's way of working and I don't think anyone wants to be the first.
In terms of long context, accurate attention retrieval from early tokens is just impossible, given the expansion of RoPE encoding for the positions, or in case of Kimi that don't use it anymore, as well as deepseek, early context is heavily compressed you lose accurate information.
If people spent more time studying about AI and how it works, they would realize that the default should be to one shot prompt your task with a big, cached system prmopt, with an user prompt that is just dynamic data, specified to the cheapest model that can do the job.
Unless you really can't do this given your problem, you should try to make a graph of well defined, step by step oneshot prompts, and THEN if your problem still can't be solved with that, then you start leveraging agents.
Despite this giving better results, and being more cost efficient, is evidently too much work then just letting the AI do all the work.
> In terms of long context, accurate attention retrieval from early tokens is just impossible, given the expansion of RoPE encoding for the positions, or in case of Kimi that don't use it anymore, as well as deepseek, early context is heavily compressed you lose accurate information.
"Every gambler knows the secret to survivin' is knowing what to throw away, and knowing what to keep." - Kenny Rogers
Humans have limited context, just like AI. The difference is that humans - at least some of the time - can figure out which pieces are more likely to be important, and therefore prioritize keeping those in the context.
Most people jump straight to agents when what they actually need is a graph.
Example: a mining company receives free-text reports from field geologists. You could have:
Geologist report -> LLM call extracts minerals we are looking for (you inject a db query result on the user prompt), locations, assay mentions and risks into structured fields -> LLM call classifies evidence into positive indicators, negative indicators and unknowns -> LLM call estimates deposit potential and confidence -> database lookups inject regional ore demand, nearby deposits, infrastructure and historical yield data -> LLM call combines geological evidence with business context -> LLM call generates an investment recommendation and rationale.
That's what I mean by graph. Every step is a separate LLM call with a well-defined responsibility, consuming the output of the previous node. Each node can be tested, benchmarked, retrained, replaced, or monitored independently.
Why would you use an agent here? You can cache every single system prompt on each call making your total token output much cheaper than having a full 'output' only token generation workflow which is what happens with agents.
There is nothing to discover. The workflow is already known. The company already knows how geologists evaluate prospects. The company already knows what data sources matter. The company already knows what the final output should look like.
You don't want the model deciding which tools to call, which reasoning path to take, or which pieces of information are important every single run. You want the exact same process applied to every report so results are consistent, measurable, auditable and debuggable.
My default is:
One-shot prompt -> if not enough -> graph of LLM calls -> if not enough -> agent.
A surprising amount of enterprise AI is really just:
Unstructured input -> extraction -> classification -> enrichment from databases -> decision support.
Not:
Unstructured input -> autonomous agent spends 20 steps deciding what to do next.
Agents make sense when the workflow itself is unknown.
If the workflow is already understood, a graph is usually cheaper, more reliable, easier to evaluate, easier to debug, and less dependent on whatever synthetic "agentic" behaviors happened to get reinforced during post-training. I am sure people default to agents mostly because it's less engineering work than explicitly modeling the process.
Thanks, that helped. I get it. Yea, that’s the harness executing the workflow with the LLM being called at the right time. That ensures the process is consistent no matter what, in contrast to the agent calling out to tools and possibly doing different things every time. Basically, standard code being in control rather than the LLM being in control. Fully agree with this model. We should use deterministic code when we want the same process or algorithm every time and choose LLM callouts when we want “fuzzy” processing that is not as deterministic.
This is a great point, and one that has been blowing my mind for a while. The public think of AI as this black box thing approaching human intelligence. They don't know that when you look under the hood of a lot of ai products you see a string of prompts that anyone could put together themselves. The companies are just putting them together in a workflow.
The general public doesn't understand how these things work, at all, and the marketing makes it sound like it is magical.
That's why this paper is important. They gave it a scenario very similar to what would happen if an AI-frenzied executive suddenly mandated that AI be used to do basic company tasks - a scenario that is playing out all over the place right now.
I jump straight to graphs (unless it truly cannot be solved with a graph), but then the stakeholders get upset that I didn't use agents. Doesn't fit their marketing plan, I suppose. I don't mind dealing with that, but I can see why most jump to agents in order to avoid the human conflict.
Sure. You could easily have the model choose a workflow/graph for execution and then call a tool that implements the execution engine for that. The tool then makes calls for model inferencing at various points in the processing.
I called it a graph because really its a DAG (Directed Acyclic Graph) , in my example I kept it simple so there was only one input to every node, but in more advanced use cases you might have a synthesis node that takes in 2 or more inputs making it more akin to an execution graph in Tensorflow for example
I disagree with the final outcome that's just in private. If it's an open brainstorm to decide on solution, you don't have time to book the secret 1 to 1.
I think it's just respecting hierarchy, disagree, raise your concerns, if your boss overrules just accept it, you made your concerns known, they and the team heard of you, if they proceed anyways they accept the risk.
No point in forcing yourself to be the shot caller when it's not your job or responsibility to make the final decision.
Accepting the team consensus and respecting hierarchy is part of the game, unless you are a business owner, you are paid to do as told as an expert give your opinion. Nothing more you can do beyond that.
Seeing the results I don't see how the results are even comparable Opus is clearly far superior in most aspects. Smoothness, design, functionality etc.
At the end of the day, the time earned is more important then the cost for big players.
The ability to spawn 10 claude agents and rush a project to outcompete someone is more important for big businesses in my imo. Also the small details that GLM missed would take significant more time to iron out, considering it already took double the time.
I do hope other (open weight) models catch up, but to act like they are anywhere close for me is a bit disingenuous.
I believe also a big factor that it is way easier to convince Trump that AI is a matter of national security, and to use geopolitical tools (NVIDIA GPU ban on China) to secure their market position as much as possible, and to make it more palatable to the public their corporate bailouts.
What a lot of people seem to not understand about the stock market, is that at it's basis it's just a supply/demand ratio. When it goes down it means someone is selling a lot, someone is cashing in, at least converting it into cash.
For me it was obvious something was afoot with earnings and performance not matching the prices, I finally understand why now thanks to this article.
The fact that there are rules for institutional investors and retail investors and us in retail have so little visibility and time to keep up, just shows more and more the game is a david vs a goliath, and we are all slingless david.
It's more than just supply and demand. It's the price discovery. So I guess you can say supply and demand curves. The curves change with the market psychology and with future expectations.
Most institutional investors are not going to outperform your diversified portfolio. It's not like professionally managed funds are killing it while individual investors are losing. There are some specific examples of funds/people who do well but on average most do ... average.
Fully agree market psychology has a big influence in prices, TESLA is a great example of this.
My main point is that most people, including the media, whenever there is a big crash in prices, like silver going down double digits, they act like the money evaporated and everyone that invested lost money.
My point is that it's not the case, it dropped because there was a huge volume of people selling, making it cheaper. The people selling converted it all for liquidity, they just 'got' a lot of money in cash to spend, and they needed it or will use it for one reason to another.
Retail investors don't have the time (unless you work in finance) to read all the news and information to be aware of situations that will trigger liquidity crunches like these past few months, while institutional investors will.
My point here is you could have performed all of the value investing in the world and you are still eating losses, standard diversification theory is to put in gold when the markets are unstable, as it appreciates in time of high volatility, we are in times of extreme volatility and gold crashed, it makes no sense unless you have visibility in the institutional investing trends.
People lost money on paper. The loss turns real when they try to sell their assets.
Prices can drop on very low volumes. All that prices tell us is what someone agreed to buy and sell at a given point in time. Some (most?) sellers are likely selling because they are planning to buy when the price is lower (i.e. they are betting the market will go down) not because they need to use it.
Generally gold is not considered an investment or a hedge against marker instability and most diversified portfolios would not have gold in them.
Yes- if I own the S&P 500 and the S&P 500 goes down then the current value of my investment has gone down.
Disagree on many points, stocks are used as collateral for debt financing, their prices can definitely trigger cascade effects and losses even if not actually sold.
Overreaching arguments that sellers are like selling because they plan to buy when it's lower, no proof and a limited view, in fact in my also overreached argument I would say the opposite, most people just want to put money on an ETF and hold it until retirement, without having to touch it, they sell because something is forcing their hand and they need the liquidity to pay for something else.
Gold is definitely a hedge for inflation and market instability which is why it's had such a big run up these past few months, and they are definitely used in most diversified portfolios, yale fund as an example, (I don't know where you got this notion from)
You just realized pledging claims on paper with multiple degrees of seperation (stocks) for anything with a trigger mechanism, and then banking on it… is a terrible idea?
I've never understood why people separate some mystical magical "market psychology" from the traditional supply and demand model.
Like...what do y'all think demand is? From tulips and cabbage patch kids to meme coins and nfts, the price fluctuates based on semi flexible supply and highly flexible demand.
You could say "ah well, according to my analysis of the market psychology of this asset, I believe price will collapse in the medium term." but I just don't see how that's any more useful than saying "I believe demand will decrease soon" or "People aren't going to want this thing forever"
I broadly agree, but the place where psychology comes in in the continuous auction process. Just like a conventional auction, when prices start to go up, some people get overexcited and bid things up further, and conversely when people become terrified and stampede for the exits, selling assets at prices that are below any reasonable fundamental value. In the supply/demand model this is essentially the stock transitioning into being a Veblen good (where demand rises as price rises) and whatever the opposite of that is called, which is an interesting phenomenon and afaics purely psychological with no rational basis (unless you think greed/fear is rational I suppose)
That said, when people claim to distill the market psychology into a single recap or analyze the market psychology to predict future price, that's pretty much nonsense.
The prices for stocks, particularly growth stocks, are very future-oriented. It’s less about what they’re doing now and more about where you think they’ll be in a few years. When we’re all reading the tea leaves, it’s not surprising that investor vibes play a role.
Current sales figures are more closely based on what people want to buy now.
Those of us calling ourselves value investors love perturbations in the market out of line with the underlying business. That’s called opportunity. Eventually the market recognizes the value. So long as you’re not playing with options or trying to get quick rich you can get rich slowly.
I approach this by always asking Opus to send an agent to explore and trace how a pipeline works. Even better if I have an integration test. Once it's fully mapped out I might ask it to dump everything it discovered on a markdown doc, clear the context and start the task. The docs folder keeps the information intact for future development.
Managing context is by far the most important skill to be effective with LLMS, in addition to having already existing clean code on the codebase.
As they read your files, you are one shot training the LLM in how to write code and how you structure it and it will adapt. With clean codebases, I found the LLMs were outputting well documented, well logged, and even tested functions by default because the other files it interacted with were like this, 'it learns'.
Additionally you have to think how they train and evaluate the model, there are so many use cases to cover, I'm pretty sure in the Reinforcement Learning part they are not going in huge long threads, but are actually benchmarking and optimizing from fresh context starts, and you should do that as much as possible in your tasks.
This black/white view on content is for lazy people who do the laziest prompts and send slop without checking, editing, copywriting, improving anything.
One of the biggest disappointments I found on the human race with the release of useful LLMs is the propensity to have the lowest denominator behaviour possible, be as lazy as possible and let themselves be cognitively replaced by AI, if you put in the SAME effort we used to put in before writing content, using ai tools as leverage I have no doubt it would be an improvement, we are just comparing apples to oranges when you consider the effort dispensed to write.
reply