Well, that sucks. Replacing the harness with something task-specific has proven very powerful in my usecases.
But you should correct: Claude is very happy to let you use whatever you want for a harness ... as long as you're on a pay-as-you-go plan. So it's not blocked. It's just not allowed on the $20 per month plan.
First, harnesses can give access to company internal tools (like the ticket queue). You could do this with MCP, but it's much harder, slower and it kind of resists doing this (if you want a bot to solve a ticket, why not start with an entire overview of the ticket in the first request to your model? This can't easily be done with MCP)
Second harnesses can direct the whole process. A trivial example is that you can improve performance in a very simple way: ask "are you sure?" showing the model what it intends to do, BEFORE doing it. Improves performance by 10%, right there. Give a model the chance to look at what it's doing and change it's mind before committing. Then ask a human the same question, with a nice yes/no button. Try that with MCP.
Of course you quickly find a million places to change the process and then you can go and meta-change the process. Like asking an AI what steps should be followed first, then do those steps, most of whom are AI invocations with parts of the tickets (say examine the customer database, extract what's relevant to this problem, ...). Limiting context is very powerful, and not just because it gets you cheaper requests. Get an AI to make relevant context for a particular step before actually doing that step ...
> A trivial example is that you can improve performance in a very simple way: ask "are you sure?" showing the model what it intends to do, BEFORE doing it. Improves performance by 10%
Put it into the "are you sure" loop and you'll see the model will just keep oscillating for eternity. If you ask the model to question the output, it will take it as an instruction that it must, even if the output is correct.
Not in my experience. I mean, it happens. But models can check if their own function calls are reasonable. And that doesn't require dropping the context cache, so it's a lot less expensive than you would probably initially think.
But you should correct: Claude is very happy to let you use whatever you want for a harness ... as long as you're on a pay-as-you-go plan. So it's not blocked. It's just not allowed on the $20 per month plan.
First, harnesses can give access to company internal tools (like the ticket queue). You could do this with MCP, but it's much harder, slower and it kind of resists doing this (if you want a bot to solve a ticket, why not start with an entire overview of the ticket in the first request to your model? This can't easily be done with MCP)
Second harnesses can direct the whole process. A trivial example is that you can improve performance in a very simple way: ask "are you sure?" showing the model what it intends to do, BEFORE doing it. Improves performance by 10%, right there. Give a model the chance to look at what it's doing and change it's mind before committing. Then ask a human the same question, with a nice yes/no button. Try that with MCP.
Of course you quickly find a million places to change the process and then you can go and meta-change the process. Like asking an AI what steps should be followed first, then do those steps, most of whom are AI invocations with parts of the tickets (say examine the customer database, extract what's relevant to this problem, ...). Limiting context is very powerful, and not just because it gets you cheaper requests. Get an AI to make relevant context for a particular step before actually doing that step ...