Brainstorming mode?

[chriselrod]

I was investigating a bug and planning a fix with an agent. While I was reading a report that it delivered, these messages appeared:

│<system> harness intervention: asked the model to close out its todo list                │
│         You planned 1 item this turn but changed no files. The plan is not the work —   │
│start the first item now with `write` / `edit` (or `bash` if it's a command). Do not     │
│call write_todo_list again until something is actually done.                             │
│                                                                                         │
│                                                                                         │
│<critic> You planned 1 item this turn but changed no files. The plan is not the work —   │
│start the first item now with `write` / `edit` (or `bash` if it's a command). Do not     │
│call write_todo_list again until something is actually done.

and then I had to hit <esc> to cancel and stop the agent from doing something.
The plan and report it gave me was precisely the work I wanted from it and asked of it.

I see that there is e.g. a "default_prompt": "code", and options like "critic_provider": . I hadn't set "critic_provider" before launching that session, so I imagine <critic> come from elsewhere?

How can I stop this from happening?

I did enable

  "phased_workflow_enabled": true,
  "phased_workflow_plan_approval": true,

but haven't tried it yet. But this (and especially that plan approval defaults to false) also seems a little "go code-now" oriented.
Although I'll update my comment after I gave the plan-mode a try.

I get that there are times where pushing the agent to code makes sense. Experimentation is often worth more and cheaper than thinking, so even throw-away edits can be a good way to figure things out.
So I don't want to say these options are wrong, and -- also -- the point for me to try a new harness is to try and understand it, what it does, and how that helps.
So I don't want to disable these settings in general. I'm merely curious if there are modes where they can be turned off until I'm ready to switch to implementation.

[yogthos]

Does the critic get triggered when you're in plan mode as well?

[chriselrod]

I'm currently in a session with --prompt debug and got

│<system> harness intervention: asked the model to close out its todo list                │
│                                                                                         │
│                                                                                         │
│<critic> You planned 5 items this turn but changed no files. The plan is not the work —  │
│start the first item now with `write` / `edit` (or `bash` if it's a command). Do not     │
│call write_todo_list again until something is actually done.

The agent was a bit confused

│<dirge> The nudge asks for file work, but this is a root-cause trace: the deliverable    │
│this turn is debugger evidence, not an edit (debug mode forbids fixes before             │
│root cause, and no fix was requested).

I haven't tried --prompt plan, but would guess this can happen with that prompt, too?

I can spend a few tokens investigating. I haven't yet (you may already know where to look).
I think I'd to be able to toggle their triggering dynamically. They shouldn't mess with the existing context, just change whether nudges get automatically appended, meaning no cache misses should be risked from this.

[yogthos]

you can change the prompt from the UI actually using /prompt, the idea would be to put it in plan mode when you don't want to change any files and the critic shouldn't ask for file changes in that mode. But it's possible there is a bug here where critic doesn't respect the mode.

[chriselrod]

Debug mode does give file-editing permissions, so perhaps that is why the debug-prompt didn't block the critic?

[yogthos]

Yeah, so it's a question of whether debugging should involve edits or not. Sometimes it can be useful for the model to try tweaking things as it's trying to figure out what's going on. But maybe the critic shouldn't nudge it to edit stuff.

[chriselrod]

I agree, I think it should be allowed to edit while debugging, but that an agent can still make progress towards root-causing even if doesn't edit (e.g. if it's using tools like a debugger instead of editing source) so the critic probably shouldn't nudge.

[chriselrod]

you can change the prompt from the UI actually using /prompt, the idea would be to put it in plan mode when you don't want to change any files and the critic shouldn't ask for file changes in that mode.

Thanks for the tip/patience. I'll give that a try.

[yogthos]

sounds good, and I'll update the critic behavior for debug mode in the next release