I find it very intriguing. I reminds me of the insight in the early days that prompting the models to "think step by step" gave big performance boosts. Here we basically tell the models "You have an internal monologue/J-space and you can use it!". Quasi an opaque 'think step by step' prompt. The models were always able to think step by step but needed to be prompted to do so. Maybe the same thing is possible with the J-Space? The huge claims are yet to be replicated/proven, though.
Great thread, I was just thinking about compaction. My current line of thought is that compaction/pruning/ctx management in general should be something ongoing and maybe recursive. For example:
User:'How is auth implemented?'
->
[thinking]
[codebase exploration with [thinking] in between, 10 file reads, 3 of which were "wrong"]
[thinking]
->
agent_response
This little exchange contains a WHAT (how auth actually is implemented) and a HOW (where that info is and how to retrieve it). Maybe this question was part of a larger task. I think that whole exchange could be summarised before it enters context, kind of like what happens with subagents. The main thread would then consist mostly of [summaries]. Eventually the context will fill up anyway and we would summarise those summaries again. Alternatively one could maintain a [master_summary], kind of like an internal state. So new [summaries] get integrated directly and the [master_summary] gets updated.
I am surprised at 'removes actual tool calls/results, tool output'. Your approach with /prune seems to be 'keep the WHAT, remove the HOW (we got here)'. I would have thought that the HOW contains some useful signal.
The regular /prune command leaves tool call 'receipts', which includes the command executed and whether it succeeded or failed, but not results. The extended prune removes both.
The how is important, but I've found all of the decisions, question, answers, and results are the most important and the tool calls themselves secondary. When necessary, the tool calls can be deleted without much being lost.
If you're doing automated data visualization, "please make plot XYZ in plotly" needs a sandbox to execute the JS or Python code. With something like this you can avoid it, you just validate it and plot it. If the LLMs are better at generating Flint than Vega-Lite specs then I see how the project can be useful.
Used to be part of the $20/mo plan but it's not anymore (not sure if they removed it completely). However GPT-5.6 is pretty good at researching if you prompt it right, I've regularly had it spend 5+ minutes researching topic with lots of web searches.
So its basically (a plan + a bunch of file reads + a first edit) injected into the context of a cheap model so the cheap model does not feel like it has to re-read the files and just continues executing and editing?
Yes this is correct. It essentially biases the cheaper model into procedural action grounded by the frontier model. To put it mode generally, it injects the cheap model context with more useful information, where it might not need to read all the files again.
reply