I spent some more time experimenting with thought partnership with language models.
I’ve previously experimented with this idea when building write-partner.
Referring back to this work, the prompts still seemed pretty effective for the goal at hand.
My original idea was to incrementally construct and iterate on a document by having a conversation with a language model.
A separate model would analyze that conversation and update the working draft of the document to include new information, thoughts or insights from the conversation.
It worked reasonably with gpt-3.5-turbo.
I’m eager to try it with claude-3.5-sonnet.
Today, I rebuilt a small version of this idea with ollama in Python.
The goal was to try the idea out focused on a local-first experience.
For this, I used a smaller model.
Initially, I tried mistral but ended up settling on llama3 as it was a bit better at following my instructions.
Instead of writing to the working file after each conversation turn, I decided to add a done command that allowed me to do that on-demand.
The results were pretty reasonable for such a small model Small models love to ignore directions about output formatting and will frequently add content like “Here’s the updated document:” even when you explicitly prompt them not to. . Something I think that I’ve been under-appreciating is that the raw conversations are the most important artifact produced from this tool and exercise. It’s possible the model won’t propose useful edits or take the conversation in a productive direction, but the value that comes from the model interactions is how the user responds to the model. This is the signal or raw material that can be refined into a more focused structure. By only persisting the document across sessions (and focusing on the document as the important artifact), I’d lost sight of the fact that the user content is what is most important. If I can effectively prompt the user into articulating answers to the model’s questions that are true to what they think, feel or want to accomplish, I can run that content through all sorts of model prompts to restructure it in more refined ways. It’s important, however, to preserve all the raw signal. In the case of write-partner, the document the model constructs is just one possible presentation of the user’s raw thoughts. To stay as true to the user’s intention, we can’t discard any of what they say. Their vision is the north star. The model is just the conduit to transform the user’s vision and ideas into a form that can help communicate the idea to others.