I’ve been following Jason’s working experimenting with different abstractions for constructing prompts and structuring responses. I’ve long felt that building prompts with strings is not the type of developer experience that will win the day. On the other hand, I’m weary of the wrong abstraction that would move the developer too far away from the actual prompt, which would make it harder to construct good prompts and steer the model. I’m not sure if this is an ORM vs. SQL conversation or if there’s an abstraction that exist as a happy medium.

Did some work with Clojure destructuring.

Unpack values into specific variables.

user=> (let [[a b c] [1 2 3]] (println a b c))
1 2 3
nil

Unpack the first N items, ignoring the rest.

user=> (let [[a b] [1 2 3]] (println a b))
1 2
nil

Unpack the first N items to variables and capture the rest as an array.

user=> (let [[a b & rst] [1 2 3 4 5]] (println a b rst))
1 2 (3 4 5)
nil

Doing math with a non-big decimal number and a big decimal number can cast down.

user=> (* 0.1 101M)
10.100000000000001

user=> (bigdec (* 0.1 101M))
10.100000000000001M

Heard the phrase “if someone wins the lottery” used today to describe a teammate leaving a team. I much prefer this to the more morbid alternatives.

I tried gpt-engineer today. I liked the approach and the setup instructions are good. I think I remember needing to use Python 3.11 instead of 3.8 that I was running, but beyond that the readme instructions we on point.

Process

You start by creating a project folder with a plaintext prompt. You start the script and point it at your project folder. The program reads your prompt then uses the LM to ask clarifying questions. The clarifying questions seem pretty effective. If you answer more than of one of the predetermined questions at once, the program seems to recognizes that and removes it from the list. Finally, it creates an actual project, with source code, pretty consistently (3/3 times I tried). I used it to try and create a 1-player Scattergories CLI game or something close.

I’ve been thinking about the concept of “prompt overfitting”. In this context, there is a distinction between model overfitting and prompt overfitting. Say you want to use a large language model as a classifier. You may give it several example inputs and the expected outputs. I don’t have hard data to go by, but it feels meaningful to keep the prompt generic or abstract where possible rather than enumerating overly specific cases in a way that obfuscates the broader pattern you’re hoping to apply. I hypothesize these overly specific examples could interfere with the model output in unintended, overly restrictive ways.

Richard WM Jones booted Linux 292,612 to find a bug where it hangs on boot. I loved reading the recounting of his process to accomplish this, by bisecting through the different versions of Linux and boot each thousands of times to determine whether the version contained the bug.

Georgi Gerganov started a company, ggml.ai, to run language models on commodity hardware.

I was listening to a podcast interview of Adam Robinson and he was discussing why he believed it is important process information with your body. He gives the example that when listening to something, he stops around once every minute and sees how it feels. He later goes on to highlight the importance of recognizing when something is “weird” and paying attention to it, or trusting one’s intuition even if rationally we can’t understand why something feels unusual. He asserts that we should take action on these intuitions – “if something seems a little bit off, it’s very off”. I could see this approach contributing to more well-rounded cognition, since it can be easier to discard these “intuitions” as unsubstantiated or lacking facts to justify them. On the other hand, unless you track how often you intuition proves correct, it could be hard to know if it’s well calibrated or trustworthy in the area in which you are applying it.

I read an article today about Scripting with Elixir that caught my eye because it touches on a problem I often struggle with: how you do easily distribute a script along with its dependencies in Python? Elixir has an example of what this could look like in the form of Mix.install. This feature allows one to distribute just the source code of your script and dependency management can be done when the script runs, without needing to distribute a mix.esx or requirements.txt file (as one does in Python) along with the source.

Today, I played around with Matt Rickard’s ReLLM library, another take on constraining LLM output, in this case, with regex. I tried to use it to steer a language model to generate structure (JSON) from unstructured input. This exercise is sort of like parsing or validating JSON with regex – it’s not a good idea. Complicated regular expressions to describe JSON are hard to write. I do love the demo that generates acronyms though. Matt also wrote parserLLM which provides to ability to use a context-free grammar to constrain the next predicted token from the language model. I prefer the context-free grammar approach at a high-level, but believe we need the language model constraints to be directly connected with the data structures we intend to use in code to effectively weave language models into our existing applications.

I’ve been trying to find a simple way to host a website that formats and serves a nice looking version of a recipe written in markdown. There are a few open source projects available, but nothing that has fit the bill yet. I briefly had the idea to try out something with Next.js and mdx but I found when I scaffolded a new app that I didn’t even recognize the project structure. Next.js has moved to the “App Router” approach for structuring projects. It’s not immediately obvious or intuitive how this works. As a batteries-included framework, it makes sense that different approaches to structure will have their own learning curves. Nevertheless, it’s a bit irritating how frequently this structure changes. I have a ~2-3 year old Next.js project that looks nothing like the project I am currently working on. And the project I am currently working on looks very different from a newly scaffolded project with App Router today.