For me, invoking a language model using a playground (UI) interface is the most common approach for my usage. Occasionally, it can be helpful to use the a CLI to directly pipe output into a model. For example

git diff --staged | llm "write a commit message for these changes"

However, I am more often inclined to open a playground and paste the bits and pieces of context I need. Maybe, it’s that refinement and followups are common enough that using a CLI isn’t nearly as flexible. The bottom line is, I far more frequently open a playground to use a language model than use a CLI. Even though most of the playgrounds have various weird and annoying behaviors, I generally still prefer them.

I enjoyed this article by Ken about production LLM use cases with OpenAI models.

When it comes to prompts, less is more

This resonated with me. I’ve found that too much instruction can lead a model to perform worse on a task.

GPT is really bad at producing the null hypothesis

This also seems to confirm what I’ve seen empirically, but I never ask for it. I ask for something like, “return an empty JSON array if you can’t find anything”.

I enjoyed Martin’s article on preserving your shell history. I implemented some of his approaches in my system config.

Gemini Pro 1.5 up and running. I’ve said this before but I will say it again – the fact that I don’t need to deal with GCP to use Google models gives me joy.

❯ llm -m gemini-1.5-pro-latest "who is the fastest man in the world?"
As of November 2023, **Usain Bolt** is still considered the fastest man in the world. He holds the world record in the 100 meters with a time of 9.58 seconds, set in 2009.  He also holds the record for the 200 meters at 19.19 seconds, achieved in 2009 as well.

Having all these models readily available is great. My hope is to play around with several to become a bit of an amateur model sommelier.

Today, I learned about Command-R model series from Cohere from Shawn’s great AI newsletter (ainews). I searched to see if a plugin was available for llm and Simon had literally authored one 8(!) hours earlier. Folks like you keep me inspired and motivated 🙏. No better workflow out there that I know of:

llm install llm-command-r

llm -m r-plus hello
Error: No key found - add one using 'llm keys set cohere' or set the COHERE_API_KEY environment variable

llm keys set cohere
Enter key: ...

llm -m r-plus "hi, who am I speaking with?"
You are speaking with Coral, an AI chatbot trained to assist users by providing thorough responses. How can I help you today?

A great article by Manuel about forever-growth of companies. I too wish we’d be more willing to celebrate enough.

I’ve been digging more into evals. I wrote a simple Claude completion function in openai/evals to better understand how the different pieces fit together. Quick and dirty code:

from anthropic import Anthropic

from evals.api import CompletionFn, CompletionResult
from evals.prompt.base import is_chat_prompt


class ClaudeChatCompletionResult(CompletionResult):
    def __init__(self, response) -> None:
        self.response = response

    def get_completions(self) -> list[str]:
        return [self.response.strip()]

class ClaudeChatCompletionFn(CompletionFn):
    def __init__(self, **kwargs) -> None:
        self.client = Anthropic()

    def __call__(self, prompt, **kwargs) -> ClaudeChatCompletionResult:
        if is_chat_prompt(prompt):
            messages = prompt
            system_prompt = next((p for p in messages if p.get("role") == "system"), None)
            if system_prompt:
                messages.remove(system_prompt)
        else:
            # I think there is a util function to do this already
            messages = [{
                "role": "user",
                "content": prompt,
            }]
        message = self.client.messages.create(
            max_tokens=1024,
            system=system_prompt["content"] if system_prompt else None,
            messages=messages,
            model="claude-3-opus-20240229",
        )
        return ClaudeChatCompletionResult(message.content[0].text)

claude/claude-3-opus:
  class: evals.completion_fns.claude:ClaudeChatCompletionFn
  args:
    completion_fn: claude-3-opus

Run with

I can’t believe I am saying this but if you play around with language models locally, a 1 TB drive, might not be big enough for very long.

As someone learning to draw, I really enjoyed this article: https://maggieappleton.com/still-cant-draw. I’ve watched the first three videos in this playlist so far and have been sketching random objects from around the house. I find that I’m not too big of a fan of my drawing as I’m doing it but when I return to it later, I seem to like it more. Apparently, this is a common experience for creatives.

Added GoatCounter to my site. I’m planning to see how I like it compared to Posthog.

I’m taking a break from sketchybar for now.

I’m currently looking into build a NL to SQL plugin or addition to datasette to use a language model to write queries.