Eugene’s article on prompting is one of the best things I’ve read recently, full stop. As also noted by Kyle, try starting with 0.8 temperature and lower only if necessary, even for deterministic use cases^{1 2}.

A few years old, excellent response written by Maxim about extracting the most value from Temporal by using it “as a service mesh for invocations of child workflows and activities hosted by different services”.

I found a very satisfying, time-saving use of a model though Cursor to generate an inline shell script to reformat some data for me. The goal was to reformat json files that looked like this

[...]

to ones that look like this

{"cells" [...]}

I used the following prompts:

for each json file in the current folder, read the json (which is a json list) and turn it into a json object with a single key, “cells” whose value is the json list

Read More…

So Apple has introduced a new system called “Private Cloud Compute” that allows your phone to offload complex (typically AI) tasks to specialized secure devices in the cloud. I’m still trying to work out what I think about this. So here’s a thread. 1/

— Matthew Green (@matthew_d_green) June 10, 2024

Matthew wrote a thread summarizing Apple’s private cloud and their security approach. Between the speed at which models are changing and their size, it’s not currently practical to run things like LLM inference on iPhones, at least not for the best available models. These models will probably always be larger or require more compute than what handheld devices are capable of. If you want to use the best models, you’ll need to offload inference to a data center.

I spent some time moving my bookmarks and workflows to Raindrop from Pocket. I’m mostly done with a short post on how and why I did that. I’d like to use the python-raindropio package to generate stubs for logs posts from links I bookmark, including comments I write about them. The Nix flake I use for this blog hampered those efforts. The python-raindropio package isn’t available as a Nix package and I’m not particularly well versed in building my own packages with Nix, so that became a bit of a rabbit hole. After poking around for other available libraries, it turns out, this one isn’t even maintained anymore. It was forked as raindrop-io-py and it looks like that might not even be maintained anymore. Not the greatest sign, but the package does still work for the time being.

I was away from the computer for a couple of weeks. That was really nice. During my downtime and in transit, these were some of my favorite things I read in the past two weeks. I’m planning to come up with a bit of a better system for logging and adding thoughts to stuff I read but for now a list will have to do.

• https://eugeneyan.com/writing/simplicity
• https://daniel.haxx.se/blog/2024/05/27/my-bdfl-guiding-principles/
• https://yekta.dev/posts/dont-microservice-do-module
• https://hiandrewquinn.github.io/til-site/posts/doing-is-normally-distributed-learning-is-log-normal/
• https://blog.railway.app/p/how-we-work-volume-iii
• https://jxnl.co/writing/2024/05/22/systematically-improving-your-rag
• https://www.oreilly.com/radar/what-we-learned-from-a-year-of-building-with-llms-part-i/
• https://thereader.mitpress.mit.edu/how-actors-remember-their-lines/
• https://materializedview.io/p/durable-execution-justifying-the-bubble
• https://jeena.net/20-years-blogging

Sabrina wrote an interesting write up on solving a math problem with gpt-4o. It turned out the text-only, chain-of-thought approach was the best performing, which is not what I would have guessed.

It’s was cool to see Simon dive into LLM-driven data extraction in using his project datasette in this video. Using multi-modal models for data extraction seems to bring a new level of usefulness and makes these models even more general purpose.

Nostalgia: https://maggieappleton.com/teenage-desktop.

I wish I had done something like this.

Maybe I can find something on an old hard drive.

I’m looking into creating a Deno serve that can manage multiple websocket connections and emit to one after receiving a message from another. A simple way to implement this is to have a single server id and track all the ongoing connections to websocket clients. I’m learning more about approaches that could support a multi-server backend.

I take an irrational amount of pleasure in disabling notifications for apps that use them to send me marketing.