I’ve been doing a bit of work with Temporal using it’s Python SDK. Temporal remains one of my favorite pieces of technology to work with. The team is very thoughtful with their API design and it provides a clean abstraction for building distributed, resilient workflows. It’s a piece of technology that is difficult to understand until you build with it, and once you do, you find applications for it everywhere you look. I highly recommend experimenting with it if you’re unfamiliar.
This conversation between Lenny and Geoff was particularly noteworthy for me because it hit on so many areas of what I’ve seen in the most effective organizations and teams I’ve been apart of as well as realigning incentives to solve a number of problems I’ve experienced that hold teams back.
We report back operational overhead, meaning the percentage of tickets that come from your product area normalized by the number of users that are using that product
Simon wrote an excellent post on the current state of the world in LLMs.
Twitter continues to talk LK-99. It seems like an easy thing to root for but hard to tell exactly what is going on.
The high-order bit that changed in AI:
— Andrej Karpathy (@karpathy) August 3, 2023
"I'll give you 10X bigger computer"
- 10 years ago: I'm not immediately sure what to do with it
- Now: Not only do I know exactly what to do with it but I can predict the metrics I will achieve
Algorithmic progress was necessity, now bonus.
Turning scaling into a systematic science is the biggest advance enabled by LLMs. https://t.co/BgHoDxeX7m
— Jim Fan (@DrJimFan) August 4, 2023
It will be interested to see if or when we hit scaling limits to training more powerful models and what our new bottleneck becomes. For now, there appears to be a lot of greenfield.
While not an entirely unique perspective, I believe Apple is one of the best positioned companies to take advantage of the recent improvements in language models. I expect more generic chatbots will continue to become commodities whereas Apple will build a bespoke, multi-modal assistant with access to all your personal data on device. This assistant will be able to do anything the phone can do (invoke functions/tools) as well as answer any question about your personal data (show me photos from Christmas in 2018). Let’s hope they name it something other than Siri.
Your project has a youthful optimism that I hope you won’t lose as you go. And in fact it might be the way to win in the long run.
I tried out Llama 2 today using ollama. At first pass, it seemed ok a writing Python code but I struggled to get it to effective generate or adhere to specific schema. I’ll have to try a few more things but my initial impressions are mixed (relative to OpenAI models).
It’s hard to think because it’s hard to think.
Finally learned that RAG stands for “Retriever-Augmented Generation” after seeing it all over the place for months. Not sure how I missed that one.
Meta released Llama 2 yesterday and the hype has ensued. While it’s exciting to see more powerful models become available, a model with weights is not the same as an API. It is still far less accessible.
A paper came out on the measurement of the degradation ChatGPT’s reasoning abilities. As real-time peer review took place over the course of the day on Twitter, the most compelling explanation that I heard to explain these findings was that OpenAI has further fine-tuned the models to respond in a manner consistent with the level of the prompt, because this is a better experience for the user. We’ll see if this explanation holds up over time.
I’ve been playing around more with nix lately.
I like what I’ve seen from it so far: declare dependencies and get an isolated shell with those dependencies.
If distributed, the environment can be trivially recreated on another machine.
So far, it’s been a struggle to get a working Python environment with dependencies setup.
I’ve gotten a lot of cryptic error messages after trying a number of different flake.nix files.
I plan to continue to experiment, but thus far the learning curve is tough.