I tried out OpenRouter for the first time.
My struggles to find an API that hosted llama3.1-405B motivated me to try this out.
There are too many companies providing inference APIs to keep track.
OpenRouter seems to be aiming to make all these available from a single place, sort of like AWS Bedrock, but not locked in cloud configuration purgatory.
The first thing I tried was playing a game of Connections with nousresearch/hermes-3-llama-3.1-405b.
It didn’t get any categories correct for the 2024-08-21 puzzle.
OpenRouter’s app showcase list is an interesting window into how people are using models.
The dominant themes are
- coding
- characters and roleplay
- writing assistant
- homework
Users of some of these apps generate almost 1B tokens per day according to OpenRouter’s stats. The highest per month (excluding the OpenRouter Playground) is over 30B.
Playing with all these models, I’ve started to crystallize a set of a few things I try to get a sense of the vibes
- play Connections
- ask “hi, who am I speaking with?”
- ask for Python code to read all JSON files from my Desktop and all subfolders
- ask “what is the most recent event you can remember?” which usually gets a response related to the date the training data snapshot was taken