How can I add videos to Google Gemini as context (is this even what their newest model is called anymore) and why is it so hard to figure it out? https://gemini.google.com only let’s me upload images. I assume I need to pay for something.
I played around with Cohere’s chat. They support web search and calculator and a python interpreter as tools as well as files and an internet search connector. I added “Web Search with Site” pointed to Steph Curry’s stats on Basketball Reference. Then I prompted
how many points did Steph Curry score in 2022?
Unfortunately, this didn’t work so I tried again with the generic web search connector which also didn’t see to work.
I tried a different prompt, but wasn’t able to get tool invocation to work consistently.
using the context from websearch, how many points did Steph Curry score in 2016-2017?
I also played around some with the calculator and the python interpreter but didn’t find anything too novel in terms of the interaction model – maybe explicit tool invocation via @ reference but unfortunately, I couldn’t get that working too consistently.
I could see it being a useful pattern though to help disambiguate a large number of tools.
What I was really keen to try was
- first, a search specific data from the web page
- next, plot that data from the context using the python interpreter
I couldn’t reach get the web search to give me any data though so I wasn’t able to try out the approach.
I like Cohere’s UI, how they do source references and a lot of parts of how tools work.
I realized, using @, I could also use multiple tools in a single inference.
I tried
using the context provided, get the average number of points did Steph Curry has scored in each season of his career, then plot the data with python
Unfortunately, this seemed to fall into a retry loop until I got an error.
Cohere has a lot of good things going here but I need the tools to work more consistently to bring it all together.
I tried the above with Perplexity and had far more success building the data table. Perplexity doesn’t have a python interpreter but it would write the code for me to create the plot.
Using these tools has generally validated my decision to mostly using direct api or playground/workbench access to models. I haven’t found and wrapped experience compelling enough to switch to as my primary tool yet.