I spent another hour playing around with different techniques to try and teach and convince gpt-4 to play Connections properly, after a bit of exploration and feedback.
I incorporated two new techniques
- Asking for on category at a time, then giving the model feedback (correct, incorrect, 3/4)
- Using the chain of thought prompting technique
Despite all sorts of shimming and instructions, I still struggled to get the model to
- only suggest each word once, even when it already got a category correct
- only suggest words from the 16 word list
Even giving a followup message with feedback that the previous guess was invalid didn’t seem to help. This was the prompt I ended up with. It wasn’t all that effective.
The game "Connections" is a word game where you start with 16 words and need to group them into 4 groups of 4. Each grouping has a category that unambiguously groups the four words together. Each puzzle has exactly one solution. Watch out for words that seem to belong to multiple categories. You will be given 16 words. Output 4 groupings of 4 words and the categories to which they belong.
Output your response in exact adherence to the following form
category: <category>
words: <word1>, <word2>, <word3>, <word4>
Make one guess at a time. I will tell you if you guess correctly or if you get 3/4 words for a category. Otherwise, I will respond with "incorrect". Each word is only in one category.
Output the remaining words after I tell you you got a correct answer.
Here are the 16 words: hello, list, trick, word, cant, fool, skinny, eggshell, lean, con giggle, scoop, boob, slope, dope, dupe
Let's think step by step. YOU MUST use words from the list. DO NOT suggest other words not in the list. You will lose the game if you do so.