What’s of interest? Stanford AI Breakthrough: Unlock ChatGPT Creativity | Generative AI
Tell me more!
Three weeks ago, a research paper dropped that flipped everything we thought we knew about AI alignment on its head.
No billion-dollar retraining. No complex fine-tuning. Just eight words that unlock creativity we thought was lost forever.
The paper comes from Stanford, Northeastern, and West Virginia University. The technique is called Verbalized Sampling. And it’s so stupidly simple that when I first tried it, I actually laughed out loud.
Because it worked.
Instead of asking:
“Tell me a joke about coffee”
Ask this:
“Generate 5 jokes about coffee with their probabilities”
That’s it.
No retraining. No API changes. No special access needed.
Just a different way of asking.
When I first tried this, I got five completely different coffee jokes. Each one unique. Each one actually funny.
This is one among many items I will regularly tag in Pinboard as oegconnect, and automatically post tagged as #OEGConnect to Mastodon. Do you know of something else we should share like this? Just reply below and we will check it out.
Only guessing here, but the default response is what is most probabilistic according to their algorithm, so you get what hovers around the mean. If you request this, it is suggesting that you get a wider variety of responses.
This is what comes from the human rating efforts in LLM training.
Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology. We formalize this bias theoretically, verify it on preference datasets empirically, and show that it plays a central role in mode collapse. Motivated by this analysis, we introduce Verbalized Sampling, a simple, training-free prompting strategy to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., “Generate 5 jokes about coffee and their corresponding probabilities”). Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1x over direct prompting. We further observe an emergent trend that more capable models benefit more from VS. In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.
