What’s of interest? How Does a Large Language Model Really Work?
Tell me more!
LLMs: Fancy Autocomplete with a Spark of Magic
“Large Language Models are the path to general artificial intelligence, with capabilities even their creators don’t fully grasp.”
“Large Language Models are just fancy autocomplete.”
Both opinions are controversial, yet correct in their own way. I like to think of LLMs as fancy autocomplete with a dash of magic.
Just like the magician I watched, there’s a simple explanation (no real magic or AGI), but also an impressive skill (crafting a powerful LLM).
Today, I aim to demystify the principles of LLMs - a subset of Generative AI that produces text output. Understanding how tools like ChatGPT generate both magical and sometimes oddly dumb responses will set you apart from the average user, allowing you to utilize them for more than just basic tasks.
This is one among many items I will regularly tag in Pinboard as oegconnect, and automatically post tagged as #OEGConnect to Mastodon. Do you know of something else we should share like this? Just reply below and we will check it out.
Thanks for sharing. I keep such links on my pearltrees account under critical digital literacies. I am currently preparing a blog post about the impact of generative AI on language teaching and learning and will share it back here.
And how nice of you to mention and link back to OEG Connect. Thus I am torn where to reply, but for now, I left some warblings as a comment on your post.
Ah, but just to pick up on I think was one of your questions is a change in what we consider as creativity, just scanned what I could from a partly paywalled newsletter things at every.to titled “Capability Blindness and the Future of Creativity”
The author notes a significant improvement in the quality of output from the newest generation of Claude, and making the task of drawing a line between AI vs Not AI increasingly muddy. One Creativity he writes:
Previous eras of creativity have mostly looked a bit like sculpting. A sculptor takes a block of material and carves it, slowly but surely, into shape. Nothing happens without her hand. Even when an assistant is involved, the sculptor pores over the project, because their human input is important at every point of the process. So too with writing, or programming, or painting.
This era of creativity is going to look more like gardening. A gardener doesn’t grow plants directly. Instead, she sets up the conditions for the garden to grow. She takes care of the soil, the water, and the sunlight—and lets the plants do their thing.
So too with AI. As more of our time is spent being model managers, we won’t be directly making as much creative work. That’s like pulling up a plant to help it grow. Instead, we’ll be creating optimal conditions and letting the models do their work.
There’s one difference, though. A gardener can’t directly modify her plants. She can’t change their DNA by hand. But a skilled model manager can take any output of a model—sentence, code, image, or video—and modify any part of it themselves.
I am not excited, but I am just one pebble in the stream.
This ties into other things I have been perusing that allude to new skill of gleaning from summaries of longform content, and wondering what happens as we are gardening from summaries of summaries of …