Tagged for OEG Connect: How Does a Large Language Model Really Work?

What’s of interest? How Does a Large Language Model Really Work?

Tell me more!

LLMs: Fancy Autocomplete with a Spark of Magic

“Large Language Models are the path to general artificial intelligence, with capabilities even their creators don’t fully grasp.”

“Large Language Models are just fancy autocomplete.”

Both opinions are controversial, yet correct in their own way. I like to think of LLMs as fancy autocomplete with a dash of magic.

Just like the magician I watched, there’s a simple explanation (no real magic or AGI), but also an impressive skill (crafting a powerful LLM).

Today, I aim to demystify the principles of LLMs - a subset of Generative AI that produces text output. Understanding how tools like ChatGPT generate both magical and sometimes oddly dumb responses will set you apart from the average user, allowing you to utilize them for more than just basic tasks.

Where is it?: https://blog.tobiaszwingmann.com/p/how-large-language-model-chatgpt-really-works?ck_subscriber_id=2143270782

This is one among many items I will regularly tag in Pinboard as oegconnect, and automatically post tagged as #OEGConnect to Mastodon. Do you know of something else we should share like this? Just reply below and we will check it out.

1 Like

Thanks for sharing. I keep such links on my pearltrees account under critical digital literacies. I am currently preparing a blog post about the impact of generative AI on language teaching and learning and will share it back here.

I look forward to seeing your post, Teresa! Peartrees looks vaulable for link curation.

I find web bookmarking tools indispensable for my own works, though maybe my tagging system is a bit organic- I keep a pile of AI links in pinboard Pinboard: bookmarks for cogdog tagged 'ai' I like in this old school tool that I can facet or combine links, like Pinboard: bookmarks for cogdog tagged 'ai+llm'

though I wonder if the next generation approach is being able to use AI to search across a collection, maybe to overcome my tagging sloppiness.

But more imporantly I’m keen to plan some future discussion sessions on topics like you suggest, for domain/topic specific discussions of applied AI.

Thank you for being active here.

1 Like

Thanks for your encouragement! Here’s my reflective post Unrealia | Espace Sisyphe

And how nice of you to mention and link back to OEG Connect. Thus I am torn where to reply, but for now, I left some warblings as a comment on your post.

1 Like

Ah, but just to pick up on I think was one of your questions is a change in what we consider as creativity, just scanned what I could from a partly paywalled newsletter things at every.to titled “Capability Blindness and the Future of Creativity”

The author notes a significant improvement in the quality of output from the newest generation of Claude, and making the task of drawing a line between AI vs Not AI increasingly muddy. One Creativity he writes:

Previous eras of creativity have mostly looked a bit like sculpting. A sculptor takes a block of material and carves it, slowly but surely, into shape. Nothing happens without her hand. Even when an assistant is involved, the sculptor pores over the project, because their human input is important at every point of the process. So too with writing, or programming, or painting.

This era of creativity is going to look more like gardening. A gardener doesn’t grow plants directly. Instead, she sets up the conditions for the garden to grow. She takes care of the soil, the water, and the sunlight—and lets the plants do their thing.

So too with AI. As more of our time is spent being model managers, we won’t be directly making as much creative work. That’s like pulling up a plant to help it grow. Instead, we’ll be creating optimal conditions and letting the models do their work.

There’s one difference, though. A gardener can’t directly modify her plants. She can’t change their DNA by hand. But a skilled model manager can take any output of a model—sentence, code, image, or video—and modify any part of it themselves.

I am not excited, but I am just one pebble in the stream.

This ties into other things I have been perusing that allude to new skill of gleaning from summaries of longform content, and wondering what happens as we are gardening from summaries of summaries of …