You nailed this EXACTLY. Taken in a vacuum, an AI-generated article will almost always be inferior to its Wikipedia equivalent.
The value add could come from generating articles that didn’t previously exist, e.g. https://infinipedia.ai/wiki/OEAwards_vs.\_Academy_Awards - and here you’ll immediately see the tricky part: Infinipedia didn’t know about OEAwards and completely hallucinated it. This can be significantly improved with better prompting (or follow-up fixes, which I tried, let’s see what version 2 of the article looks like), better models, and more compute.
The tricky part is that improved grounding in reality could make user experience worse because 1) slower speed / less responsiveness: running a more extensive web search (or deep research) and parsing the results takes time, and 2) higher cost: better models and more tokens for better thinking cost $$$ – not a huge deal if you have dozens or hundreds of pages in a wiki (school or personal wikis), definitely a problem for a free-to-use public facing website where a random person or a bot can hammer out millions of pages.
Teaching students to think critically about AI-generated outputs, their pitfalls and limitations, could be a super useful learning experience. Also, teaching them how to spot incorrect information and prompt about fixes, teaching them to summarize and link the knowledge, THAT would be something they could totally use in real life.
My guess would be that they’re running an LLM prompt of medium complexity, with some lightweight search. Definitely not a lot of heavy web scraping in the background: perhaps to save the costs, perhaps to keep the site snappy. This works well for mainstream-ish prompts with low complexity and little ambiguity but the site stumbles a bit for trickier pages. Which is why I would LOVE this to exist as an open source tool – imagine the ability to tweak this to to re-run page generation with deep research (if you see that the original page sucks), imagine the support for manual edits, the ability to add personal or crowdsourced comments / notes / marginalia,…
Interconnection of personal notes (human-generated) with knowledge aggregated from the web (AI-generated), with annotations going both ways (e.g. AI summarizing a browsing session around a topic, a human writing about learnings from AI-powered research) sounds quite interesting. I wonder how the field will evolve and what tools become mainstream.
My current PKM system is mostly manual and human, and I’m exploring ways to leverage AI to extend it / make it better and more useful, and perhaps combine the human-ai parts. I have a hunch there is a huge opportunity to extend our “second brain” further.
How do other people bring together AI-generated text and human-generated notes?
