Back in 2018, I wrapped up a grueling 10-course Data Science specialization with a capstone project: an app that predicted the next word based on user input. Mining text, calculating probabilities, generating predictions—the whole works. Sound familiar?

Fast forward to today, and I’m at Microsoft exploring how this same technology is reshaping content creation from an instructional design perspective—how do we create content that works for both human learning and AI systems?

Since ChatGPT exploded in November 2022, everyone’s talking about “AI-ready content.” But here’s what I kept wondering: why do we need chunk-sized content? Why do Metadata and Q&A formats suddenly matter more?

The Fundamental Shift: From Finding to Generating

When you search in pre-AI time, the system is trying to find answers to your questions. It crawls the web, indexes content by keywords, and returns a list of links ranked by relevance. Your experience as a user? “I need to click through several links to piece together what I'm looking for.“

Generative AI search changes the search experience entirely. Instead of just finding existing content, it aims to generate new content tailored to your specific prompt. The result isn’t a list of links – it’s a synthesized, actionable response that directly answers your question. The user experience becomes: “I get actionable solutions, instantly.“

This isn’t a minor improvement – it’s a different paradigm.

_{Note: I’m simplifying the distinction for clarity, but the divide between “traditional search” and “generative AI search” isn’t as clear-cut as I’m describing. Even before November 2022, search engines were incorporating AI techniques like Google’s RankBrain (2015) and BERT (2019). What’s different now is the shift toward generating new content rather than just finding and ranking existing content.}

How the search processes actually work

As I’ve been studying this, I realized I didn’t fully understand how different the underlying processes really are. Let me break down what I’ve learned:

How does traditional search work?

Looking under the hood, traditional search follows a pretty straightforward path: bots crawl the internet, break content down into individual words and terms that get stored with document IDs, then match your search keywords against those indexed terms. Finally, relevance algorithms rank everything and serve up that familiar list of blue links.

How does generative AI search work?

This is where it gets fascinating (and more complex). While AI systems start the same way by scanning content across the internet, everything changes at the indexing stage.

Instead of cataloging keywords, AI breaks content into meaningful chunks and creates “vector embeddings,” which are essentially mathematical representations of meaning. The system then builds real-time connections and relationships between concepts, creating a web of understanding rather than just a keyword database.

When you ask a question, the AI finds relevant chunks based on meaning, not just keyword matches. Finally, instead of handing you links to sort through, AI synthesizes information from multiple sources to create a new, personalized response tailored to your specific question.

The big realization for me was that while traditional search treats your query as a collection of words to match, AI is trying to understand what you actually want to do.

What does this difference look like in practice?

Let’s see how this works with a simplified example:

Say we have three documents about computer performance:

Document 1: “Upgrading your computer's hard drive to a solid-state drive (SSD) can dramatically improve performance. SSDs provide faster boot times and quicker file access compared to traditional drives.“

Document 2: “Slow computer performance is often caused by too many programs running simultaneously. Close unnecessary background programs and disable startup applications to fix speed issues.“

Document 3: “Regular computer maintenance prevents performance problems. Clean temporary files, update software, and run system diagnostics to keep your computer running efficiently.“

Now someone searches: “How to make my computer faster?“

Traditional search breaks the question down into keywords like “make,” “computer,” and “faster,” then returns a ranked list of documents that contain those terms. You’d get some links to click through, and you’d have to piece together the answer yourself.

But generative AI understands you want actionable instructions and synthesizes information from all three sources into a comprehensive response: “Here are three approaches you can try: First, close unnecessary programs running in the background... Second, consider upgrading to an SSD for dramatic performance improvements... Third, maintain your system regularly by cleaning temporary files and updating software...“

How have the goals of content creation evolved?

This shift has forced me to rethink what “good content” even means. As a content creator and a learning developer, I used to focus primarily on content quality (accurate, clear, complete, fresh, accessible) and discoverability (keywords, clear headings, good formatting, internal links).

Now that generative AI is here, these fundamentals still matter, but there’s a third crucial goal: reducing AI hallucinations. When AI systems generate responses, they sometimes create information that sounds plausible but is actually incorrect or misleading. The structure and clarity of our source content plays a big role in whether AI produces accurate or fabricated information.

Why this shift matters for content creators?

What surprised me most in my research was discovering that AI systems understand natural language better because large language models were trained on massive amounts of conversational data. This realization has already started changing how I create content—I’m experimenting with question-based headings and making sure each section focuses on one distinct topic.

But I’m still figuring out the bigger question: how do we measure whether these strategies work? How can we tell if our conversational language and Q&A formats truly help AI systems match user intent and generate better responses?

In my next post, I want to show you what I discovered when I dug into the technical details. The biggest eye-opener for me was realizing that when traditional searches remove “filler” words like “how to” from a user’s query, it’s stripping away crucial intent—the user wants actionable instructions, not just information.

The field is moving incredibly fast, and best practices are still being figured out by all of us. I’m sharing what I’ve learned so far, knowing that some of it might evolve as technology does.