Mastering Jagged Intelligence and Model Selection Routing: Driving Success in GPT5 Generative AI Pilots

Many modern organizations run Microsoft Copilot pilots to validate whether the $30 per user per month subscription delivers enough value. The success of that pilot often determines how quickly and broadly the company moves forward with full license adoption.Generative AI isn’t like most technologies; its intelligence can feel uneven. Beyond factors like prompt engineering and overall strategy, one issue often creates confusion and slows adoption: “jagged intelligence.”

The key is to embrace generative AI and Copilot with your eyes open and aware. In this post, we’ll talk about how to navigate gen AI’s strengths and considerations so that your pilot successfully assesses the value of Copilot licenses for your organization.

So, what exactly is “jagged intelligence”? Jagged intelligence is a term to represent how Copilot, and other generative AI tools, might ace one kind of task and then totally stumble on another. On top of that, the model router (or the AI model that determines the best large language model to analyze a prompt’s complexity and requirements) can sometimes choose a Large Language Model (LLM or model) that’s not the best fit. It’s not that Copilot isn’t capable, it just has its quirks. Gen AI can be like having a team member who’s an expert at corporate communication but gets a bit lost choosing a design for the latest newsletter. This unevenness can surprise people who expect Copilot to be uniformly smart across the board.

While evaluating potential use during a pilot period, a user might be less likely to trust and use Copilot after their requests are not uniformly successful. A common example used to illustrate how Copilot can fail is to ask it, “How many of the letter ‘r’ are in the word strawberry?” On the other hand, a user could successfully follow up by asking Copilot to review all my Teams messages from the last five days and identify outstanding action items. The latter is a far more useful and complex task for Copilot to complete.

The key is to understand that this is normal and not a sign that Copilot can’t be helpful or add value. For the tasks that people do day to day, encourage users to experiment with Copilot in different parts of their processes and workflows.

Let’s review another use case: this time, asking Copilot to create a status deck for an update to management. Copilot will not be very effective at gathering information and creating a deck all at once. However, a user can ask Copilot to gather the latest status on their project and then use it as a source for creating a deck. Once the deck draft is created, Copilot can even create images or charts to add value to the presentation.

This jagged intelligence phenomenon means Copilot may not be good at creating a deck from scratch, but it can help you effectively and efficiently prepare for creating the final deck with much less effort than traditional methods. (Author’s note: This segmented approach might be a thing of the past with the recent announcement of Office Agent. I’m excited to try out these new agents!)

Once users know where the jagged edges are, you can work with them and set your team up for a smoother pilot experience.

Now, let’s bring it back to the model router, which chooses which version of LLM is the best for each prompt and output. As we now know, some LLMs excel at certain tasks, but there is a delicate balance between speed and the quality of the response. Imagine a rideshare app that assigns the nearest driver to you, not necessarily the one with the most comfortable car or best rating. You’ll get a quick response, but maybe not the ride quality you were hoping for.

In GPT‑5, the “router” works similarly: it selects a fast model for speed, even if a deeper, more nuanced model would better serve your needs.

The good news is that once you know this can happen, you can plan for it.

Now that we understand jagged intelligence and model routing, let’s focus on making your pilot successful and triggering the correct LLM for the best possible result. Ask your users to start small with tasks where simple prompting excels, like drafting emails, summarizing documents and generating ideas, which will build confidence with quick wins. As users get more confident, ask them to gradually introduce more complex tasks, triggering a deeper thinking model, and using any stumbles as learning moments to refine expectations. Copilot evolves fast, so today’s limits may disappear soon. Over time, patterns will emerge showing where the AI shines and where guidance helps.

As those patterns emerge, it can be helpful to give your team a “pilot playbook” or FAQ document that includes:

In short, the more you embrace the Copilot’s quirks and use them as a learning tool, the more successful your pilot is going to be.

Copilot with GPT-5 (and tools like it) are incredibly powerful, but not infallible. Jagged intelligence and model selection routing are baked into the experience, and if you know that going in, you can build a smarter, smoother pilot from the start.

Instead of expecting the AI to be perfect, focus on learning what it does well, where it struggles and how your team can use that knowledge to get value. That’s what turns a clunky pilot into a real-world success.