Tools like Microsoft Copilot and other enterprise AI agents already do a remarkable job of parsing and understanding existing content. They can take a 50-page strategy document or a dense report and surface relevant insights faster than most of us could manually.
But when it really matters, high-stakes answers, governance decisions, regulatory compliance, or executive briefings, relying solely on raw, unstructured documents can introduce risk. A little re-formatting goes a long way in ensuring agents don’t just “find something useful,” but consistently ground their responses in the best possible representation of the information.
The answer, of course, is that it’s not about re-engineering every single document. Just as with information risk management, the real value comes from applying effort where it matters most, the knowledge sources that underpin critical decisions, external obligations, or authoritative guidance.
That’s why curating documents with structure, clarity, and machine readability in mind is so important. It’s less about rewriting everything from scratch or forcing markdown everywhere, and more about shaping existing knowledge into forms that both humans and machines can use effectively when the stakes are high.
What inspired me to write this post was an exercise in building an Agent to help people with Microsoft 365 Information Architecture decisions. Over the years I’d collated a 100-page document full of patterns and archetypes, and I found myself wondering: is this really the right format to use for grounding?
Why good grounding starts with good documents
AI agents don’t “skim” like humans. They index text in chunks, use headings to structure meaning, and ignore what they can’t parse (such as complex layouts, unlabeled images, or footnotes).
That’s why a 100-page all-in-one PDF may be a weak source, while a concise 10-page guide with clean formatting can ground the agent more effectively
1. Scope and Granularity
- Avoid monoliths: Don’t drop 100 pages of mixed content into one file.
- Go modular: Create smaller, topic-focused documents (5–20 pages).
- Bound your themes: Each document should stand alone (e.g. “SharePoint Site Patterns”).
Why? → Modular, thematic content improves retrieval accuracy and reduces irrelevant matches.
2. Structure and Style
- Use hierarchical headings (H1, H2, H3).
- Start sections with a 2–3 sentence summary.
- Prefer lists, steps, and bullets over dense narrative.
Why? → Headings and summaries act as natural anchors for chunking, improving how agents locate the right passage.
3. Formatting Matters
- Use clean tables for structured comparisons.
- Avoid multi-column layouts, nested tables, tracked changes, or “final_v7” filenames.
- Write numbered steps for processes.
- Add captions and short summaries to every image or diagram.
Why? → Complex formatting often gets stripped. Text-first, consistent structure ensures key information survives ingestion.
4. Diagrams and Processes
- Keep the visual for humans, but don’t stop there.
- Add a summary in plain text below the figure.
- Optionally include a Mermaid.js version of the process so the logic survives as text:
You prompt any AI to produce Mermaid diagram syntax by describing you process, asking it to articulate it as a set of steps and then ask for a mermaid.js diagram format.
5. Metadata and file handling
- Use descriptive filenames (e.g., IA_Metadata_Patterns.docx).
- Store version control in your system (SharePoint, Confluence), not in file names.
- Populate document properties like title and keywords.
- Why? → File names and properties are often indexed. Clear metadata improves match quality.
The Payoff
When you curate documents with grounding in mind, you reduce drift, ambiguity, and irrelevant answers. The agent retrieves cleaner context, and users trust the system more.
Put simply:
- Well-prepared knowledge sources = well-grounded agents.
- Don’t just build the agent. Build the ground it stands on.
- Ground rules: Better documents make better agents.
So we understand the ‘ground rules‘, now let’s get practical. Here’s how you can use prompts to transform your current documents into strong grounding material.”
Prompts for restructuring documents
1. Break into Modular Sections “Reformat this section into a standalone piece that could live as its own document. Give it a clear H1 title, a 2–3 sentence summary at the top, and structure the rest with H2/H3 headings.”
2. Add Summaries Up Front “Provide a concise executive summary (3 sentences) for this section that captures its key message and purpose.”
3. Turn Dense Text into Bullets or Steps “Convert this long paragraph into a set of clear bullet points or numbered steps that highlight the main ideas.”
4. Clean Up Formatting Noise “Remove tracked changes, comments, and any references like ‘see above’ or ‘as noted below.’ Rewrite the text so it stands on its own.”
5. Make Images Machine-Readable “For the diagram referenced here, write a 2–3 sentence plain-text summary of what it shows. If it’s a process, draft a simple Mermaid.js version of the flow.”
6. Simplify Tables “Rebuild this table with plain text rows and columns. Use simple headers and ensure no merged cells. Add a one-sentence intro explaining what the table compares.”
7. Reframe as Question/Answer (optional, for FAQ-style sources) “Rewrite this section as a set of Q&A pairs where each answer is 2-4 sentences.”
8. Check for Stand-Alone Readability “Rewrite this so it does not rely on context from previous sections. Make sure it’s understandable if read in isolation.”
How to use these prompts
- Copy a section from your document.
- Paste it into your AI tool with one of the prompts above.
- Replace the original section with the reformatted version.
- Repeat for each section until the document is modular, structured, and grounding-ready.
Final thoughts
About the author
Ready to get started with AI agents?
Get in touch today.
Australia
+61 3 9111 0082
New Zealand
+64 9 947 6799
Japan
+81 3 6820 1085
Get in touch