Or How AI Will Help Solve One of the Biggest problems in Software Knowledge Management
Picture this: You’re a developer who just joined a team working on a complex payment processing system. Your first task is to add support for a new payment provider. You open up the company wiki, hoping to understand the current architecture. The most recent diagram was last updated when “Gangnam Style” was last trending. The service documentation? It’s scattered across Confluence, GitHub, and that one Google Doc that legends say still exists somewhere.
Sound familiar? Let’s talk about why this happens and how AI is about to fundamentally change how we approach knowledge management in software development. And no, the solution isn’t hiring a documentation fairy who magically updates all your wikis at midnight.
A Very Typical Scenario
Let’s use a real-world example to illustrate the current state of documentation:
@Retry(maxAttempts = 3, backoff = @Backoff(delay = 1000, multiplier = 2))
public PaymentResponse processPayment(PaymentRequest request) {
// Payment processing logic
}In a traditional setup, here’s what typically happens:
- A developer writes this code and adds a comment explaining the retry mechanism
- Someone creates a wiki page about the payment system’s resilience patterns
- Six months later, there is an incident that leads to another developer changing a retry configuration at 3:00 AM and resolving a problem at hand.
- The documentation becomes outdated.
- Rinse and repeat a thousand times across multiple systems until we have to have a project to update all of the documentation instead of delivering valuable business features.
Ah, the circle of developer life! It’s like watching a documentary about technical debt in its natural habitat.
Quite a few of you will say: “Well duh, of course your documentation will be out of date, you need to have the discipline to keep it up to date”. This sentiment is not wrong in it’s logic, but it’s wrong in its sentiment. Relying on humans not to make mistakes is as brittle as a system can get. There has to be a better way.
The Three Fundamental Truths of Software Knowledge Management
1. Documentation is NOT the Source of Truth
The actual behavior of your system isn’t determined by what’s written in Confluence or discussed in stand-ups – it’s determined by the code, data, and infrastructure. That retry mechanism above? Its real behavior is defined by the code and configuration, not by what someone wrote about it six months ago while hopped up on their fifth espresso of the day.
2. Documentation Exists Because Source of Truth is Not Feasible
The source of truth for your system may not be accessible to those that need to understand it. Your consumer shouldn’t need to read your code to understand the behavior. Your product manager shouldn’t have access to the configuration store to see how a tenant is set up.
Think of documentation as a translator between “machine-speak” and “human-speak.” Because let’s face it, if code could speak directly to humans, it would probably sound like your GPS when you take a wrong turn – constantly recalculating and slightly judgy.
3. Documentation is Question-Driven
Documentation is only needed when there is a question. Think about all of the times that you went to the documentation. You probably had a specific topic in mind. Maybe you just wanted to know how all of the components fit together in a diagram. Or perhaps, you wanted to know how the back off strategy works. It all starts with a question.
It’s like that emergency manual in your kitchen drawer – you only read it when smoke is coming from your toaster, and you’re wondering if that’s normal. (Spoiler: it’s not.)
The AI Revolution in Knowledge Management
With these 3 truths in mind. Let’s think about how Knowledge Management should change with an advent of AI.
Let AI generate the documentation for your specific question, in the format that you want from the source of truth that matters.
While we want to go to the source of truth, we don’t because there may be limitations in our ability to do so. So we close this gap by creating documentation. With AI we no longer need to do that. AI is able to look at your code directly to reason about the answer. With rise of AI Agents and the tooling around them, we should even be able to go to the data and infrastructure directly.
Because documentation is question driven, we have found the EXACT use case for much of our generative AI experience. Ask a question about your system and have AI generate the documentation in the way that you like it. You want diagram, you get a diagram. You want a podcast, you get a podcast. You want an interpretive dance, you get…well…let’s not get carried away.
It also solves the problem of having to bring documentation to the person seeking it. How many times have you had to point others to the existing documentation or find duplicates. It’s a failure in accessibility. But now you don’t need to. You will find any answer in one place.
Consider these different types of questions a developer might ask:
Developer: "What's the retry mechanism for failed payments?"
AI: *analyzes current code*
"The payment processing service uses exponential backoff with:
- Maximum 3 retry attempts
- Initial delay of 1 second
- Multiplier of 2 (so delays are 1s, 2s, 4s)
This is implemented via the @Retry annotation in PaymentService.java"
...Presents a diagram of how this works...
Developer: "Why do we use exponential backoff?"
AI: *analyzes git history and linked design docs*
"The exponential backoff was implemented after an incident in July 2023 where payment provider downtime caused a cascade of immediate retries, making the outage worse. Or as the incident report eloquently put it: 'Everything was on fire, and we were adding more matches.'"Reference Solution
If you are struggling to visualize how this all fits together in practice, then let me give you a little reference solution. Keep in mind that AI is still in it’s infancy, so just like documentation I write about in this article, this reference architecture is likely to become stale quick. But hopefully it will help you visualize how it might fit together.
In the diagram above, you will see that I am depicting the use of AWS Bedrock. I can create multiple agents using multi agent collaboration feature. The orchestrator agent will receive a question and then utilize other agents to provide a comprehensive answer.
The other agents may be a vendor provided. For example AWS can provide an agent that has an ability to look up your infrastructure. Or Jira may provide an agent that can browse through your stories.
You may also create custom agents. These agents could go to your own data sources like you own code or databases. There are various techniques like RAG that are easy to set up using AWS Bedrock Knowledge Bases.
This is just one approach. There are many others and you don’t even need to use AWS to do so. Hopefully this gives you an idea for how this can work in practice.
But wait…
Of course there are some problems with this thinking. We have to acknowledge that documentation isn’t just about about code and data. There is a lot of context behind WHY we build something. Documentation can help us understand the bigger picture, the business use case and the intricacies of how people operate at scale. Wouldn’t we lose this if we fully rely on AI?
Yes. But it’s a manageable problem. In the past, I wrote about how we have to adopt software best practices that make AI most effective. We have to do the same with documentation. My recommendation is to close the gap by providing structured documentation for what isn’t in the source of truth (yet).
For example, if you keep your architectural decision records in the code, then you have a structured format record for all decisions that you make. You can even use AI to help you write them up. Not only would this allow AI to evaluate them and improve the generated code, but it will also help to give it context that code and data just can’t give.
Let’s not forget that vendor agents would help bridge the gap on other context too.
Conclusion
The future of knowledge management isn’t quite about eliminating documentation – it’s about being smarter about what we document and how we access that knowledge. By focusing on documenting context and letting AI handle the details, we can create a more efficient and maintainable knowledge management system.
What’s next? I recommend that you try to find ways to experiment with this concept. You may not have the appetite nor bandwidth to pursue this idea in full yet. And let’s be honest we are still at the infancy of what is possible. However, do think about some of the smaller steps that can incrementally get you there. Maybe start to adopt structured docs like ADR and include them in code. Run a few experiments to see if you can utilize a vendor agent for your wiki (you may already have one). Slowly pave your way to the destination and adopt as it evolves.
I would love to hear about your experiences and feedback. Don’t hesitate to reach out and talk.
Leave a Reply