Translated from the Korean original. Some screenshots show the original Korean interface.
Turning Knowledge and Experience into Assets with an LLM Wiki
Hello, I'm Jinwoo Son from Integrated Technology Research Team 3.
I handle technical support for our monitoring solutions, SQL tuning consulting, and technical planning for product screens. Lately, more of my work involves using AI to bring the knowledge and experience I've built up in the field into product screen planning.
Once I started using AI at work, one thing kept bothering me: I had to write the same background explanation over and over. On top of that, my own memory fades over time. I needed a way to keep what I had learned as my own knowledge asset.
So in this post I'll share how I built and used a personal wiki, and what came out of it.
1. I kept doing the same work from scratch
Looking back on my work, I often re-researched things I had already researched and re-explained things I had already explained. Nothing I figured out stuck, so the same work started from scratch every time. In hindsight, it had disappeared both from my memory and from the memory of the AI session where I had asked about it.
1-1. My memory fades
Technical support engineers build up experience and knowledge as they move between customer sites. Even with the same product, each customer has a different environment, fails at different points, and has operators who look at and wonder about different screens.
Most of what we learn this way never gets written down.
Which screen operators check first during an incident,
which time ranges come up empty because of our product's collection interval,
what a metric should be divided by when it is shown as a percentage. We just figure it out at the time and try to remember it.
Over time, those memories become hazy. To confirm the details again, I have to dig up old files or redo the research from the beginning.
1-2. AI fills the gaps by making things up
When I started using AI at work, I ran into a similar problem.
To have it review a plan, I first have to explain the background: how our product is structured, where this metric is collected, what kind of value counts as abnormal. Once I type that background into the chat, I get a useful answer.
AI tools today remember conversations to some extent, so the background I typed stays around for a while. But not permanently. As time passes and the conversation grows, it fades the same way human memory does. Then I end up writing the same explanation from scratch.
The bigger problem is that the AI doesn't tell me “I don't remember.” It fills the faded gaps with plausible-sounding content.
It treats metrics our product doesn't have as if they existed, and invents criteria it claims we agreed on earlier. A made-up answer is more dangerous than no answer. You move on to the next step without knowing it's wrong.
Corrections are no different. Even after I tell it “our product doesn't have that metric,” the correction fades along with the conversation, and before long the same error shows up in the same place.
What they have in common
The two look like different problems, but they have the same structure: something that already existed somewhere evaporates, and it takes time to recreate it. One evaporates from my head, the other from the AI session. And both are side tasks you have to get through before the actual work.
I needed a way to keep what I had figured out in a form I could reuse as-is next time.
2. I learned about LLM Wiki at a team tech talk
I found the answer at one of our team's tech talks. Integrated Technology Research Team 3 holds a Homecoming Day once a month. Because our work keeps many team members out at customer sites, we pick a day to come back to the office and share what we've studied or tried in our work.
That was where I first heard about the LLM Wiki pattern published by AI researcher Andrej Karpathy.
LLM refers to AI models such as ChatGPT or Claude. LLM Wiki is an approach where you use a personal wiki that the AI itself writes and maintains as your knowledge store.
I thought it could solve my problem. Instead of explaining things anew every time, I could keep what I'd explained once as files.
Until then, I had only collected materials and had the AI find and read the relevant passages each time I asked a question. With that approach, though, the AI searches from scratch every time it builds an answer. If an answer requires tying together five documents, it redoes that work every time you ask. Yesterday's synthesis is gone today.
Karpathy's approach is different. Instead of organizing things when a question comes in, the AI updates the wiki in advance every time new material arrives.
When new material comes in, it reads it and merges the key points into existing pages. It fixes links between documents and flags any point that contradicts earlier content. So yesterday's synthesis is still there today, and the wiki grows richer the more material you add.
I started right after hearing it. The goal was to store the knowledge I'd gained in the field and the knowledge I'd research in the future in the same format, and reuse it as evidence in the next piece of work.
3. Let me show you my wiki first
The wiki here is a personal wiki kept as text documents, in a format called Markdown, in a folder on my PC. I left creating and editing the files to Claude Code, an AI agent that can read and write files on my computer directly.
I'll start with what the finished wiki looks like. Section 5 covers how I actually create and maintain it.
3-1. Three layers: raw, wiki and conventions
The structure has three layers.
- Raw: Deep-research results (in-depth research handed off to AI), product source analyses and official documents are collected as-is. The AI only reads them and never edits them. This is the only layer where you can go back later and re-check the evidence behind a decision.
- Wiki: Refined knowledge extracted from the raw sources. It is split into folders by domain and managed entirely by the AI. People only read it and ask questions.
- Conventions (CLAUDE.md): A rules file that defines how to handle the wiki. Claude Code reads it before starting any work (other AI tools use AGENTS.md). This file is what makes the AI act as the wiki's maintainer.
Two special files are added on top.
index.md is the full table of contents the AI reads first, and log.md records what was added when.Operating the wiki comes down to three actions: ingesting new material, querying the wiki, and periodically linting it. Since the wiki is left to the AI, there are also safeguards to keep made-up content out. Each statement gets a grade for how far it has been confirmed, and each value gets the date it was checked, so when the AI uses something as evidence it can see whether it is a confirmed fact or a guess. I added one more step on top: verification, where values written in the wiki are queried against the product again.
All three layers are just Markdown files. So the AI reads them straight from the folder without any separate integration. If you keep knowledge inside a service, it disappears the moment you can no longer use that service, but files stay. Push them to a repository like GitHub and they can become an asset the whole team shares.
If you write folder and file names into the conventions, the AI follows them. Nobody needs to memorize them.
3-2. Folder layout and graph view
The screen above shows the wiki folder opened in Obsidian, an editor that opens a Markdown folder as-is. What is expanded in the folder list on the left is the raw material, stored in the same units it came in.
For
wiki/, where the knowledge extracted from the raw sources goes, I split folders by domain: aws-ai, exemone, postgresql-rca, sql-tuning, and so on. I didn't split them by project because the same knowledge is used across multiple projects.In the graph view on the right, the two large clustered nodes in the middle are
index and log. Almost every document links to them.Open any document and you'll find two more things at the top: an evidence grade showing how far the content has been confirmed, and the date the values were checked firsthand (
verified).3-3. What changed from typing background into a chat window
Compared with the approach from section 1, where I typed the background into the chat window every time, here is what changes.
Aspect | Chat prompt | Local wiki + agent |
Knowledge lifespan | Fades over time | Stays in files and is read as-is in the next session |
Background | A person copies in the background every time, even parts that won't be used | The agent picks and reads only the documents it needs |
Error correction | Recurs in the next session even after you point it out | Fixes stay in the files, and the next session reads them |
Output | Text (a person copies and applies it) | Actual file edits, through to verification |
Sharing | Personal chat history | Documents the team reads |
On a table alone, the difference may not look big. What actually changed became clear as I built three deliverables.
4. Applying the wiki to real work
I applied the wiki to three pieces of work: the integrated monitoring home of exemONE, our company's integrated monitoring solution; AWS Cost Advisor; and PostgreSQL root cause analysis (RCA). The integrated monitoring home and Cost Advisor are screens, and RCA is a set of rules for determining the cause of an incident and the action to take. The wiki played a different role in each case.
4-1. exemONE integrated monitoring home
The first deliverable was a redesign of the exemONE integrated monitoring home. The home is the first screen you see after logging in, and it puts every monitored area on one screen: applications, databases, infrastructure, cloud, Kubernetes and message queues. Its purpose is to let infrastructure operators quickly judge which area has a problem right now.
My knowledge and experience lean toward databases, yet I had to decide what to show even for areas I had never worked with. So instead of starting from my own experience, I decided to start from what the product already had, and took exemONE's data collection DB as the starting point.
First, I put what was in my head, along with the structure and items of the tables the product collects and stores, into the wiki. But a list of columns alone wasn't useful. What I needed came next.
- Why is this data collected? What situation the item was created to show
- What kind of value is it? Whether it is a point-in-time value or a cumulative one, what a utilization rate is divided by, and what the unit is
- What decisions do operators use it for?
For areas where these three couldn't be filled in, I supplemented them with deep-research, checked them against what the product actually collects, and recorded in the wiki only what holds true in our environment. Following along as I read, I gradually came to understand areas I hadn't known.
I built a first draft based on that wiki. I gave three AI agents the wiki and the data exemONE had actually collected, and assigned them the roles of infrastructure expert, screen planner (UI/UX expert) and operator who will look at this screen every day.
I let the three perspectives give opinions and push back on each other, and I only decided what to adopt as I watched the discussion. I refined the resulting draft several times, all the way to a working screen.
While I was building the screen, the knowledge in the wiki was revised and filled in as well, because points where my notes didn't match the real data kept coming up. A value I had noted as missing would turn up in another table, or a metric I thought was usable turned out not to fit on the screen. Each time, I fixed the wiki.
After going through this once for each area, I could explain with evidence what should go on the screen and why, even for areas I had never worked with. When adding a new feature or screen tab, I could pick up right where I left off without re-explaining the history up to that point, because the wiki kept that history.
That said, this is as far as I can go on my own. Areas I have never operated myself, such as Kubernetes or networking, need to be revised based on feedback from experts in those areas. Only people who have done that work could judge whether the draft would hold up in the field.
4-2. AWS Cost Advisor
Next is a screen for looking at AWS costs. AWS Dashboard, which we are building in-house, is a project for viewing resources from multiple AWS accounts on one screen, and Cost Advisor is the screen within it that deals with billing. Our existing products had screens for monitoring the cloud, but nothing on the billing side.
Since the cloud bills you for what you use, reducing costs remains an ongoing task. I built the Cost Advisor screen proactively so it would be ready when the requirement came up.
I had dealt with resources and configurations related to DB services, but the rest was unfamiliar, and building a cost screen meant researching the pricing model separately as well.
Pricing varies by instance type (instances being virtual servers) and region, and storage and I/O are billed separately on top of that. To show “Save $22/month” on the screen, every one of these calculations has to be right. That $22 means halving the cost by switching an instance that costs $45.26 a month on demand to one that costs $22.63.
I set the order like this.
- Research the AWS pricing structure with Claude Code's deep-research (instance family scheme, billing units, license models, and what actually changes when resizing)
- Save the results as-is under
raw/aws-ai/
- Extract only the facts needed for screen design and organize them in the wiki atomically (each item states only one fact)
- Design the screen based on that wiki
This screen is the result. I built Cost Advisor as a working screen as well.
As I directed the AI to do the research and designed the screen, what I, as the human, had to think about was making the screen answer the operator's questions. Putting myself in an operator's shoes, there were three things I'd want to know: How much do I save? Why was that decided? Will changing it cause any resource problems?
- Recommendation list Resources are classified as downsize, upsize, delete or right-sized, along with how much the monthly cost would change if you follow the recommendation. Operators can use this list to find resources where they can cut costs.
- Decision criteria Click a resource in the list to compare its current spec with the recommended spec and see, item by item, whether the change is safe. The judgment is based on the peak usage window, and a graph shows current utilization alongside expected utilization after the change. The items compared differ by resource type (server, DB, disk). Operators can check the evidence themselves and decide whether to change the class.
- Class change candidates All classes (the spec tiers for servers and DBs), both up and down, are shown with their monthly prices, along with whether each one is feasible based on current usage. Click one, and the decision results and expected utilization for that class are redrawn in the decision criteria and metric trend comparison area on the right. Operators can simulate the utilization trend after a change before actually changing the class.
I also wrote this screen's decision logic back into the wiki, that is, which grade is given to which utilization range. That way, the same criteria can be reused as-is when other resource types are added later.
I put the researched prices into the wiki too. Since prices keep changing, I recorded the date each value was checked so the oldest ones get re-queried and updated first. The more unfamiliar the knowledge is, the less reason there is to memorize it, and the more it changes over time, the better it is to keep it in the wiki and pull it out when needed rather than carry it in your head.
4-3. PostgreSQL root cause analysis (RCA)
There is also an in-house project to add root cause analysis (RCA) to AIOps, exemONE's AI-based operations analysis feature. When a DB slows down or fails, AIOps detects it and raises an alert.
Among the many infrastructure components, I was put in charge of the PostgreSQL platform.
Existing product alerts only go as far as “a certain metric has spiked.” AIOps aims to tell you what the problem is right now, why it happened and how to act on it.
However, customer sites where exemONE is deployed are usually isolated from the internet, so queries can't be sent to AI services on the other side. That's why the knowledge the AI uses to make judgments has to be built into the product in advance. Building that knowledge-based decision framework is the job of the division members on this project, including me.
So I organized common incidents into five signals: lock waits, CPU spikes, IO load, Vacuum issues and WAL pressure. A lock wait is a state where work is stalled waiting for a previous operation to finish, and CPU spikes and IO load are states where computation or disk reads and writes pile up. Vacuum is the process that cleans up deleted data, and WAL is a log where changes are written down first.
A problem with any one of these can slow the DB down, and when several occur at once, a combined analysis is needed.
I defined each case where signals overlap as a situation, and wrote rules for what the cause is and how to act in each one. With AI's help, I worked out what triggered what when several signals appeared together.
Every piece of evidence behind those rules lives in the wiki. The wiki's postgresql-rca domain has 13 pages covering everything from signal definitions to the combination matrix, per-situation rules and the verification environment.
Next came verification. I used AI to build simulation scripts that reproduce each situation on a real DB. I reproduced each situation twice and checked whether the predefined rules matched what actually happened.
Anything new I learned from verification went back into the wiki. It's the same cycle as when I built the two screens.
Our division is turning these rules into RAG training data (material the AI refers to when answering), and the results go into AIOps. The next step is to keep verifying and ultimately apply it in real operations. Along the way, situations I haven't defined yet or rules that need fixing will come up, and I plan to reflect those in the wiki too and keep building them up as knowledge assets.
5. How to create and maintain the wiki
Here is how I create and maintain the wiki shown in section 3, in the order I actually did it.
5-1. Start with Karpathy's guide
Karpathy published the original on GitHub Gist. You can copy and paste the whole text, or just give the agent the URL and it will read it. I opened Claude Code in an empty folder and told it this:
Claude Code creates the folder structure and the conventions file. The procedures for adding material, asking questions and linting, as well as
index.md and log.md, are already in Karpathy's document, so they come out without any extra instructions. From then on, a single line like “ingest this material” is enough.The two lines about the vault and Web Clipper are tailored to my environment, so they only take effect if you write them out.
A vault is what Obsidian calls the folder it opens. Install Obsidian and set your wiki folder as a vault, and the document list appears on the left and you can see how documents are connected as a graph. For the graph to be drawn, links between documents need to use the
[[file name]] format.The conventions don't need to be complete from the start. The guidelines and rules you write in
CLAUDE.md can be adjusted to fit the nature of your work as you go. I changed my folder criteria midway too, and only added evidence grades much later.5-2. Ingest research into the wiki
Whatever I learn from research gets ingested into the wiki as I go. I usually research in two ways.
One is Claude's deep-research. Give it an instruction like “Research the AWS pricing model” and the results come out as a document. I put that result into the raw folder as-is.
The other is saving web pages I find and read myself. With Obsidian Web Clipper, a browser extension, the page you are viewing is saved to the wiki with a single click. I use it the way you'd bookmark a URL, but unlike a bookmark that only keeps the address, the body text comes along too. The title, source URL and save date are attached automatically.
Either way, the ingested original stays untouched, and the wiki pages take only what they need from it.
5-3. Save useful answers to the wiki
When a question gets a useful answer, I turn it into a wiki page as-is. Even a single comparison table or a single decision criterion is worth keeping. Left only in chat history, it gets hard to find over time, but in the wiki it gets reused as evidence in the next question and the next plan.
5-4. Grade the evidence for every statement in the wiki
Every statement in the wiki will eventually be used as evidence for a decision. So I classify how far each piece of content has been confirmed as measured, cited or inferred.
Grade | Definition | Requirement |
Measured | Value confirmed by directly running against a primary source | Leave a reproducible query and path in the text |
Cited | Based on a reliable primary document or source code. Not executed | State the file:line or document location |
Inferred | Circumstantial evidence and inference | Note the evidence along with possible counterexamples |
If a page mixes several grades, it takes the less-confirmed one.
Without this distinction, guesses and unconfirmed content become evidence for the AI's answers, and any plan built on them goes off track too. If you don't separate confirmed facts from unconfirmed ones, eventually you can't trust your own wiki.
5-5. Record when values were checked, and update them
What you organize in the wiki drifts from reality over time. In my work, this happens when a product version upgrade changes a metric's naming rules or storage location. Query exactly as the wiki says, and not a single row comes back. The AI doesn't check for that gap; it uses what's written as evidence as-is.
So every new page gets the date its values were checked firsthand (
verified). During periodic linting, pages with old dates are flagged as candidates for updating. The dates each item was added and checked are logged one line at a time in log.md.5-6. Have the AI lint it once a quarter
As documents pile up, they start to drift apart. Pages say different things about the same subject, or documents end up connected to nothing. So once a quarter, I have it run a lint.
It then lists contradictions, stale content, those isolated documents, concepts that appear often in the text but have no page of their own, and broken links. It's the kind of task people put off when they do it themselves, so I leave it to the AI.
6. Closing thoughts
So far, I've used AI with the wiki as evidence to put together early technical planning drafts and, through vibe coding, build screens too. Here are three things I took away from the process.
6-1. What I built only goes as far as a draft
Thanks to vibe coding, where you build programs by instructing AI in plain language, we've reached a point where you can build what you've only imagined. That's how I built the integrated monitoring home and Cost Advisor. But this approach has clear limits.
It may be possible to build programs for yourself or for people inside your organization, but a commercial solution product has to be designed for performance, security, scalability and incident response as well. Performance and security design is a specialized field that requires long-accumulated experience. That's why the programs I built with vibe coding are only drafts, and they need review and final development by the development organization.
6-2. The more I used AI, the more domain knowledge I needed
In all three projects, the most time went into deciding what to build and how to make it convenient.
- exemONE integrated monitoring home What information to show first from an infrastructure operator's point of view, and how the screen should behave
- AWS Cost Advisor What cloud operators want to know when looking at costs, and how the screen should answer those questions
- PostgreSQL RCA Which combinations of signals to break common incidents into, and what to treat as the cause and what action to recommend for each combination
All three require foundational knowledge, and the depth of that knowledge and the ideas behind it determine the quality of the result.
When AI started becoming part of everyday life, I thought about it a lot. What is my role in the AI era, and where should I spend more of my time? For a while, I spent a lot of time learning tricks for getting AI to do things. But many of those tricks mattered less as AI improved.
What I need to focus on narrowed down to three things.
- Using domain knowledge to decide the order in which to drive the planning
- Turning work procedures that only existed in conversation into a sequence of screens and data, shaped so they can be implemented as a program
- Sorting out whether the procedures and data the AI produces are right for this domain
Work usually exists only as a vague sequence in someone's head. Once you decide where to put what on the screen and what data to load at which point, the AI handles the implementation from there.
But AI will happily produce metrics that don't exist in our environment and screens we can't use. Only someone who knows the work can tell which of the AI's outputs are actually usable. So the more you use AI, the clearer the human's part becomes.
6-3. A wiki that has learned my knowledge and experience helps with those judgments
If knowledge exists only in your head, you have to explain it again every time it's needed. Once it's organized in the wiki, the AI reads it first and then makes its judgment. Because what I know becomes the evidence, the answers that come out land closer to the direction I have in mind.
So whether I'm deciding what to build or coming up with a new idea, the knowledge I've built up becomes the material for my judgment. The screen drafts and decision logic in section 4 came out that way. I started this to cut down on explaining, but in the end, the wiki ended up holding my judgment criteria for me.
The wiki was a device that keeps the domain knowledge I've built up from being used once and thrown away, so it can be reused in the next job and keep accumulating on top of that. Keeping context as an asset and judging the AI's output with domain knowledge is what remains my job. Whatever time I've saved from re-explaining things, I now spend on that judgment.
Sources
LLM Wiki: A pattern for building personal knowledge bases using LLMs - Andrej Karpathy What Is Andrej Karpathy’s LLM Wiki? - MindStudio (2026.07.22)
Related articles
