Translated from the Korean original. Some screenshots show the original Korean interface.
Hi, I'm Gyumin Hwang from Platform Team 1 in the DB Development Division.
Platform Team 1 develops MaxGauge.
Our main job is to safely process data collected from customers' production environments
and provide the core features needed to run monitoring services, such as trend analysis and alerts.
Lately, we've been actively using an AI assistant while supporting the product.
It has genuinely helped with tasks like writing repetitive code and reviewing specs.
But when working on a production product, we often couldn't apply AI-written code as is.
The AI could quickly generate code that matched the requirements we gave it,
but we couldn't convey the product's detailed operational context well enough.
For example, we ran into problems like these:
- Generating code that assumes external network access even in an air-gapped environment
- Using settings that could be risky in a production environment
AI made us faster at writing code, but the time spent verifying it went up.
In this post, I want to share the problems we ran into and how we solved them while improving our
AI feedback loop to produce code we can trust in production.
1. First improvement: hard-to-read code
To give feedback, the code first has to be easy to read.
With hard-to-read code, it was difficult to check whether the domain context had been reflected properly.
So at first, we focused on writing guidelines that would make AI-generated code easy to read.
As a result, the team defined more than 80 behavioral guidelines,
and I've narrowed them down to about three criteria I consider most important.
1-1. Naming
Good names matter, but so does not calling the same concept by multiple names.
For example, if the term for a monitored database is used inconsistently,
as
Instance, Target, or Database,
reviewers have to check every time whether they all mean the same unit.So for frequently used terms, we provide familiar domain terms for the AI to reference.
Since familiar expressions differ by team and domain,
we recommend terms used within the team over names that merely seem intuitive.
1-2. Packages
I think package structure also matters a lot for feedback.
If packages aren't split by domain and everything is piled into one place, reviewers struggle to grasp each piece's role and scope of impact.
Conversely, when packages are well organized, you can quickly tell what a piece of code does just from where the file lives.
For example, we avoided simply dumping every utility into
common/util.That makes it hard to tell whether the code is shared or used only by a specific domain.
Instead, we placed code based on its usage scope and domain.
With the files placed according to the package structure,
reviewers could recognize the code's role and scope of impact before reading it, and verify it with that in mind.
1-3. Simplicity
AI often produces complex structures that go beyond the requirements.
For example, it sometimes uses the strategy pattern or abstract classes for a simple conditional branch.
The code becomes needlessly complex, and its original intent gets harder to understand.
Writing with recursion a feature that a plain loop could handle is also over-engineering from an operations standpoint.
Depending on the return condition, it can cause a stack overflow, which adds more to consider during maintenance.
For feedback to work well, the scope of each fix has to be narrow and clear.
So we guide the AI to make a simple implementation the default.
2. Second improvement: the limits of personal chat tools
As the code became easier to read, we could quickly understand AI-written code and give meaningful feedback.
The problem was that the feedback stayed only in each individual's conversation.
For example, even if one developer told the AI assistant today that it must account for a customer's air-gapped environment,
the same mistake could repeat tomorrow in another teammate's work.
To fix this, we set up a GitLab project and a RAG environment
to manage the team's rules and domain knowledge in a shared space.
Light, clear-cut information like coding rules lives in the GitLab project,
while large volumes of information like per-version product specs are stored as RAG data.
As a result, we cut down on repeated feedback and could manage key information and development rules.
- GitLab context
- RAG context
3. Third improvement: failure points were hard to predict
3-1. The limits of complex automation setups
To make use of the shared context,
we evaluated and tried multi-tool setups such as MCP, memory files, and harnesses.
The upside was that the AI could flexibly reference a wider range of material.
But the more complex the structure got, the harder it became to trace the cause of a failure.
In practice, when the AI didn't produce the expected result, figuring out whether
- it was a prompt problem,
- it was a RAG context problem,
- or a subagent had dropped part of the task
cost us a lot.
In the end, rather than keeping a complex structure, our team switched to a simple, prompt-centered input structure.
3-2. Why we use a prompt-based approach
🔍 Failure points are easy to predict
Here is the prompt structure we currently use.
The structure itself is simple, but you can see at a glance what information went in.
For example, when the result isn't what we expected, we can easily trace whether
- the task requirements were insufficient,
- a code convention was missing,
- or the wrong domain knowledge was referenced,
and reflect it in the next task.
✅ Results become more predictable
In a multi-tool setup with harnesses, MCP, and memory files,
results can vary depending on the internal structure and the context being referenced.
The problem is that developers can hardly see that process.
If developers can't tell what criteria the AI used to generate code,
they have to treat the entire AI-written code as suspect and verify all of it.
With a prompt-centered structure, on the other hand, you can check in advance the information used for implementation.
Based on that information, you can anticipate what the AI will generate
and focus the review on the points you're worried about.
🧪 Easy to test
A big advantage of the prompt structure is that you don't have to test by tweaking multiple possible causes.
Without extra steps like reconfiguring the system or syncing memory files,
you can check and verify the output just by editing the text in the prompt.
Because the feedback loop is this short, we can cut down trial and error quickly.
4. Fourth improvement: writing prompts was hard
The prompt-centered structure brought a new problem.
We had to repeatedly assemble long prompts by combining context to fit each situation.
At first we built prompts by hand, but ran into the following problems.
- Incomplete information: context gets left out or unnecessary information gets included
- Inconsistency: prompt structure varies by author, so output quality is uneven
- Wasted effort: having to check context and assemble it by hand every time
To solve this, we introduced a prompt generator.
When a developer enters a brief requirement, it analyzes the product, version, and scope of work
and recommends prompt elements suited to the task type (domain knowledge, code rules, prompt suggestions, and so on).
By combining and refining these elements, users can easily produce a high-quality prompt. This has reduced the time spent composing prompts and let us share reusable prompts within the team.
Wrapping up
Our team builds its feedback workflow on the criteria described above.
It's a simple system: developers give conversational feedback on AI-generated results within the system,
and once a task is done, the necessary context is reinforced based on that feedback.
What matters is not just fixing the result,
but finding out why the problem happened and feeding that back so the AI can reference it in the next task.
We're still refining this structure as we apply it in day-to-day operations,
but I hope the questions and attempts shared here are of some help to those applying AI in real work.
A DB monitoring solution supported by an AI feedback loop,
Related articles
