Case StudyHow ContextQA's engineers came to merge 97.5% of what an AI agent wrote

ContextQA builds an AI-powered software testing platform, and it ships fast, often several times a week. Every release can change what a user sees or what an API returns, so its documentation is never finished. Last quarter ContextQA handed that upkeep to EkLine, an AI-native documentation platform: EkLine's agents find what changed, write or correct the page, and raise it for a quick review. Over the quarter that kept about 4 in 5 of the docs current, and ContextQA never added a documentation hire to do it.

The short version

The agent created or brought 90 of ContextQA's 113 pages up to date in one quarter. The number that matters most is not how much the agent produced, but how much the team kept: it merged 97.5% of what the agent proposed, 79 updates, and set aside only two. That is the line between volume and value. Engineers approved the updates the way they approve code, and spent the rest of their time building the product.

What did EkLine deliver in one quarter?

Over 90 days, EkLine kept about four in five of ContextQA's documentation pages current, 90 of the 113 pages in the docs. Engineers reviewed and merged the work; no one was hired to do it. The shift, in plain terms:

The agent finds the work, does the work, and I just review. We shipped a quarter of documentation without hiring a writer. As a founder, that is the kind of leverage I want.

Why is keeping docs current so hard for a fast-shipping product?

Because the product changes faster than anyone can write it down. ContextQA founder and CEO Deep Barot runs a team pointed at the product, not at the docs, which is the normal trade a startup shipping every week makes. Writing a page is quick. Keeping a hundred of them in step with a product that changes weekly is the part that never fits in anyone's day. By the time someone notices a page is stale, the product has often moved again.

For ContextQA the stakes run deeper than for most companies. Its own testing agents read its documentation, so a page that falls behind does not just cost a customer a few minutes, it feeds outdated information into the product. Current docs are part of how ContextQA works, not a nice-to-have, and increasingly they are how the outside world learns about it too. That is why keeping them right pays off twice.

How does EkLine work with your engineering?

EkLine connects to the repository behind the docs and watches the engineering work as it happens. The agents read the code, not a summary of it, so they catch every change that should update the docs at the source. Two agents do most of the work, and the team can also reach EkLine directly.

It reads every code change

EkLine grades every merged pull request for whether it touches the docs. EkLine raises documentation updates for high-confidence changes and notes them on the original pull request; it flags lower-confidence changes for a person to decide. Reading the source directly is what lets it catch the renamed field or the sample that no longer runs, the drift a prose review misses.

It hunts for drift on a schedule

A second agent runs on a cadence, compares what shipped over the past couple of weeks against what the docs say, and raises the updates that close the gap. When support or marketing spots something out of date, they flag it to EkLine in Slack and it opens the fix. Nobody has to file a ticket for any of it.

Does reviewing AI-written documentation create overhead?

No. The updates came in small and clean, so reviewing one mostly meant reading it and approving it, the same as any small code change. Two-thirds merged as a single commit, which in review terms means the page was right the first time. The structure, the writing, and the code samples were correct enough to ship without rework.

The exceptions were the ones that should be. The deep accuracy work, aligning the MCP tool samples and the API parameters with the live reference, drew real back-and-forth, ten to eighteen comments each, before it merged. The human review landed where the technical risk was, and the routine updates moved through on their own.

What happens when the UI or the API changes?

A product that redesigns its interface leaves its docs describing the old screen for a while, and EkLine caught those as they happened. It realigned button labels, navigation paths, and statuses to the redesigned app, and rebuilt the test-plan pages for the new layout. Code drifts the same way. As the API moved, examples fell a step behind. An execute_test_plan sample still passed an argument the API had dropped. A network-log example was no longer valid JSON, and MCP tool signatures had changed. EkLine kept the samples running. Developers who copy them get working code instead of an error and avoid support questions.

Which guides did EkLine write?

As ContextQA added features, EkLine added the guides to match, the ones a testing team's support fields questions about all day. It wrote first-test tutorials for mobile and API, mock-API testing, debugging failing tests, building and scheduling a regression suite, and generating a test suite from requirements. It covered configuring the Knowledge Base and running tests across browsers. It also wrote up the settings and permissions that had lived in the team's heads. The full quarter of work breaks down like this:

About seven of every ten updates put a new guide or a correction in front of a reader. With a guide to point to, a new user can self-serve the answer instead of waiting on support.

The docs are measurably carrying that load. Over three months GitBook logged 383 questions from visitors and answered 79% of them outright from the documentation, which it estimates saved around $1,000 in support cost. The pages doing that answering are the ones EkLine refreshed.

EkLine writes the guides and fixes the samples faster than we could file the tickets about them. My engineers stay on the product, and our customers get answers without waiting on us.

Can customers and AI assistants find the docs?

Increasingly, yes, and part of EkLine's work was making sure of it. A doc page only helps if people can find it. EkLine gave 37 pages titles aligned to what customers search for and added missing meta descriptions. It tuned the docs home and the Web Testing landing page to their primary keywords. EkLine also cross-linked role pages, core concepts, and tutorials so the set reads as a connected whole—the way search crawlers and AI assistants move through docs.

That matters more than it used to, because people are no longer the main audience. Over the last three months ChatGPT alone pulled more pages from ContextQA's documentation than every human visitor combined, and Meta AI pulled about four times as many. Anthropic's and OpenAI's crawlers are in there too, and the machine-readable index at llms.txt was requested 170 times. 50% of the pages those systems ingest most are ones EkLine wrote or corrected, including the docs home, the MCP server overview, and the roles and permissions reference. When an assistant answers a question about ContextQA, these are the pages it is reading.

Ask an assistant how to schedule a regression suite in ContextQA and the answer comes back from the tutorial EkLine wrote. Ask how to set up the MCP server and it returns the steps EkLine corrected.

What's next

ContextQA now runs its documentation the way it runs its code: changes land, get reviewed, and ship, without anyone carrying it as a full-time job. The same current docs feed its own testing agents, so keeping them right pays off twice. Next, ContextQA is bringing more of its team into the loop. Engineers and support can send changes straight into the docs as they work, and more of the upkeep runs on its own.

Frequently asked questions

How long did the documentation work take?

EkLine worked ContextQA's docs over one quarter, April 7 to July 6, 2026. In 90 days, it shipped 79 updates live. It brought about 80% of the documentation set, 90 of 113 pages, up to date, at a steady pace of roughly six delivered per week.

Did ContextQA hire a technical writer?

No. ContextQA added no documentation hire. EkLine's agents drafted and corrected the pages, and the existing engineering team reviewed and merged them the same way they review code. The team kept 97.5% of what the agent proposed.

How long does EkLine take to set up?

About 20 minutes. EkLine connects to the repository behind the docs and starts reviewing pull requests the same day. In the first 15 days ContextQA's team had improvements landing without lifting a finger.

How does EkLine keep the documentation accurate?

EkLine reads the code on every pull request and grades whether it changes the docs, raising a documentation update automatically when confidence is high. A second agent runs on a schedule to catch drift between what shipped and what the docs say, so pages stay in step with the product.

Do ContextQA's docs show up in AI assistant answers?

Yes. Asked real ContextQA questions, AI assistants answer from ContextQA's own documentation, including the regression-suite tutorial EkLine wrote and the MCP setup steps it corrected. Accurate, findable pages are what get cited.

Why does documentation accuracy matter so much for ContextQA?

Because ContextQA's own testing agents read its documentation. A page that falls out of date does not just cost a customer time; it feeds outdated information into the product. Current docs improve the product itself, not only the customer experience.

Who is ContextQA?

ContextQA is an AI-powered software testing platform. Teams use it to build and run web, mobile, and API tests, with AI that generates tests and self-heals them when the interface changes.

Ready for docs that keep up with your product?

EkLine takes about 20 minutes to set up, and ships at least 15 improvements to your knowledge base in the first 15 days. If you do not see the value, you walk away with 15 better pages. Start your trial.

Built with EkLine, the AI-native documentation platform for engineering teams.