Skip to main content

Introducing Manifold

· 3 min read
Rohan Kulkarni
Co-Founder, Ashr

Your agent already generated its own training data​

Every enterprise running an LLM agent in production is sitting on the exact dataset it needs to make that agent better, and throwing it away.

The calls happen. Someone in ops notices the model misread a field on an invoice and fixes it by hand. The fix lands in a ticketing system, or a spreadsheet, or nowhere. Next month the same model makes the same mistake, because nothing connected the correction to the model that needed it.

Manifold is the thing that connects them.

What it does​

Manifold captures every interaction with a post-trained agent, freezes the good ones into versioned datasets, and turns them into the next fine-tune — then evaluates, promotes, and monitors it in production.

Concretely, six stages:

Capture. Wrap your LLM client and every call lands in an append-only ledger — model, messages, output, tokens, latency. Capture is fire-and-forget; it can never block or break your request path.

Correct. Domain experts mark what the model got wrong. Corrections accumulate against the interactions that produced them, not in a separate system that loses the link.

Freeze. Turn a slice of that ledger into an immutable, versioned dataset, split into train and holdout. Frozen means a result stays reproducible: you can always say which rows trained which model.

Train. Fine-tune an open-weight model on that cut, on managed GPUs. LoRA adapters, so a run takes hours, not weeks.

Evaluate. Score every candidate on the same frozen holdout, against the metric that matters for the task — not a generic benchmark. Candidates compete on fixed evidence; the comparison is apples to apples by construction.

Promote and monitor. Promote the winner behind an alias. Traffic moves without a redeploy, and the resulting calls land back in the ledger — which is the next dataset.

That last sentence is the whole point. It is a loop, not a pipeline.

Who it's for​

Manifold is built for teams that have an agent in production, real volume, and in-house expertise about what "correct" means:

  • Document processing — invoices, claims, contracts. Correctness is a schema and a set of fields, so it can be measured rather than vibed.
  • Browser and computer use — where a task either completed or it didn't, and frontier models are expensive per step.
  • Coding agents — where correctness is a test suite that passed or didn't, a task burns hundreds of calls, and the conventions of one codebase are exactly what a general model has to be re-told in every prompt.
  • Tool and function calling — where the failure mode is a malformed call, and your internal tools are nothing a general model has seen.

The common thread: the task is narrow, the volume is high, and your data is the part no frontier lab has.

It deploys into the customer's VPC, or a zero-data-retention alternative. For most of these teams that isn't a preference — it's the reason a general API was never an option.