Ashr Labs TypeScript SDK

Test your AI agent against real scenarios, then watch it in production. One SDK, one API key, two products:

• Testing Platform — generate datasets, run your agent against every scenario offline, compare expected vs. actual behavior, and submit results for server-side grading. This is the EvalRunner / tool() / agent-adapter surface, and it's where most people start.
• Observability — trace your agent's behavior in production (LLM calls, tool invocations, latency, errors). This is the client.trace() surface. Traces render in the Observability panel of the dashboard. Requires the observability feature flag on your tenant.

The two are independent. You can run evals without ever touching tracing, and vice versa.

Install

npm install ashr-labs

Node.js 18+ (the SDK uses the built-in fetch). Zero runtime dependencies — the core ships nothing transitive. The Anthropic and OpenAI agent adapters use those providers' SDKs, declared as optional peer dependencies you install only if you use them. See Installation.

60-second example

Define your agent:

import { AnthropicAgent, tool } from "ashr-labs";

const lookupOrder = tool({
  name: "lookup_order",
  description: "Look up the status of a customer order.",
  parameters: {
    type: "object",
    properties: { order_id: { type: "string" } },
    required: ["order_id"],
  },
  fn: ({ order_id }) => shop.getOrder(order_id as string),
});

const agent = new AnthropicAgent({
  model: "claude-fable-5",
  system: SYSTEM_PROMPT,
  tools: [lookupOrder],
});

Run it against a dataset:

import { AshrLabsClient, EvalRunner } from "ashr-labs";

const client = new AshrLabsClient("tp_your_api_key_here");

const runner = await EvalRunner.fromDataset(client, 818);
const created = await runner.runAndDeploy(agent, client, 818);

const graded = await client.pollRun(created.id as number);
const m = (graded.result as Record<string, unknown>)
  .aggregate_metrics as Record<string, number>;
console.log(`Passed: ${m.tests_passed}/${m.total_tests}`);

Any object with respond() and reset() works as an agent — see Testing Your Agent.

Dataset IDs are shown in hex on the dashboard. #00000332 is integer 818 — pass 818. See Quick Start.

Where to go next

You want to…	Read
Get an API key and make your first call	Quick Start
Install and verify the SDK	Installation
Run a full eval end to end	Testing Your Agent — start here for evals
Cut your setup code with tool() + adapters	API Reference → Tools / Agent adapters
Trace your agent in production	Observability
Look up an exact method signature	API Reference
Handle errors	Error Handling
Authenticate / manage API keys	Authentication
Copy-paste recipes (CI, batch, reporting)	Examples

Advanced surface: VM Integration for browser/desktop agents that attach session logs to results.

Support

Questions or feature requests: support@ashr.io or book a call.