Agent Runtime

Package: @tagna/udiot/agents Added: v0.16.0 Runtime: Node.js 22.17+

The agent runtime turns UDIOT's older single-call AI helpers into reusable building blocks for multi-step AI products. It is provider-agnostic, observable, retryable, and designed to be composed with telemetry and sandbox workspaces.

What It Provides

OpenAI support is installed with UDIOT. Anthropic support is an optional peer; install @anthropic-ai/sdk in the consuming app before using AnthropicProvider or createLLMClient({ provider: 'anthropic' }).

Basic Completion

import { createLLMClient } from '@tagna/udiot/agents';

const llm = createLLMClient({
  provider: 'openai',
  defaultModel: 'gpt-4o-mini',
});

const response = await llm.complete({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Summarize UDIOT in one sentence.' }],
  runId: 'run_123',
  agent: 'docs',
  step: 'summarize',
});

console.log(response.text);
console.log(response.tokensIn, response.tokensOut);

runId is optional. When present, the client publishes a RunEvent with token counts, duration, cost, cache status, and outcome.

Token counts are always reported. Dollar cost is optional: UDIOT ships no vendor prices, because they change too often to keep correct in a framework. To get costUSD, pass your own prices (USD per million tokens). Dated ids such as gpt-4o-2024-08-06 use the longest matching entry. Without a price, costUSD is undefined, never a misleading 0.

const llm = createLLMClient({
  provider: 'openai',
  priceTable: {
    'gpt-4o-mini': { inputPer1MTok: 0.15, outputPer1MTok: 0.6 },
  },
});

apiKey belongs to one vendor: the one named by provider, else OpenAI unless the key starts with sk-ant-. It is never sent to the other vendor. Use openaiApiKey and anthropicApiKey to configure both.

Structured Output

Use AgentRunner when you need a stable JSON shape. The runner retries when the model returns invalid JSON or a value that fails the schema.

import { AgentRunner, createLLMClient } from '@tagna/udiot/agents';
import { z } from 'zod';

const llm = createLLMClient({ provider: 'anthropic' });

const classifier = new AgentRunner({
  name: 'ticket-classifier',
  systemPrompt: 'Classify support tickets. Return JSON only.',
  model: 'claude-haiku-4-5',
  llm,
  outputSchema: z.object({
    priority: z.enum(['low', 'medium', 'high']),
    summary: z.string(),
  }),
});

const result = await classifier.run('Checkout fails after payment redirect.');
console.log(result.output.priority);

Tool Use

ToolBox exposes application functions to a model. Each tool has a Zod input schema, so invalid calls are rejected before the function runs. The schema sent to the model keeps descriptions, string and number limits, nullability, defaults, records and tuples, and closes objects with additionalProperties: false. Zod 4 schemas (zod/v4) are converted with Zod's own converter. A type with no JSON Schema form, such as z.function(), throws at describe() instead of reaching the model as {}.

import { createLLMClient, createToolBox, fromGenerator } from '@tagna/udiot/agents';
import { z } from 'zod';

const tools = createToolBox();

tools.register(
  fromGenerator({
    name: 'lookupOrder',
    description: 'Look up an order by id.',
    inputSchema: z.object({ orderId: z.string() }),
    generator: async ({ orderId }) => ({ orderId, status: 'paid' }),
  }),
);

const llm = createLLMClient({ provider: 'openai' });

const response = await llm.runWithTools({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'What is the status for order A123?' }],
  tools,
  maxToolIterations: 3,
  runId: 'order_run_123',
});

Tools can declare timeoutMs (the call is aborted through ctx.signal) and requiresApproval (each call needs approveToolCall to return true, for example after asking a person). A run can set toolTimeoutMs as a default, parallelToolCalls: true to run one turn's calls concurrently, and budget: { maxTokens, maxCostUSD } to stop with code BUDGET_EXCEEDED. promptCaching: true marks the system prompt and tool definitions as cacheable for Anthropic; OpenAI caches long prompts automatically.

tools.register({
  name: 'refundOrder',
  description: 'Refund an order in full.',
  inputSchema: z.object({ orderId: z.string() }),
  requiresApproval: true,
  timeoutMs: 10_000,
  execute: async ({ orderId }, ctx) => payments.refund(orderId, { signal: ctx.signal }),
});

await llm.runWithTools({
  model: 'gpt-4o-mini',
  messages,
  tools,
  approveToolCall: async ({ name, input }) => confirmWithOperator(name, input),
  budget: { maxTokens: 50_000 },
});

runWithTools() uses the same retry policy, circuit breaker, cost accounting and run events as complete(). Each assistant turn that requests tools is kept in the conversation with its toolCalls, which OpenAI and Anthropic require before they accept the tool results. Arguments that are not valid JSON are reported back to the model as a tool error instead of running the tool. When you build a tool conversation by hand, set toolCalls on the assistant message and toolCallId on each tool message that answers it.

Agent Loops

AgentLoop is for deterministic orchestration around one or more steps. It is not a model by itself. You provide state and steps; UDIOT handles iteration limits, final outcome, duration, and optional run-event publishing.

import { AgentLoop } from '@tagna/udiot/agents';

const loop = new AgentLoop<{ attempts: number }>({
  name: 'retry-build',
  initial: { attempts: 0 },
  maxIterations: 3,
  steps: [
    {
      name: 'run-build',
      async run(state) {
        if (state.attempts >= 2) return null;
        return { attempts: state.attempts + 1 };
      },
    },
  ],
});

const result = await loop.run();
console.log(result.outcome);

The default oscillation detector hashes the state at the end of each iteration and stops when an end state repeats, so steps that pass the state through unchanged are fine. If a step throws, the loop publishes a loop:end event with outcome failure and rethrows the error.

Caching And Retries

createLLMClient() installs an in-memory cache by default. Calls only use it when cacheKey is provided:

await llm.complete({
  model: 'gpt-4o-mini',
  cacheKey: 'docs:summarize:v1',
  messages: [{ role: 'user', content: 'Summarize the README.' }],
});

Transient provider failures such as 429, 502, 503, 504, timeout, and connection reset are retried with exponential backoff and jitter, waiting at least as long as a provider's Retry-After header asks (a request to wait more than a minute fails immediately). Aborting the request's signal stops a pending backoff. A per-provider circuit breaker opens after sustained transient failures; client errors such as 400 and caller aborts do not count.

Migrating From AIClient

AIClient and createAIClient() still work from @tagna/udiot, but they are compatibility shims over LLMClient.

Before:

import { createAIClient } from '@tagna/udiot';

const ai = createAIClient(process.env.OPENAI_API_KEY);
const code = await ai.generateCode('Create a Button component');

After:

import { createLLMClient } from '@tagna/udiot/agents';

const llm = createLLMClient({ provider: 'openai' });
const response = await llm.complete({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Create a Button component' }],
});

const code = response.text;

Use AgentRunner when you need JSON output and ToolBox when the model must call framework or application functions.