Cortex · guide

A run that waits

A run that is waiting for a person can be saved as JSON, and resumed from it later. This file does both halves in one process; in an app they are two requests, or two servers.

Install it

pnpm add @niscorp/cortex @niscorp/signal zod openai

Step 1 of 4

The model

createSignal makes the client that calls the model. Cortex is handed this client, here for Groq, and Signal reads the key from GROQ_API_KEY.

The model

import { z } from 'zod';
import { createSignal } from '@niscorp/signal';
import { defineAgent, defineTool, resumeRun } from '@niscorp/cortex';
import type { RunSnapshot } from '@niscorp/cortex';

const llm = createSignal('groq');

Step 2 of 4

An agent that always asks

policy is the short way to write the common gates. This one says the refund tool needs a person’s approval, by the tool’s id.

An agent that always asks

const refund = defineTool({
  id: 'refund',
  name: 'refund',
  description: 'Refund a booking to the card it was paid with',
  input: z.object({ booking: z.string(), amount_cents: z.number() }),
  execute: ({ booking, amount_cents }) => ({ booking, refunded_cents: amount_cents }),
});

const desk = defineAgent({
  id: 'desk',
  instructions: 'Refund a booking when the customer asks for it.',
  tools: [refund],
  policy: { tools: { requireApproval: ['refund'] } },
});

Step 3 of 4

Save it where it stops

run.snapshot() is the whole run as plain JSON: the conversation so far, the call that is waiting, and what has been used. It can go in a database row.

The run is then aborted here. Nothing has to stay in memory while a person decides.

Save it where it stops

const first = desk.run('Please refund booking B-2041, 58 euros.', { llm });
const saved = await new Promise<string>((resolve) => {
  first.onEvent((event) => {
    if (event.type === 'approval-required') resolve(JSON.stringify(first.snapshot()));
  });
});
first.abort('saved');
console.log('saved', saved.length, 'characters');

Step 4 of 4

Resume it

resumeRun takes the agent and the document. The resumed run asks its gates again for the waiting call, so approval-required is sent again, with a new id.

Answer with the id from that event. The id inside the saved document belongs to the run that was saved.

Resume it

const snapshot = JSON.parse(saved) as RunSnapshot;
const second = resumeRun(desk, snapshot, { llm });
second.onEvent((event) => {
  if (event.type === 'approval-required') second.approve(event.approval.id);
});

const result = await second.result;
console.log(result.ok ? result.output.response : result.error.message);

Then

Run it.

This file asks a model, so it is not run on this site. When the model asks for the refund, it prints how long the saved JSON is, and then the model’s sentence saying the refund was made: the function ran in the second run, after the approval.

Run it

GROQ_API_KEY=your-key npx tsx waits.mts