Signal · guide

A test without a model

Hand Signal a client of your own and it answers in place of the provider. Your test says what the model writes, and Signal does the rest as it always does.

Install it

pnpm add @niscorp/signal zod

Step 1 of 3

What the model will write

Two replies, in the order they are asked for. The first has the seats as a word.

What the model will write

import { z } from 'zod';
import { createSignal } from '@niscorp/signal';

const replies = ['{ "show": "The Tempest", "seats": "two" }', '{ "show": "The Tempest", "seats": 2 }'];

Step 2 of 3

A client that answers with them

Signal calls a provider through chat.completions.create, and reads the answer as a stream of chunks. This one hands back the next reply as one chunk. (step reads one whole completion instead of chunks.)

Signal is given it as client. No package is loaded and no request leaves the machine. A key is still asked for: any string will do.

A client that answers with them

const client = {
  chat: {
    completions: {
      create: async () =>
        (async function* () {
          yield { choices: [{ index: 0, delta: { content: replies.shift() ?? '' }, finish_reason: null }] };
          yield { choices: [{ index: 0, delta: {}, finish_reason: 'stop' }] };
        })(),
    },
  },
};

const signal = createSignal('groq', { client, apiKey: 'test' });

Step 3 of 3

The call under test

From here it is the code you would ship. Signal checks the first reply against the schema, retries with what was wrong, and takes the second.

The call under test

const Booking = z.object({ show: z.string(), seats: z.number() });

const { response, meta } = await signal
  .schema(Booking)
  .onRetry((error) => console.log('failed:', error.message))
  .complete('Two for The Tempest on Saturday, please.');

console.log(response, '· retries:', meta.retries);

Then

Run it.

No model was asked. The first line is what Signal said of the first reply, and the second is the booking it took from the next one.

Run it

npx tsx test.mts

It prints

failed: Response failed schema validation: seats: Invalid input: expected number, received string
{ show: 'The Tempest', seats: 2 } · retries: 1