Type Alias RunEvalOptions

How to run a test: fresh (no reuse), and, for a test continued in a new request, its leg and the judge's spend so far.

type RunEvalOptions = {
    fresh?: boolean;
    judgeCostUsd?: number;
    leg?: number;
    requestStartedAt?: number;
}

Properties

fresh?: boolean

Never reuse an earlier test's output.

judgeCostUsd?: number

What earlier legs' judge spent.

leg?: number

Which leg this is (1 for a new test).

requestStartedAt?: number

When the request running this leg started, if earlier than this call (tests queued one after another): the time limit counts from then.