Evalkeeper
    Preparing search index...

    Function casesInInstructions

    • The cases whose input text appears in the instructions: all of one of its texts (when at least minWhole characters), or a run of whole words at least minChars long, after evening out case and punctuation. Use it to warn, not to fail: an example may belong in the instructions, and then the case should be rewritten or retired.

      Parameters

      • cases: readonly { id: string; input: unknown; title: string }[]

        Each case's id, title, and input.

      • instructions: readonly string[]

        Everything the model is told: system text, prompt templates, schema and tool descriptions (instructionParts in @crowandraven/evalkeeper/ai-sdk gives these).

      • opts: { minChars?: number; minWhole?: number } = {}

        Other lengths than IN_INSTRUCTIONS_MIN_CHARS and IN_INSTRUCTIONS_MIN_WHOLE.

      Returns CaseInInstructions[]

      The cases found, each with its longest matching words.

      casesInInstructions(
      [{ id: "c1", title: "Broncos", input: { message: "the Broncos are hopeless" } }],
      ["Not hopeless: “The Broncos are hopeless!” is about a team."],
      ); // [{ caseId: "c1", title: "Broncos", found: "the broncos are hopeless" }]