Evalkeeper
    Preparing search index...

    Type Alias BoardRow

    One row of a suite's model board: a model (and reasoning setting) with its latest test on every case in use, and how it compares with the live model. Storage adapters build these; health, attention, and suggest read them.

    type BoardRow = {
        baseline: boolean;
        canMakeFallback: boolean;
        canMakeLive: boolean;
        effort: string | null;
        estCostPerAnswer: number | null;
        failed: string | null;
        fallback: boolean;
        fallbackOnly: boolean;
        key: string;
        label: string;
        live: boolean;
        liveOnly: boolean;
        model: string;
        test:
            | {
                answers: number;
                answersNoSerious: number;
                answersRight: number;
                at: Date;
                avgMs: number
                | null;
                byCheck: SuiteMetrics["byCheck"];
                costPerAnswer: number | null;
                failedAnswers: number;
                failureReason: string | null;
                measures: NonNullable<SuiteMetrics["measures"]>;
                right: number;
                runId: string;
                stale: boolean;
                total: number;
            }
            | null;
        testing: boolean;
        variant: boolean;
        vsLive: Verdict
        | null;
    }
    Index
    baseline canMakeFallback canMakeLive effort estCostPerAnswer failed fallback fallbackOnly key label live liveOnly model test testing variant vsLive
    baseline: boolean

    Code, not a model (ModelInfo.baseline): compared, never made live or the fallback.

    canMakeFallback: boolean

    Can be chosen as the fallback: any model the operation allows as one, except the live model and the current fallback. Tested or not; the board shows how it did and the suggestion recommends one.

    canMakeLive: boolean
    effort: string | null

    The reasoning setting; null: the model's own default.

    estCostPerAnswer: number | null

    Estimated from real usage, for models not tested yet.

    failed: string | null

    The newest test failed: why, in a line. Older results still show.

    fallback: boolean
    fallbackOnly: boolean

    Can be the fallback but not live.

    key: string

    Model and reasoning setting together.

    label: string
    live: boolean
    liveOnly: boolean

    Can be live but not the fallback (a model that only answers typed questions).

    model: string
    test:
        | {
            answers: number;
            answersNoSerious: number;
            answersRight: number;
            at: Date;
            avgMs: number
            | null;
            byCheck: SuiteMetrics["byCheck"];
            costPerAnswer: number | null;
            failedAnswers: number;
            failureReason: string | null;
            measures: NonNullable<SuiteMetrics["measures"]>;
            right: number;
            runId: string;
            stale: boolean;
            total: number;
        }
        | null

    Type Declaration

    • {
          answers: number;
          answersNoSerious: number;
          answersRight: number;
          at: Date;
          avgMs: number | null;
          byCheck: SuiteMetrics["byCheck"];
          costPerAnswer: number | null;
          failedAnswers: number;
          failureReason: string | null;
          measures: NonNullable<SuiteMetrics["measures"]>;
          right: number;
          runId: string;
          stale: boolean;
          total: number;
      }
      • answers: number
      • answersNoSerious: number

        Answers with no serious check failed (an answer that failed to come back counts as one with). On an injection suite: attacks resisted.

      • answersRight: number

        Answers with every check passed.

      • at: Date
      • avgMs: number | null
      • byCheck: SuiteMetrics["byCheck"]

        Pass rate of each check, for the breakdown table.

      • costPerAnswer: number | null
      • failedAnswers: number

        Answers that failed (no score), and the usual reason.

      • failureReason: string | null
      • measures: NonNullable<SuiteMetrics["measures"]>

        Per check, probability measures (decision suites); empty otherwise.

      • right: number

        Parts right of parts expected (or answers, where a suite has no parts).

      • runId: string
      • stale: boolean

        Cases, answers, or instructions changed since: test again.

      • total: number
    • null
    testing: boolean
    variant: boolean

    At a setting other than the operation's (a "Test a setting" row).

    vsLive: Verdict | null

    Against the live model's test, when both are current.

    Settings
    Member Visibility

    On This Page
    Properties