Evalkeeper
    Preparing search index...

    Type Alias CheckResult

    One check on one output. null: doesn't apply to this case.

    type CheckResult = {
        given?: string;
        key: string;
        matchedBy?: "meaning" | "words";
        matches?: { actual: string; expected: string; why: string }[];
        measures?: Record<string, number>;
        note?: string;
        pass: boolean | null;
        units?: { right: number; total: number };
    }
    Index
    given? key matchedBy? matches? measures? note? pass units?
    given?: string

    What a decision check's answer gave, right or wrong ("34% true, bar 40%", "bug (90%)"), so a run page can show the probability beside the bar it was read at.

    key: string
    matchedBy?: "meaning" | "words"

    "meaning": a judge model matched them; "words": shared words did.

    matches?: { actual: string; expected: string; why: string }[]

    For matching checks: which expected item matched which output, and why.

    measures?: Record<string, number>

    Numbers measured on this answer, averaged across a test: a decision's brier, logLoss, confidence, and correct (see decisionCheck).

    note?: string

    Why it failed (or a detail worth showing), in plain words.

    pass: boolean | null
    units?: { right: number; total: number }

    Partial credit: of the parts this answer should have (e.g. expected items), how many it got right. The headline score sums these.

    Settings
    Member Visibility

    On This Page
    Properties