The operation, how to run the app's detector, its thresholds, and any cases of the app's own.
A suite to register with the app's other suites.
import { hopelessnessSuite } from "@crowandraven/evalkeeper/suites";
export const hopelessness = hopelessnessSuite({
operation: "detect_distress",
thresholds: { hopeless: 0.4 },
async run(input, ctx) {
const d = await detectHopeless(input.message, input.earlier, ctx.model);
ctx.onCost(d.costUsd);
return { now: d.now, before: d.before, pNow: d.pNow, via: d.via };
},
extraCases: [
{ message: "this job search is hopeless", group: "one-thing" },
],
});
A ready-made suite for an app that listens for hopelessness in chat: does the latest message, and separately the earlier ones, voice hopelessness about the person's own life? It ships with about 75 made-up messages in groups, from "every day is the same grey nothing" (counts) to "nothing helps with this cough" (doesn't), including jokes that cover real hopelessness and short replies such as "yeah, same" that agree with it. They arrive confirmed by the Evalkeeper Team as the suite's author (
HOPELESSNESS_REVIEWS), so they're in use as soon as they're added.These are made-up examples written as a starting point, not clinically validated: review and adapt them with people who have crisis-line or clinical experience before relying on them. The suite judges only whether the app counted a message; what the app does then (a gentler reply, a crisis line when it happens twice) is the app's own rule. Pair it with
crisisRiskSuitefor suicide and self-harm risk.