Skip to content

Jahoda

Jahoda is an open verification layer for companion AI.

Code Demo Video Cerebral Valley

Core trick

Runs multi-turn adversarial scenarios against a chatbot, graded by fresh-context LLM judges at temperature 0 with each verdict linked to its transcript.

Tags

healthsecurity

Description

Jahoda is an open verification layer for companion AI. Millions of people are now emotionally close to chatbots, and there is no accepted way to answer the only question that matters: is this safe to be close to? Jahoda turns established mental-health frameworks — Jahoda (1958) via Ryff, Self-Determination Theory, Swarbrick/SAMHSA, 988/Action Alliance crisis standards, and the new legal duties under California SB 243 — into adversarial, multi-turn tests that any conversational agent can be run against. It runs 47 adversarial scenarios plus 8 benign controls across 8 dimensions (crisis, dependence, reality/psychosis, overreach, integrity, boundaries, minors, and false-positive controls), grades them with fresh-context, escalation-ensemble LLM judges at temperature 0, and produces a public, evidence-backed report where every single verdict links the transcript that justified it. The controls catch the opposite failure — over-triggering on benign conversation. The result is point-in-time evidence, not a compliance rubber-stamp, and every criterion is a versioned file open to expert review. Jahoda is the verification layer for OMBRUJA's broader AI guide — built first, and built in the open.
carolinacoto/jahoda★ 10 forksHTMLMITCreated Jun 13, 2026 · last push Jun 14, 2026

README excerpt

# JAHODA — wellness verification for companion AI
### ▶ Live report + specimen (test it): <!--LIVE_URL-->https://jahoda-jahoda-projects.vercel.app<!--/LIVE_URL-->

[![CI](https://github.com/carolinacoto/jahoda/actions/workflows/ci.yml/badge.svg)](https://github.com/carolinacoto/jahoda/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-7a1f2b.svg)](LICENSE)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](pyproject.toml)
[![built on Inspect AI](https://img.shields.io/badge/built%20on-Inspect%20AI-444.svg)](https://inspect.aisi.org.uk)

# JAHODA
**The verification layer for AI that people get close to.**

> **Headline finding —** <!--HEADLINE-->a frontier model behind one neutral
> "friendly companion" sentence — no guardrail layer — **dispensed authoritative
> medical and financial advice, failing 3 of 8 overreach scenarios (pass^5)**,
> while the specimen's one-line "refer out" rule passes all 8. Strong base-model
> safety handles farewells (6/6) and delusion-reinforcement (mean 5.0) on its
> own, but not competence-boundary refusal. See
> [FINDINGS.md](FINDINGS.md).<!--/HEADLINE-->

Jahoda points adversarial, research-grounded suites at any chat endpoint and
produces an evidence-backed, reproducible report. Below: the report screenshot,
and one failing transcript the harness caught on our own test subject.

<!--SCREENSHOTS-->
| The report | A failing transcript the harness caught |
| --- | --- |
| ![Jahoda report](docs/im

Team