OATHRA / DEVELOPER LAB
Change the words.
Inspect the verdict.
The existing evidence lab and saved Arena negotiation. Edit the utterance below to inspect the completion rules. This page does not place real phone calls.
Install v0.1.19 with the latest fixes · Copy buttons use the GitHub Release package. npm remains at 0.1.0.
This is the product’s own evidence engine. The agent has already declared “your reservation is confirmed”, but the shop has not confirmed anything — so confirmed is the one field still open. Swap the shop’s reply and the verdict changes. Your input stays in this tab; no phone call is placed.
Twenty-six seconds of the verdict changing.
A screen recording of the lab above, actually being used. Every time the shop’s reply is swapped, the same checker runs again. The camera moves, the cursor and the captions were added in the edit; the screen and the verdicts were not.
At this size the text inside the recording is not legible. Full screen makes the fields readable.
Add evidence checks to the voice agent you already have.
Keep your carrier and model. Use oathra verify in v0.1.19 to check saved transcripts locally and get JSON fields, missing requirements and supporting utterances. ActionProof also gives conversation, confirmation, system and outcome evidence one typed model. Provisional, pending-approval and confirmation-needed phrases do not count as completed reservations. No API key needed.
A typed SDK at oathra/evidence and a LiveKit integration example are included.
Check a transcript in your browser →
Use your own transcripts → · Star on GitHub to follow updates
Checks supported language rules; does not verify a venue’s booking system. The LiveKit example is type-checked, not tested on a live call.
25 seconds: from transcript to evidence
Actual browser interaction with the saved GPT-4o-mini / hotel simulator recording. This is not a real phone call. Try it yourself →
Follow up without pushing the callee.
After the required booking details settle, Oathra states the purpose and asks consent once. With consent, it asks one contract-declared question per turn. A decline, hold or ambiguous reply ends intake immediately; only explicit answers go to intake.json and summary.md.
48 seconds: consent-based follow-up intake
Recorded from the real Arena simulator: booking confirmed → consent → two explicit answers → intake.json. No API key or real phone call. View the scenario →
Watch success turn back into incomplete.
30 seconds · recorded interaction with the real evidence engine · captions, no audio. Browser frames captured at 4 fps and encoded at 30 fps. Not a phone-call recording.
Share your use case in Discussion #14 →
Follow-up intake can branch by scene with startAfter, dependsOn and choices. Declined, held, ambiguous or unmatched replies are not saved.
A real-model negotiation
GPT-4o mini negotiating with a simulated hotel that lists at ¥23,500. No real hotel was called. The run is redrawn at 3× from the saved events. Not a recording.
confirmed = yes (callee) lands at the top of the evidence. Nothing the agent says can fill that row.Play
Watch two agents haggle. Then answer the phone yourself.
Ten scenarios ship with the simulator: a restaurant that is full at 7 pm (in English too: oathra play restaurant-reservation-en, and a consented follow-up intake version), an “impossible” hotel with a secret floor price, a bulk purchase, a serial number read-back, a trap where every slot is full and the only correct answer is “I could not book it,” and three casual calls with a friend where “probably!” never counts as a promise. Pit models against each other and share the card.
$ npx oathra battle impossible-hotel \
--agent scripted --agent openai --agent gemini \
--png card.png --json run.json
Impossible Hotel
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
scripted ████████████████████ 6540
openai:gpt-4o-mini ███████████████████▋ 6444
gemini:gemini-flash-latest ███████████████████▌ 6410
SUCCESS TIME COST SCORE
scripted ✓ 69s $0.00 95.4
openai:gpt-4o-mini ✓ 95s $0.00 89.5
gemini:gemini-flash-latest ✓ 79s $0.00 90.1
False completions: 0
The run behind the video. All three closed under ¥20,000 (¥19,900 / ¥19,900 / ¥20,000) — and every “win” is anchored to the hotel saying “your reservation is confirmed.”
This run first produced one false completion: a hotel line that merely restated the price was being counted as a “yes”. The evidence engine was fixed and the same three calls re-run to zero. That is what the battle command is for. Points reflect price and time; cards export as SVG/PNG and the run as JSON.
The three agents calling at once (63 s, sound on; Japanese audio, English captions). The windows are the Arena replaying the recorded calls.
Try to fool it
In the app's practice (練習), choose 自分が相手役 and answer the AI's call as the clerk with replies like these. The evidence engine on its own treats none of them as confirmed; what the screen shows also depends on the rest of the conversation.
- Restaurant: “probably fine, but it’s not confirmed yet”→ not confirmed
- Restaurant: “7 pm is full, but 7:30 works”→ the refused 7 pm is not an offer; bare 7:30 remains unresolved until am/pm is clarified and the time is accepted
- Hotel: “you’re booked”, then “the rate is ¥23,500”→ the confirmation goes stale and has to be re-obtained
Found a phrasing that slips through? Open an issue. That is the most useful contribution.
Call
Same runtime, real phone. Bring your own carrier. Bring your own model.
Twilio keeps its direct media-streams fast path. Plivo and any SIP trunk go through a SIP gateway (LiveKit by default, Cloud or self-hosted). The voice engine is a separate choice: GPT-Live was the most natural on our test calls, so it is the default; a Deepgram + LLM + TTS pipeline is there when you want control or lower cost.
$ pnpm oathra setup phone # from a clone; use the v0.1.19 command above for the release package Choose voice engine ❯ GPT-Live Recommended · best conversation quality Gemini Live gemini-3.8-live · full-duplex Pipeline Deepgram + any brain + OpenAI TTS How would you like to make calls? ❯ Twilio direct media streams · verified Plivo SIP via gateway Custom SIP Setting up Twilio ✓ Verify credentials account · Full ✓ Find a voice-capable number +1 947 ··· ✓ Check outbound permission for Japan ✓ Media Streams configured per call ✓ Twilio is ready. $ npx oathra phone doctor --to +81… Carrier ✓ authenticated · active · caller ID · Japan enabled Media · requires a live call — oathra phone test Voice Engine ✓ GPT-Live (gpt-live-1) Latency Oathra → api.twilio.com 244 ms Cost JP mobile · twilio · $0.185/min READY ✓
- Guided, not one-click-liesProviders that need a human step (caller-ID verification, geo permissions, regulatory bundles) return a
user_actionwith a link; Oathra polls until it is done. Everything the API allows is automated. - Test without paying for minutes
oathra phone testhas three levels — Local (telephony ¥0, API usage), Gateway ¥0, PSTN paid. You stop burning a carrier minute on every bug. - Doctor tells you which layer brokeCarrier · SIP gateway · media · voice engine · latency · cost, with a fix line for each failed check.
Prove
「予約できました」 is not evidence.
Oathra returns a VerifiedResult. Each field is anchored to an utterance from the other party (and to audio on real calls). Ten thousand mutated callees produce zero false completions — reproducible with one command.
{
"field": "time",
"value": "19:30",
"source": "callee",
"transcript": "19時はいっぱいですが、19時半でしたら空いております。",
"span": "19時半",
"explicit": true,
"verified": true,
"note": "accepted with value restated by caller",
"audio": { "startMs": 14210, "endMs": 18960 }
}
Success = Connected
∧ date.verified ∧ time.verified
∧ partySize.verified ∧ confirmed.verified
∧ constraintsSatisfied
$ npx oathra eval --adversarial 10000
never-confirm · wrong-restate · negate-then-offer
silent-hangup · caller-echo-trap · tentative-hold
echo-question · confirm-then-change · cancel-after-confirm
hold-then-reply · voicemail · transfer · ask-back · dialect-confirm
False Completion: 0 / 10000 adversarial runs 6.2s
- Refusals never become offers. 「19時はいっぱいですが」 produces no claim for 19:00.
- Corrections supersede. 「13日、いや14日」 keeps the 14th, with a supersedes edge.
- Hedges do not confirm. 「たぶん大丈夫」 is neither agreement nor confirmation.
- Only the callee can confirm. The agent saying it booked is recorded and ignored.
- Confirmations go stale. Change the price after 「承りました」 and it must be re-confirmed.
- Walking away is not accepting. 「わかりました、他を探します」 verifies nothing.
Mix and match
Carriers on one side, voice engines on the other.
| Carrier | |
|---|---|
| Twilio direct | verified on real calls |
| Plivo sip | via gateway · PSTN unverified |
| Custom SIP sip | any trunk |
| Telnyx · Wavix · Sinch · didlogic | v0.2 |
| Yours | oathra provider create phone <id> |
| Voice engine | |
|---|---|
| GPT-Live recommended | full-duplex · $0.05/min |
| Gemini Live | gemini-3.8-live · full-duplex · not yet verified on a real line |
| Pipeline | Deepgram + any brain + TTS |
| Brains | OpenAI · Gemini · Ollama · scripted |
| Local | experimental |
OATHRA
│
Call Runtime ── Evidence Engine ── VerifiedResult
│
┌──────────┴──────────┐
Phone Layer Voice Layer
│ │
Twilio Direct · SIP Gateway GPT-Live · Gemini Live · Pipeline · Local
│
LiveKit Cloud / self-host → Plivo · Custom SIP · (Telnyx …)
API SETUP
Ask for only the keys you need, with the right link.
Start with the browser evidence lab or oathra demo; neither needs an API key. When you move to a real phone, setup phone asks for the selected engine’s keys first, then walks through your carrier.
- Enter credentialsOpenAI, Deepgram, Twilio, Plivo and LiveKit links are shown beside the prompt. Values are saved to
.env; secrets are masked in a TTY. - Run the doctor
oathra doctorandoathra phone doctoridentify missing layers and show a fix. - Test in stagesLocal (telephony ¥0, API usage) → Gateway (telephony ¥0) → PSTN (real call, paid).
$ pnpm oathra setup phone # from a clone; use the v0.1.19 command above for the release package Choose voice engine ❯ GPT-Live needs OPENAI_API_KEY Pipeline needs DEEPGRAM_API_KEY + OPENAI_API_KEY OpenAI API key Get it at: https://platform.openai.com/api-keys (input is masked) How would you like to make calls? ❯ Twilio direct media streams Plivo SIP via gateway Custom SIP SIP via gateway ✓ .env updated · run `oathra phone doctor` next
Unexpected result? Report the utterance sequence and result →
Ready to demonstrate. Still measuring real-world reliability.
L1 · Technical demonstration and scoping
Explore an implemented phone-agent runtime and evidence checks that expose incomplete outcomes. There are three documented development calls. A 100-call success rate, p95 response time and measured cost per call are not yet available.
Paid pilots (L2) still need consent, retention, outbound controls and tenant isolation. Production (L3) needs monitoring and incident response. No production SLA is offered.
Read the validation scope and next gates →Where does the data go?
Text entered on this page → the evidence engine in your browser. Input text is not transmitted.
Real call audio → your chosen carrier → Oathra → your chosen voice and AI providers. Recordings and events are stored in .oathra/calls/ on the machine running Oathra. Audio and transcripts reach external providers depending on your configuration.
What is verified and what is not (the current implementation status) lives in the README. Simulation evaluation results and the three development phone calls measure different things; neither establishes production reliability.
Start with two agents on the phone.
Want to try it on your own reservation, front-desk or confirmation calls, including real-call verification? Ask about a pilot → (Private inquiry form.)
Keep decisions with their evidence.
Saved calls also produce a human-readable summary.md and an intake.json record with verified decisions and utterance-linked answers. You can form an operational profile from explicit answers, while Oathra never infers a callee’s attributes or sensitive traits. Extra intake runs only after required details are settled and only when the contract declares its purpose, fields, question limit, consent and stop-on-decline behavior; consent and answers retain utterance IDs and timestamps, and an ambiguous or hesitant reply stops immediately.