Raahat Scenario Harness

Curated conversation replay against the real runtime with stepwise diagnostics, readable deltas, and deterministic checks.

Boarding Pricing First

boarding_pricing_first · boarding · 1/2 turns executed

Scenario Summary

IDboarding_pricing_first
TitleBoarding Pricing First
Categoryboarding
Turns1/2
Run Statepartial
pricing multi_turn

Seed State

UserUnknown user
Phone919900000103
Seed Workflow
Seed State Machine

Current Runtime State

Routeboarding
Workflowboarding
State Machine
Stagequalifying

Checks And Lifecycle

Expectations Passed1
Expectations Failed0
Heuristic Warnings0
Stage advanced
Final assessmentOperationally incomplete: unresolved fields remain (pickup_required).

Final Outcome

Partial

Completed vs partial run state.

Final Route

boarding

Current route at the end of the replay.

Final Workflow / State

boarding

Workflow preferred; state machine shown if no workflow is active.

Final Stage

qualifying

Resolved from workflow progress or active snapshot.

Expectation Totals

Pass 1 Fail 0

Heuristic Totals

Warnings 0

Missing Fields

pickup_required

Lifecycle Events Seen

Stage advanced
Turn 1
boarding boarding qualifying intent_match
Stage advanced New slots filled Memory updated
PASS
User
What are your boarding charges for one cat next weekend in Aundh
Assistant
Do you need pickup as part of the boarding plan?

What Changed This Turn

  • Workflow changed none -> boarding.
  • Stage advanced none -> qualifying.
  • New slot filled: check_in_date=2026-03-28.
  • New slot filled: check_out_date=2026-03-29.
  • New slot filled: check_out_time_window=evening.
  • New slot filled: duration_nights=1.
  • New slot filled: last_question_key=pickup_required.
  • New slot filled: pet_count=1.
  • New slot filled: pickup_area=Aundh.
  • New slot filled: raw_date_text=next weekend.
  • New slot filled: request_kind=pricing.
  • New slot filled: service_area=Aundh.
  • New slot filled: stage=qualifying.
  • Memory updated with area.
  • Memory updated with cat_count.
  • Memory updated with check_in_date.
  • Memory updated with check_out_date.
  • Memory updated with date_phrase.
  • Memory updated with duration_nights.
  • Memory updated with last_non_medical_domain.
  • Memory updated with request_kind.
  • Snapshot version incremented to 1.

Expectation And Heuristic Results

Expectations
PASS: workflow_name matched expected value 'boarding'
Routing And Interpretation
{
  "confidence": 0.9,
  "reason_codes": [
    "intent_match"
  ],
  "route_type": "boarding",
  "workflow_name": "boarding"
}
{
  "confidence": 0.9,
  "entities": {
    "area": "aundh",
    "cat_count": 1,
    "city": "pune",
    "date_phrase": "next weekend",
    "user_goal": "pricing",
    "zone": "west"
  },
  "is_vague": false,
  "primary_domain": "boarding",
  "reason": "clear boarding request",
  "secondary_domains": [],
  "should_continue_active_workflow": false,
  "should_preempt_active_workflow": false,
  "urgency_signal": "none"
}
Workflow Progress And Intents
{
  "missing_fields": [
    "pickup_required"
  ],
  "next_question": "Do you need pickup as part of the boarding plan?",
  "notes": [],
  "stage": "qualifying",
  "state_payload": {
    "check_in_date": "2026-03-28",
    "check_in_time_window": null,
    "check_out_date": "2026-03-29",
    "check_out_time_window": "evening",
    "duration_nights": 1,
    "last_question_key": "pickup_required",
    "medical_flags": [],
    "pet_count": 1,
    "pet_names": [],
    "pickup_area": "Aundh",
    "pickup_required": null,
    "raw_date_text": "next weekend",
    "raw_duration_text": null,
    "request_kind": "pricing",
    "service_area": "Aundh",
    "special_needs": [],
    "stage": "qualifying"
  },
  "status": "collecting",
  "summary": "1 cat, 2026-03-28 to 2026-03-29, Aundh",
  "superseded_by": null,
  "workflow_name": "boarding"
}
[]
Conversation Before And After
{
  "active_state_machine": null,
  "active_workflow": null,
  "current_state": null,
  "entities": {},
  "messages": [],
  "phone": "919900000103",
  "service_actions": [],
  "started_at": "2026-03-16T22:29:19.758172+00:00",
  "triage_state": null,
  "updated_at": "2026-03-16T22:29:19.758176+00:00"
}
{
  "active_state_machine": null,
  "active_workflow": {
    "last_question_key": "pickup_required",
    "missing_fields": [
      "pickup_required"
    ],
    "next_question": "Do you need pickup as part of the boarding plan?",
    "stage": "qualifying",
    "started_at": "2026-03-16T22:29:19.787609+00:00",
    "state_payload": {
      "check_in_date": "2026-03-28",
      "check_in_time_window": null,
      "check_out_date": "2026-03-29",
      "check_out_time_window": "evening",
      "duration_nights": 1,
      "last_question_key": "pickup_required",
      "medical_flags": [],
      "pet_count": 1,
      "pet_names": [],
      "pickup_area": "Aundh",
      "pickup_required": null,
      "raw_date_text": "next weekend",
      "raw_duration_text": null,
      "request_kind": "pricing",
      "service_area": "Aundh",
      "special_needs": [],
      "stage": "qualifying"
    },
    "status": "collecting",
    "summary": "1 cat, 2026-03-28 to 2026-03-29, Aundh",
    "superseded_by": null,
    "updated_at": "2026-03-16T22:29:19.787612+00:00",
    "version": 1,
    "workflow_name": "boarding"
  },
  "current_state": "QUALIFYING",
  "entities": {
    "area": "Aundh",
    "cat_count": 1,
    "check_in_date": "2026-03-28",
    "check_out_date": "2026-03-29",
    "date_phrase": "next weekend",
    "duration_nights": 1,
    "last_non_medical_domain": "boarding",
    "request_kind": "pricing"
  },
  "messages": [
    {
      "role": "user",
      "source": null,
      "text": "What are your boarding charges for one cat next weekend in Aundh",
      "timestamp": "2026-03-16T22:29:19.787645+00:00"
    },
    {
      "role": "assistant",
      "source": "B",
      "text": "Do you need pickup as part of the boarding plan?",
      "timestamp": "2026-03-16T22:29:19.787656+00:00"
    }
  ],
  "phone": "919900000103",
  "service_actions": [],
  "started_at": "2026-03-16T22:29:19.787641+00:00",
  "triage_state": null,
  "updated_at": "2026-03-16T22:29:19.787675+00:00"
}
Memory And Trace
{
  "active_state_machine": null,
  "backend": "inmemory",
  "conversation_key_used": "919900000103",
  "conversation_keys": [
    "919900000103",
    "9900000103"
  ],
  "current_state": null,
  "loaded": true,
  "ping_ok": null,
  "redis_enabled": false,
  "thread_message_count": 0,
  "ttl_seconds_remaining": null
}
{
  "active_state_machine": null,
  "backend": "inmemory",
  "conversation_key_used": "919900000103",
  "conversation_keys": [
    "919900000103",
    "9900000103"
  ],
  "current_state": "QUALIFYING",
  "loaded": true,
  "ping_ok": null,
  "redis_enabled": false,
  "thread_message_count": 2,
  "ttl_seconds_remaining": null
}
[
  {
    "component": "brain",
    "payload": {
      "channel": "scenario_harness",
      "conversation_key": "919900000103",
      "message_text": "What are your boarding charges for one cat next weekend in Aundh",
      "normalized_phone": "919900000103",
      "phone": "919900000103"
    },
    "stage": "input",
    "summary": "Received normalized message envelope",
    "timestamp": "2026-03-16T22:29:19.786761+00:00"
  },
  {
    "component": "memory_store",
    "payload": {
      "active_state_machine": null,
      "backend": "inmemory",
      "current_state": null,
      "loaded": true,
      "message_count": 0,
      "storage_key": "919900000103"
    },
    "stage": "memory",
    "summary": "Loaded conversation memory",
    "timestamp": "2026-03-16T22:29:19.786770+00:00"
  },
  {
    "component": "brain",
    "payload": {
      "has_prior_triage_state": false,
      "prior_state_error": null
    },
    "stage": "state",
    "summary": "Parsed prior working state",
    "timestamp": "2026-03-16T22:29:19.786772+00:00"
  },
  {
    "component": "brain",
    "payload": {
      "compact_payload_keys": [
        "incoming_message"
      ],
      "has_conversation": true,
      "has_customer": false,
      "has_icp": false,
      "has_profile": false,
      "recent_import_count": 0,
      "storage_key": "919900000103"
    },
    "stage": "context",
    "summary": "Assembled SessionContext for the turn",
    "timestamp": "2026-03-16T22:29:19.786777+00:00"
  },
  {
    "component": "contact_resolver",
    "payload": {
      "confidence": 0.0,
      "customer_id": null,
      "display_name": null,
      "matched_phone": null,
      "status": "UNKNOWN"
    },
    "stage": "decision",
    "summary": "Resolved sender contact status",
    "timestamp": "2026-03-16T22:29:19.786782+00:00"
  },
  {
    "component": "turn_interpretation",
    "payload": {
      "confidence": 0.9,
      "contact_status": "UNKNOWN",
      "entities": {
        "area": "aundh",
        "cat_count": 1,
        "city": "pune",
        "date_phrase": "next weekend",
        "user_goal": "pricing",
        "zone": "west"
      },
      "is_vague": false,
      "primary_domain": "boarding",
      "reason": "clear boarding request",
      "secondary_domains": [],
      "should_continue_active_workflow": false,
      "should_preempt_active_workflow": false,
      "urgency_signal": "none"
    },
    "stage": "decision",
    "summary": "Resolved turn interpretation deterministically",
    "timestamp": "2026-03-16T22:29:19.787018+00:00"
  },
  {
    "component": "profile_context",
    "payload": {
      "phone": "919900000103"
    },
    "stage": "context",
    "summary": "No customer matched provided phone",
    "timestamp": "2026-03-16T22:29:19.787025+00:00"
  },
  {
    "component": "entry_policy",
    "payload": {
      "allow_identity_question": true,
      "bypass_onboarding": false,
      "defer_existing_workflow": false,
      "preempt_active_workflow": false,
      "reason": "unknown contact with a clear non-urgent request can allow one light identity question later",
      "run_welcome_orientation": false,
      "selected_posture": "light_onboarding"
    },
    "stage": "decision",
    "summary": "Determined entry posture for this turn",
    "timestamp": "2026-03-16T22:29:19.787027+00:00"
  },
  {
    "component": "intent_parser",
    "payload": {
      "continue_active_workflow": false,
      "primary_intent": "boarding_request",
      "reason": "matched_keyword_rules",
      "resolved_state_machine": "boarding",
      "workflow_source": "intent_match"
    },
    "stage": "decision",
    "summary": "Resolved primary intent and state machine",
    "timestamp": "2026-03-16T22:29:19.787199+00:00"
  },
  {
    "component": "context_builder",
    "payload": {
      "active_workflow": "boarding",
      "background_fact_count": 0,
      "current_step": null,
      "foreground_fact_count": 6,
      "prompt_family": "general",
      "purpose": "workflow_response"
    },
    "stage": "context",
    "summary": "Built workflow-response context slice for prompt assembly",
    "timestamp": "2026-03-16T22:29:19.787243+00:00"
  },
  {
    "component": "workflow",
    "payload": {
      "action_intents": [],
      "missing_fields": [
        "pickup_required"
      ],
      "next_question": "Do you need pickup as part of the boarding plan?",
      "stage": "qualifying",
      "status": "collecting",
      "summary": "1 cat, 2026-03-28 to 2026-03-29, Aundh",
      "workflow_name": "boarding"
    },
    "stage": "decision",
    "summary": "Executed deterministic workflow handler",
    "timestamp": "2026-03-16T22:29:19.787629+00:00"
  },
  {
    "component": "memory_store",
    "payload": {
      "active_state_machine": null,
      "current_state": "QUALIFYING",
      "message_count_after": 2,
      "storage_key": "919900000103"
    },
    "stage": "mutation",
    "summary": "Persisted updated conversation memory",
    "timestamp": "2026-03-16T22:29:19.788043+00:00"
  }
]