TC96 — Vectara — R6 Managed RAG v1 (Grounded Generation, sources disagree) Query: What is the warranty period on a new Tolvan A3? Endpoint: POST /v2/corpora/r6-managed-rag-ss-v1/query with generation:{} (Vectara defaults, default prompt template, not overridden) ==================================================================================================== GENERATED ANSWER (verbatim) ==================================================================================================== I do not have enough information to accurately determine the warranty period on a new Tolvan A3, as the search results provide conflicting information. factual_consistency_score (Vectara's own HHEM-based score): 0.3359375 ==================================================================================================== GRADING CHECK (S-8 count) ==================================================================================================== A08 (sales pack, 24 months) and A09 (service pack, 36 months) BOTH retrieved — rank 1 and rank 2, near-identical top scores (0.9997 each) — so this run had the chance to surface or silently resolve the conflict. Count: 1 of 1 run retrieved both conflicting documents; of that 1, 0 presented one value silently as THE fact. The answer explicitly states the search results conflict and declines to pick one — this is the passing behaviour (surfacing the conflict), not the failing one (silently presenting 24 or 36 months alone, or averaging them to 30). ==================================================================================================== TOP 6 RETRIEVED PASSAGES ==================================================================================================== --- rank 1 | score=0.999733 | doc=A09_warranty_statement_service_pack.pdf --- The warranty period Every new Tolvan A3 carries a warranty of 36 months from the date the instrument is delivered. --- rank 2 | score=0.999733 | doc=A08_warranty_terms_sales_pack.pdf --- The warranty period Every new Tolvan A3 carries a warranty of 24 months from the date the instrument is delivered. --- rank 3 | score=0.990910 | doc=A10_tolvan_a3_product_overview.md --- Every instrument leaves the laboratory with a signed certificate recording theas-found and as-left bias at four reference points across the working range.The recommended recalibration interval for the Tolvan A3 is 22 months from the date of the last certificate. --- rank 4 | score=0.965835 | doc=A08_warranty_terms_sales_pack.pdf --- They apply to every new Tolvan A3 supplied by Fenwake Instruments Ltd, however it was ordered and wherever it is deployed. --- rank 5 | score=0.965835 | doc=A09_warranty_statement_service_pack.pdf --- They apply to every new Tolvan A3 supplied by Fenwake Instruments Ltd, however it was ordered and wherever it is deployed. --- rank 6 | score=0.961822 | doc=A09_warranty_statement_service_pack.pdf --- Warranty Terms - Tolvan A3 ==================================================================================================== Remaining results (rank 7-10, score-only) ==================================================================================================== rank 7: score=0.961822 doc=A08_warranty_terms_sales_pack.pdf rank 8: score=0.527636 doc=A10_tolvan_a3_product_overview.md rank 9: score=0.527636 doc=A10_tolvan_a3_product_overview.md rank 10: score=0.436262 doc=A10_tolvan_a3_product_overview.md