Week of June 27–July 3, 2026
A lot shipped this week. Here's what's new.
Response Inspector
After a test run, you can now open any step result and inspect the raw API response directly. Write a JMESPath expression, see what it extracts in real time, edit it, and save it as a variable your next step can use.
This matters most when you're setting up Context Mapping for the first time, or debugging why an extracted value isn't what you expected. Instead of writing an expression, running the test, checking the result, tweaking, and repeating — you stay in the inspector and iterate without re-running the full test.
JMESPath for Context Mapping 🔥
Context Mapping now supports JMESPath expressions when mapping scenario data to test variables. If your scenarios contain nested objects or arrays, you can drill into them precisely rather than mapping only top-level fields. Combine this with the Response Inspector and you have a fast loop for setting up complex data-driven tests.
Live run progress
Tests now update in real time as they run. You see each result the moment it finishes rather than waiting for the suite to complete. For longer runs, this makes it much faster to spot a pattern — or a problem — early.
Stop on failure
New option per test: stop the run when a step fails rather than continuing to evaluate downstream steps. When you want fast feedback and don't want to spend evaluator tokens on steps that depend on something already broken, turn this on.
Dashboard trend charts 🔥
Project view now shows a pass-rate trend chart across recent runs. At a glance you can see whether quality is moving up or down over time, and catch regressions before they become patterns.
Context mapping preview
See exactly how your scenario data will map to test variables before you run anything. Each variable shows its resolved value for each scenario — so you know what the test will actually receive before it runs.
Boolean and number types in scenarios
Scenario fields now support boolean and number types alongside strings. If your API expects a boolean flag or a numeric ID, set it correctly in the scenario data rather than working around it with string values.
Also this week: switched the AI evaluator to OpenAI for more consistent qualitative scoring; fixed the Scenario Matrix for edge cases with no applicable scenarios; fixed spurious 100% confidence scores on some evaluations; improved LLM evaluator error handling.