The product worked exactly as designed
The team's browser tests passed. The landing page loaded. Plan selection worked. Signup created an account. The onboarding routes returned expected responses.
The unresolved question was human: could a person without internal context understand the page and complete the intended journey?
The designers had seen every iteration. The agent had access to the product brief. Neither could recreate a true first encounter.
The cold-start condition is part of the task
The Taskin brief states:
Scenario: “You lead operations at a 30-person software company. Start from this public landing page and try to create a trial workspace.”
The participant must confirm they have not previously seen the product or page. They receive no internal positioning document.
The brief asks them to record:
- what they think the product does after 20 seconds;
- who they believe it is for;
- which CTA they choose;
- where they hesitate;
- unfamiliar terms;
- expected price or trial conditions;
- whether signup succeeds;
- what they expect after entering the product;
- the first meaningful action they can identify.
The participant does not contact support, read internal documentation, enter real payment details, or disclose unnecessary personal information.
The human executes the scenario, not a general review
Taskin routes the request to a willing usability participant. They accept the scenario, device requirement, recording method, privacy rules, and compensation.
During execution, they attempt the task naturally. A high-severity issue needs evidence: a timestamp, screenshot, exact label, or failed action. The participant separates what happened from what they would change.
That distinction prevents the result from becoming a loose page critique.
Structured result
The workflow receives:
prior_exposure: false
scenario_state: completed, blocked, or abandoned
stopping_point: <route or component>
time_to_signup: <actual duration>
time_to_first_value: <actual duration or unavailable>
initial_product_description: <participant's words>
issues:
- severity
- step
- observation
- evidence
- participant_interpretation
- optional_recommendation
deviations: <actual exceptions>Acceptance test
The task passes when:
- the participant confirms the starting condition;
- every required stage is attempted;
- scenario state is explicit;
- high-severity findings have evidence;
- observed behavior and recommendations are separate;
- personal data is redacted;
- the result names uncertainty rather than inventing intent.
A participant failing to complete the flow can still complete the research task successfully. The failure is the evidence.
How the product team uses the result
The agent can group issues by page, component, and severity. It can identify repeated labels or routes and prepare a change list.
A product owner decides which findings justify action. One person's confusion is not proof that every user will fail. It is a concrete signal that can be tested with additional participants.
After changes, the company runs the same scenario with a different cold-start participant. Reusing the same person would weaken the first-exposure condition.
Measures that matter
- scenario completion;
- time to signup;
- time to first meaningful action;
- high-severity issues;
- evidence completeness;
- findings accepted by the product owner;
- issues missed by automated QA;
- change between test rounds.
Do not publish an invented conversion lift. A credible result reports what was observed and, later, any measured change in real product data.
When this pattern fits
Use this pattern for landing pages, signup, checkout, onboarding, account setup, help flows, and agent-driven interfaces where comprehension matters.
It is not a replacement for accessibility testing, security testing, analytics, representative research, or a larger usability program.
What the company kept on record
The company retained the test build, the exact scenario, the participant starting condition, recording consent, the task reference, the returned evidence, its product decisions, the changes shipped, and the follow-up test.