C1
No explanation
Reviewers see only the generated banner and make the decision themselves.
C1 : No explanation
CHI EA '26 - Published
Yonsei IRB Approved
From Workslop to Work
AI explanations can help reviewers, but they can also become shortcuts. This study shows how explanation design redistributes judgment labor.
CHI EA '26. 10 marketing practitioners, 18 AI-copy reviews, 3 explanation formats.
Participants
10 practitioners
Protocol
3 explanation formats
Stress test
Trap items as stress test
The same explanation helped some reviewers and became a shortcut for others.
0.0%
wrong suggestions accepted when explanation became a safety cue.
0%
wrong suggestions accepted when reviewers kept independent judgment.
same task - same explanations - opposite outcomes
Motivation
When AI drafts look complete, judgment work shifts to the reviewer.
Reviewers still check brand fit, audience, and risk - work that explanations can either support or obscure.
Research question
How do AI explanations reconfigure judgment labor?
The study follows how explanation format changes attention, effort, and responsibility.
Method
A trap-item study across three explanation formats
Findings 1 — Judgment Labor
Four dimensions of judgment labor
Meta-judgment
Reviewers judged the AI's critique while judging the banner itself.
"The AI leaves the actual thinking to the human."
P5 / detailed explanation
Translation
Vague AI concerns had to become concrete design critique.
"Changing the framing would fix it."
P4 / review revision
Coordination
Participants balanced review rules with consumer persuasiveness.
"It follows the rules, but it does not feel attractive."
P8 / rule and appeal
Emotional regulation
Reviewers actively held back comments that could over-shape their evaluation.
"I have to actively hold it back."
P5 / judgment protection
Findings / response patterns
Same explanation, opposite outcomes
Explanations are not trust-calibration tools. They are interface signals that organize judgment.
Implications
From verdicts to scaffolded judgment
Three interface patterns for preserving human judgment while reducing review burden.
DI 1
Risk-aware verification
Add friction only when a claim needs checking before approval.
Request for ReviewClose
Don't miss this winter - book now or you'll regret it
Unlock after checks
Limitations
Exploratory: N = 10, one creative-review domain, short-term lab setting.
What's next
Follow-up study under IRB review with UW and Yonsei collaborators.
Take-aways
Four takeaways for judgment-centered AI interfaces.
Explanations act as interface signals.
They shape attention, caution, and responsibility rather than simply calibrating trust.
Review is skilled judgment labor.
Participants translated AI feedback, checked it against standards, and protected their authority.
The same cue can split outcomes.
A helpful explanation for one reviewer became a shortcut for another.
Better tools structure action.
Interfaces should expose evidence, uncertainty, next steps, and handoff responsibility.