Overview
Stellagent released a survey on July 22, 2026 testing how accurately five major AI services (ChatGPT, Claude, Gemini, Perplexity, and Microsoft Copilot) answer questions about the return and warranty policies of five home appliance retailers (Bic Camera, Yodobashi Camera, Yamada Denki, K's Denki, and EDION). Across 100 question patterns (5 retailers × 5 AIs × 4 scenarios), the accuracy rate was 60.0% while 62.0% of answers included a source problem, showing that accuracy alone cannot measure safety.
For more information, see the press release on PR TIMES.
The accuracy rate across 100 question patterns was 60.0%, while 62.0% of answers included a source problem
Background
Consumers are beginning to use generative AI to research questions before and after a purchase. After buying an appliance, customers often need to confirm details based on official terms or FAQs, such as initial defects, returns, exchanges, extended warranties, point refunds, and on-site repairs. For these questions, accuracy on deadlines, eligibility, refund methods, and warranty coverage matters more than convenience.
This survey is not a criticism of home appliance retailers. It was conducted to verify that even when official information exists, general-purpose AI does not always retrieve it in the right context or summarize it correctly, and to show the need for specialized AI support that answers based on official data.
Key Findings
- The primary judgment across 100 question patterns was 60 correct (60.0%), 25 incorrect (25.0%), and 15 unanswerable (15.0%). Of the 25 incorrect answers, 15 (15.0%) carried a real-harm risk.
- 62 question patterns (62.0%) included a source problem, such as a nonexistent URL, an unofficial URL, or an official URL for the wrong topic. Because source problems are an additional flag spanning the primary judgments, they are not summed with correct, incorrect, or unanswerable.
- By AI, the accuracy rate was 65.0% for ChatGPT and Microsoft Copilot, 60.0% for Claude, and 55.0% for Gemini and Perplexity. The source-problem rate was highest for Microsoft Copilot at 19 of 20 (95.0%), while ChatGPT was 8 of 20 (40.0%).
- By scenario, the years and coverage rate of extended warranties (S2) was lowest at 48.0%, while the number of free air-conditioner warranty repairs (S4) was highest at 80.0%. Initial defects and returns/exchanges (S1) and refunds of points paid (S3) were both 56.0%.
Survey Overview
| Item | Details |
|---|---|
| Survey name | Accuracy survey of major AI answers on home appliance retailers' return and warranty policies |
| Subjects | Five retailers (Bic Camera, Yodobashi Camera, Yamada Denki, K's Denki, EDION) and five AI services (ChatGPT, Claude, Gemini, Perplexity, Microsoft Copilot) |
| Survey date | July 21, 2026 |
| Method | AI answer experiments and desk research of official sites |
| Volume | 100 evaluated question patterns (5 retailers × 5 AIs × 4 scenarios), 268 AI answer logs in total |
| Conducted by | Stellagent Inc. |
Implications for Home Appliance Retailers
Official FAQs and terms need to be structured so that AI is less likely to misread them, not only as pages for human readers. Returns, initial defects, warranties, and point refunds share similar terminology, and normal returns versus initial defects, free versus paid warranties, and used versus granted points are easily confused. Even an answer with a source URL is not necessarily based on the correct page unless it is verified as official, on-topic, and not an outdated rule.
In customer support, businesses need specialized AI connected to official data, FAQs, terms, and product, order, and membership conditions rather than relying on general-purpose AI. Return deadlines, warranty years, coverage, and point refunds in particular are areas where misinformation directly affects customer expectations and inquiry load.
The detailed judgments by AI and scenario, the breakdown of real-harm risks, the classification of source problems, and the methodology are available in the accuracy survey of major AI answers on home appliance retailers' return and warranty policies.
Looking Ahead
Stellagent will continue studying the accuracy and safety of AI answers for retail and e-commerce businesses and helping them design and deploy specialized AI support that connects to official data and guides customers accurately.