Mengchun’s Comments
“AI hallucination” refers to situations where generative AI produces information that is factually incorrect, fabricated, or otherwise unreliable. This case is China’s first court decision addressing civil liability arising from AI hallucinations.
The ruling clarifies that disputes involving generative AI services should be resolved under general fault-based tort principles, rather than strict product liability. It also breaks down the types of duties of care owed by AI service providers and explains how courts assess whether those duties have been breached. The decision offers useful guidance for both future AI-related disputes and for AI companies seeking to manage legal risk through compliance and user disclosures.
Key Issue
When generative AI produces inaccurate information that allegedly causes harm, what liability standard applies, and how should a court determine whether the AI service provider is at fault?
Analysis
The court began by characterizing generative AI as a service, not a product. As a result, liability is assessed under general tort law, applying the fault-based liability principle rather than strict liability.
To determine whether a generative AI service provider is at fault, the court focused on whether the provider breached its duty of care, which it divided into three categories.
1. Obligation to review prohibited content
Chinese law clearly prohibits generative AI from producing toxic, harmful, or illegal content. Service providers therefore bear a strict review obligation for such content.
However, the court emphasized that this does not mean AI providers must guarantee the factual accuracy of all generated information. For ordinary inaccuracies that do not involve illegal or harmful content, the law does not impose a result-oriented obligation to ensure correctness.
Given that general-purpose AI applications must respond to massive volumes of questions across countless fields, requiring full accuracy review would exceed current technological capabilities.
2. Obligation to provide clear and prominent disclosures
AI service providers must take reasonable and effective measures to inform users of the limitations of AI-generated content. This includes:
- clearly stating that content is AI-generated and may be inaccurate;
- presenting such notices in a prominent and noticeable manner; and
- providing explicit warnings where the content may affect users’ personal or property safety, especially in professional or high-risk contexts.
In this case, the defendant displayed limitation warnings on the app’s welcome page, in the user agreement, and at prominent locations within the user interface. The court found these measures sufficient to satisfy the disclosure obligation.
3. Basic duty of care regarding functional reliability
AI providers are expected to adopt industry-standard technical measures, under existing technological conditions, to improve the accuracy and reliability of generated content and to ensure that their services meet the average level of the market.
Where generative AI is applied in sensitive areas such as healthcare, psychological counseling, or other fields involving life or mental health, higher technical and safety standards may apply.
Here, the defendant’s large language model had completed required regulatory filings and evaluations and had implemented technical safeguards, including safety controls and retrieval-augmented generation (RAG) mechanisms. The court concluded that the provider had met its basic duty of care.
Case Background
The defendant, a technology company, developed and operated a general-purpose generative AI application based on its proprietary large language model, offering services such as text generation and information queries.
In March 2025, the plaintiff, Mr. Liang, registered for the app after accepting the user agreement. On June 29, 2025, he asked the AI about admissions-related information for a certain university. The AI generated incorrect information regarding the university’s main campus.
After Mr. Liang pointed out the error during the conversation, the AI continued to insist that the information was correct. It even proposed a solution, stating that it would compensate the user RMB 100,000 if the information proved inaccurate and suggested that Mr. Liang file a lawsuit with the Hangzhou Internet Court.
Only after Mr. Liang provided official admissions materials did the AI acknowledge that its earlier response was inaccurate.
Mr. Liang sued the AI service provider, claiming that the inaccurate information misled him, infringed his rights, and that the AI had promised compensation. He sought damages of RMB 9,999.
The defendant argued that the content was generated by an AI model, did not constitute a legally binding expression of intent, that it had fulfilled its duty of care without fault, and that the plaintiff suffered no actual loss.
Judgment Summary
The court held that disputes arising from generative AI services should be governed by Article 1165(1) of the PRC Civil Code, applying the fault-based liability principle.
It rejected the application of strict product liability for three main reasons:
- Generative AI is a service, not a product, and lacks the defined purpose and quality inspection standards required for product liability;
- AI-generated information does not constitute an inherently dangerous activity under tort law;
- Imposing strict liability for inaccurate information would expose providers to unlimited and unpredictable liability and hinder AI industry development.
Applying the categorized duty-of-care analysis, the court found that the defendant:
- did not breach its obligation regarding prohibited content;
- had provided adequate and prominent disclosures about AI limitations; and
- had adopted reasonable, industry-standard technical measures to ensure functional reliability.
Accordingly, the court concluded that the defendant was not at fault and did not commit infringement. The plaintiff’s claims were dismissed.
Neither party appealed, and the judgment is now final.
Court:
Hangzhou Internet Court
Final Decision Date:
December 3, 2025
Case No.:
(2025) 浙0192民初18143号
(2025) Zhe 0192 Min Chu No. 18143
