xAI, the company behind Elon Musk’s Grok chatbot, has filed its first lawsuit over allegations that a user used the system to produce child sexual abuse material (CSAM). The complaint targets a South Carolina man arrested earlier this year and frames Grok as a neutral tool whose outputs are the responsibility of users who direct it.
Lawsuit details and the alleged activity
The lawsuit names Terry Wayne Harwood, who was arrested on charges related to possession and distribution of CSAM. According to xAI’s complaint, Harwood used two Grok accounts—identified in the filing by long alphanumeric usernames—to prompt the system repeatedly between December 8 and February 18 to undress or “nudify” non-sexual images of multiple individuals, including a girl who appeared to be about 10 years old.
xAI says some of Harwood’s requests were blocked by Grok’s safeguards, citing an example in which the chatbot refused a prompt that explicitly asked to “remove all her clothing.” The company did not publish examples of prompts that reportedly succeeded, nor did it detail the methods the user allegedly used to circumvent filters. The South Carolina attorney general’s office confirmed the criminal case is pending and described the charges as involving CSAM produced or distributed “through the use of an artificial intelligence platform,” though it did not specifically verify Grok’s role.
xAI’s legal argument: users bear responsibility
Central to xAI’s complaint is the argument that Grok is a neutral generative tool and that all outputs flow from user prompts and directions. The company is pursuing Harwood for breach of contract and damages, and it seeks a judicial recognition that users, not xAI, are liable for illegal content created through Grok.
xAI’s terms of service, emphasized in the complaint, prohibit requests that undress or sexualize real people, depict likenesses of minors in pornographic ways, or otherwise exploit children. The filing asserts Harwood “flagrantly violated” those terms and “went to great lengths to circumvent” Grok’s technological safeguards. xAI is asking the court to enforce an indemnity clause that would shift liability for CSAM and nonconsensual intimate images onto users.
The company framed part of the filing as a preemptive move to reduce potential legal fees and exposure should victims pursue claims against xAI for material generated through the defendant’s account.
Victim claims, class action, and NCMEC findings
The lawsuit arrives amid mounting accusations from victims and advocates that xAI has not done enough to help identify perpetrators. In a separate proposed class action, a young woman alleges her stepfather took his own life after being exposed for creating roughly 7,000 sexualized images of her and allegedly distributing them online. That complaint claims xAI refused to help police identify the user who uploaded her image to Grok.
Lawyers for victims cited a 2026 National Center for Missing & Exploited Children (NCMEC) report saying 90 percent of xAI’s CyberTipline reports were not actionable because xAI did not include user information that could help law enforcement find perpetrators. xAI maintains it reports CSAM it uncovers to NCMEC, but plaintiffs contend the company’s reports often omit identifying data.
What xAI and critics say about outputs and liability
Elon Musk previously posted on X that “anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content,” a warning that xAI reiterates as part of its defense strategy. The company argues that users who craft prompts to produce unlawful images should be treated the same as anyone who uploads illegal material.
Still, the lawsuit raises a broader legal question: can a company avoid liability for harmful outputs by treating its model as a neutral tool and assigning responsibility to users? xAI insists that both inputs and outputs are the content of the user, governed by the service terms they accept at sign-up. The complaint requests damages for harms tied to Harwood’s alleged conduct, including reputational harm and exposure to third-party claims.
Observers point to a potential complication: the U.S. Copyright Office does not consider AI-generated works to be human-created, an institutional view that could affect legal arguments tying users to AI-produced content. Whether courts will accept xAI’s framing—that users are the creators and thus solely liable for illegal outputs—remains unsettled.
As this case proceeds, it is likely to influence both the pending class-action litigation and broader debates about platform responsibility, content moderation, and the legal status of AI outputs. xAI did not immediately respond to requests for comment on the complaint.
Source: Ars Technica AI