Secure Bulletin Navigating the cyber sea with knowledge
Home > Articolo > Grok AI Chatbot Tricked Into Leaking Private Chats Through Encrypted Prompt Injection
Grok AI Chatbot Tricked Into Leaking Private Chats Through Encrypted Prompt Injection
Read Time:3 Minute, 32 Second

A newly disclosed attack against xAI’s Grok chatbot shows how easily an AI assistant can be turned into an unwitting spy on its own users. Researchers at Adversa AI found that Grok could be manipulated into leaking private conversation data without the victim ever clicking a link, opening an attachment, or approving anything on screen — a so-called zero-click compromise.

Hiding Commands Inside Encrypted Text

The core of the attack is deceptively clever. Instead of embedding a plain-text instruction that Grok’s safety filters might catch, the attackers wrap their malicious command in AES-256-GCM ciphertext. On its own, encrypted gibberish looks harmless to an input filter scanning for suspicious keywords or known jailbreak phrases. The trick only works because Grok has access to a Python code-execution environment as part of its agentic toolset.

When Grok encounters the encrypted blob, it uses its own runtime to decrypt the payload — effectively decoding the attacker’s instructions for them. Once decrypted, the model treats the output as if it were legitimate internal state rather than untrusted external input, and proceeds to act on it without further scrutiny.

From Decryption to Data Exfiltration

After decrypting the hidden instructions, Grok can be steered into building a URL request that quietly ships user data to a destination the attacker controls. According to the researchers, the data at risk includes the user’s name, approximate location, subscription tier, and portions of chat history. Because the exfiltration is disguised as a normal outbound web request, it can blend into legitimate traffic and evade cursory detection.

What makes the technique notable is the absence of any user interaction. Traditional phishing or malware delivery typically requires a victim to click something. Here, simply having Grok process a page, document, or message containing the encrypted payload is enough to trigger the chain, which is why researchers are labeling it a zero-click attack.

A Symptom of a Broader AI Agent Problem

The Grok case is the latest example of a pattern security teams have been warning about for months: as AI assistants gain the ability to browse the web, run code, and call external tools, the boundary between “instructions from the developer” and “content the model happens to be reading” gets blurry. Attackers have repeatedly shown that hidden text in web pages, documents, or even images can be interpreted by an AI system as commands rather than data.

Encrypted prompt injection raises the bar further by evading keyword- and pattern-based filters that many AI security products still rely on. If a model has any general-purpose ability to decrypt or transform data, that capability itself becomes a potential attack surface.

What Researchers Recommend

Rony Utevsky, the researcher who led the disclosure, outlined a set of mitigations aimed at containing exactly this kind of abuse. The recommendations include:

  • Quarantining content fetched from external sources so it cannot directly invoke privileged tools
  • Requiring explicit user consent before an agent contacts a new destination, with fully resolved arguments shown to the user
  • Keeping detailed per-session traces of what an agent read, decided, and executed
  • Alerting on suspicious sequences such as untrusted content ingestion followed by code execution and unexpected outbound network activity

As the researcher put it, defenders need to “quarantine fetched pages away from privileged tools, require consent for new destinations with fully resolved arguments, keep per-session traces, and alert on the sequence of untrusted content, code execution, and unexpected egress.”

Why This Matters for Organizations

AI chatbots are increasingly embedded in customer support tools, internal knowledge assistants, and coding environments, often with access to sensitive account or session data. A vulnerability that lets an attacker exfiltrate that data without any user action collapses one of the last remaining defenses many organizations rely on — user vigilance.

Security teams evaluating or deploying agentic AI tools should treat any capability that lets a model execute code, decrypt data, or make arbitrary outbound requests as a high-risk surface requiring the same scrutiny given to traditional remote code execution vectors. Until vendors harden these agentic pipelines by default, enterprises are advised to restrict what external content their AI assistants are allowed to autonomously process and to monitor for unusual outbound requests originating from AI services.

Share: Twitter  |  Facebook  |  LinkedIn
Join the discussion

This is a blog in the Fediverse: you can find this article everywhere with @blog@securebulletin.com and every comment/answer will appear here.

If you want to comment on Grok AI Chatbot Tricked Into Leaking Private Chats Through Encrypted Prompt Injection, use the discussion on Forum.

>> forum community

Comments

Leave a Reply