As AI applications fully embrace the cloud, emerging components and complex supply chains—while offering convenience—have also led to a sharp rise in risks from configuration flaws and vulnerability exploitation, making the AI security landscape in the cloud increasingly severe. In response to this trend, NSFOCUS conducted analysis of 48 typical global data breach incidents in 2025, with 21 of these incidents directly related to AI. Analysis reveals that these incidents primarily stem from four typical attack vectors:
- Data leaks caused by cloud infrastructure misconfigurations
- Design logic flaws and permission abuse in AI components
- Prompt injection attacks
- Theft of LLM service resources due to stolen cloud credentials
These vulnerabilities and misconfigurations, once exploited, would pose a direct threat to core assets such as model parameters, chat logs, and AI keys.
Security Recommendations for Data Leaks Caused by Cloud Infrastructure Misconfigurations
In 2025, multiple global data breach incidents occurred due to improper tenant configurations, including:
- September 2025: Researchers discovered an Elasticsearch instance linked to VyroAI that was improperly configured with access controls. This exposed 116GB of real-time user logs from three of the company’s AI applications: ImagineArt, Chatly, and ChatbotxAI.
- August 2025: Researchers identified an unprotected and publicly exposed Kafka Broker instance containing vast amounts of personal user data. This broker processed real-time data streams for two AI applications: “Chattee Chat-AI Companion” and “GiMe Chat-AI Companion”. The exposure led to the leak of sensitive information from over 400,000 users, including 43 million chat records, over 600,000 images and videos, IP addresses, device identifiers, and purchase logs.
These cases demonstrate that the breaches were not direct attacks on AI models themselves, but rather exploited configuration oversights in the underlying infrastructure that AI services rely on. Ultimately, this led to the exposure of user data and private conversations. Such incidents underscore a critical insight: AI system security must encompass the entire technology stack and system lifecycle.
To address these vulnerabilities, users of AI components should prioritize minimizing the exposure of third-party dependencies and implementing automated configuration audits for AI assets. Key measures include:
(1) Restrict Public Access by Default All
AI services should have public internet access disabled by default. Access should be limited to internal networks or VPNs. For APIs that must remain publicly accessible, IP whitelisting must be strictly enforced.
(2) Conduct Rigorous Permission Audits
Focus on scanning the permission settings of object storage buckets, Elasticsearch, and vector databases. It is essential to ensure that no databases are left unprotected or exposed on default ports without password authentication.
Security Recommendations for Prompt Injection Attacks
The number of data breaches caused by prompt injection attacks is rising rapidly. Emerging attack methods—such as manipulating AI models into executing malicious instructions through crafted prompts, or even embedding sensitive information in images to evade traditional detection—pose severe threats to data security. Additionally, the continuous evolution of AI technologies, such as multimodality and agent-based systems, while driving innovation, also introduces new risks. In particular, the integration of AI models with third-party applications, though enhancing convenience, can lead to cross-user sensitive data leaks if permissions are improperly configured.
To mitigate these risks, users of AI components should isolate and filter both model inputs and outputs. Key measures include:
1. Filter Input Prompts
Users can physically separate system instructions from user inputs, and explicitly include directives in system prompts to ignore any high-risk requests.
2. Deploy AI Security Guardrails
Users can implement pre-model safety filters to detect and block common injection patterns and suspicious input features.
3. Sanitize Model Outputs
Users can apply regular expression matching and keyword filtering to model outputs to prevent the disclosure of sensitive or malicious content.
Security Recommendations for LLM Service Resource Theft Due to Stolen Cloud Credentials
The root causes of LLM Key Jacking attacks are: attackers using stolen cloud credentials, and victims failing to set up cost alerts for their LLM services. Security recommendations should address both issues.
1. Recommendations for AI Component Users: Implement Key Lifecycle Management
According to the Verizon 2025 Data Breach Investigations Report (DBIR), credential leaks remain a severe issue, with 50% of leaked keys found in public code repositories. Among these, 39% are web application credentials, with 66% being JWT authentication tokens. For cloud keys, Google Cloud API credentials account for the highest proportion at 43%. Alarmingly, the median time to remediate credential leaks is 94 days, creating a prolonged exposure window that allows attackers to easily bypass authentication mechanisms. To mitigate this risk, AI component users should avoid hardcoding API keys in code repositories or environment variables. Instead, use cloud providers’ key management services, such as AWS Secrets Manager or Azure Key Vault, to dynamically retrieve credentials. Additionally, configure automatic key rotation to minimize the validity period of leaked credentials.
Image Excerped from Verizon report
2. Enable Cost Circuit Breaker Protection: Leverage vendor payment models to control costs.
Most vendors offer prepaid and auto-recharge options. For example, DeepSeek and OpenAI provide a prepaid system where users deposit a fixed amount; once depleted, the service stops, limiting financial risk. Users should avoid enabling auto-recharge, as attackers can only drain the prepaid balance. Additionally, vendors typically offer hard quota management for metrics such as RPM (Requests Per Minute) and TPM (Tokens Per Minute). Set TPM limits to match your business needs precisely. This approach can significantly slow down an attacker’s ability to exhaust funds, as they will be constrained by the predefined usage thresholds.
Security Recommendations for AI Component Design Logic Flaws and Permission Abuse
Currently, AI components such as Microsoft Copilot are deeply integrated with internal knowledge bases like SharePoint, Wikis, and code repositories. This integration introduces two major risks: First, AI can transcend document formats, directly reading and understanding vast amounts of unstructured data stored on these platforms. Attackers no longer need to know specific filenames or paths; they can exploit AI’s semantic understanding by asking questions, allowing AI to leverage its indexing permissions over underlying SharePoint/Wiki data to easily uncover and present hidden sensitive information. Second, deeper integration often requires broader AI-related permissions, increasing the risk of data leaks.
To address these risks, AI component users are advised to:
1. Enforce Credential Non-Persistence: Mandate that credentials be stored exclusively in dedicated password management tools. Collaborative documents should only contain reference links, and plaintext credential recording must be strictly prohibited.
2. Implement Sensitive Data Sanitization: Deploy sensitive data discovery tools to conduct comprehensive scans and cleanups of platforms within the AI’s indexing scope, such as SharePoint.
3. Tighten Permissions and Prevent Shadow AI: Restrict the scope of AI agents and enforce isolation. Review default SaaS configurations and globally disable the ability for regular users to create custom agents to prevent the proliferation of uncontrolled shadow AI.
