Denial of Service (DoS) and Denial of Wallet (DoW) Attacks: Emerging Threats in GenAI Security

The rapid adoption of Generative AI (GenAI) in enterprises, financial services, and critical infrastructure introduces a new attack surface for cybercriminals. Among these, Denial of Service (DoS) attacks and Denial of Wallet (DoW) attacks present serious risks, disrupting operations and compromising financial security as well as brand and reputation risks. Understanding these threats is crucial for organizations integrating AI-powered applications and chatbots.

‍

‍

Denial of Service (DoS) Attacks on GenAI

Among the most immediate and disruptive threats are Denial of Service (DoS) attacks. These tactics are familiar in cybersecurity, but now they’re evolving to exploit the unique characteristics of GenAI systems. 

‍

Let’s take a closer look at how DoS attacks can target and cripple LLM-powered applications.

‍

‍

DoS Attack Techniques Targeting LLMs

LLMs are susceptible to denial-of-service (DoS) attacks that exploit their processing constraints, potentially leading to performance degradation or service outages. According to OWASP Top 10 for LLM, attackers use various tactics to overwhelm these systems, including:

‍

• Automated Task Overloading – Tools like LangChain or AutoGPT generate excessive requests, straining computational resources.
  ‍
• Resource-Intensive Queries – Inputs with complex structures or unusual patterns force the model to consume disproportionate processing power.
  ‍
• Recursive Context Expansion – Crafted inputs trigger continuous context growth, leading to excessive memory and CPU usage.
  ‍
• Variable-Length Input Flooding – Attackers submit numerous inputs designed to reach the model’s processing limits, exploiting inefficiencies.

‍

Beyond resource exhaustion, attackers can manipulate API behaviors to amplify disruption. A Sourcegraph security incident demonstrated how a leaked admin token was used to alter API rate limits, enabling a surge in request volumes that could destabilize services. These risks emphasize the need for robust protections against LLM-targeted DoS attacks.

‍

‍

How Do DoS Attacks Target LLM Powered Chatbots?

DoS attacks aim to overwhelm a system with excessive requests, rendering it unavailable to legitimate users. In the context of GenAI, attackers can exploit these models by:

‍

• API Overload: Flooding AI inference endpoints with high-volume queries, leading to slowdowns or crashes.
  ‍
• Token Consumption Attacks: Exploiting language models by generating excessive, resource-intensive prompts, exhausting computational resources.
  ‍
• Adversarial Prompting: Crafting inputs designed to force an AI system into loops, unnecessary processing, or invalid responses, causing operational inefficiencies.

‍

Another research categorizes DoS attacks as adversarial techniques designed to degrade AI performance by overwhelming the computational resources of the system. These attacks typically target vulnerabilities in model inference processes. Attackers can achieve this by sending high-complexity queries that demand excessive computation, triggering memory exhaustion by exploiting inefficient memory allocation, or forcing the AI into infinite loops by crafting inputs that recursively expand context or overwhelm the model's capacity for processing.

‍

‍

Case Study: DoS attack via OpenAI's ChatGPT API 

Recent research found a security flaw in OpenAI's ChatGPT API allows attackers to launch large-scale distributed denial-of-service (DDoS) attacks by exploiting the API's handling of HTTP POST requests to the /backend-api/attributions endpoint, which lacks limits on the number of hyperlinks that can be included in a single request. 

‍

This research highlighted that the vulnerability enables attackers to overwhelm target websites by causing OpenAI's servers to generate massive volumes of HTTP requests. Elad Schulman, CEO of Lasso Security Inc., emphasized the risks, stating that "ChatGPT crawlers initiated via chatbots pose significant risks to businesses, including damage to reputation, data exploitation and resource depletion through attacks such as DDoS and denial of wallet."

‍

‍

The Impact of DoS on GenAI Applications

A successful DoS attack on GenAI can disrupt business operations, GenAI-powered customer support, fraud detection, and automation systems become unresponsive.

‍

It could also increase costs as cloud-based AI services charge based on usage. For example, attackers can drive up costs by forcing excessive resource consumption. And finally, it can affect real-time decision making as security, healthcare, and financial GenAI systems rely on continuous operation; a DoS attack can delay or prevent critical responses.

‍

‍

Mitigation Strategies: How to Defend Against DoS Attacks on GenAI Models

‍

1. Behavioral Anomaly Detection: Real-time monitoring for unusual behavior, such as sudden traffic spikes or abnormal input patterns, allows for quick identification and mitigation of potential DoS attacks, enabling prompt actions like blocking malicious users.
   ‍
2. Model and API Access Controls: Implementing robust access controls for models and APIs, including authentication and authorization mechanisms, reduces the risk of unauthorized access, protecting resources from overload and manipulation during attacks.
   ‍
3. Rate Limiting & Traffic Filtering: Restricting request volumes and filtering malicious traffic helps prevent abuse by limiting the impact of high-frequency attacks.
   ‍
4. AI-Aware WAFs (Web Application Firewalls): Detecting and blocking adversarial prompt-based attacks in real-time by leveraging AI-driven WAFs ensures protection against malicious queries.
   ‍
5. Adversarial Training: Adversarial training involves exposing models to adversarial inputs during training to enhance robustness, enabling GenAI models to better detect and resist attacks targeting computational resources.
   ‍
6. Efficient Resource Management: Efficient allocation of computational resources helps mitigate DoS attacks by managing memory usage and optimizing inference processes, preventing resource exhaustion from high-cost queries or infinite loops.
   ‍
7. Redundancy and Distributed Architecture: Building redundant systems and implementing a distributed architecture ensures that a DoS attack on one instance does not affect the entire system, maintaining overall availability and performance.

‍

‍

Wallet Attacks: The Financial Risks of GenAI

While DoS attacks threaten availability, another growing concern lies in the financial domain. GenAI systems integrated into payment platforms, trading bots, and digital wallets introduce new vectors for exploitation through wallet attacks. These threats target the integrity and security of financial operations powered by AI.

‍

What Are Wallet Attacks?

Wallet attacks involve exploiting AI systems that manage financial transactions, cryptocurrencies, or digital payments. These attacks can have severe financial and operational consequences, targeting the integrity of automated trading, digital wallet security, and fraud detection mechanisms.

‍

Key Types of Wallet Attacks:
‍

1. AI-Powered Trading Bots: Attackers manipulate AI-driven market predictions to gain financial advantages. According to research from the Wharton School, AI trading algorithms can inadvertently learn to collude, leading to manipulated market conditions and financial losses. 
   ‍
2. Prompt Injection for Fraudulent Transfers: Attackers craft deceptive prompts to trick AI-powered financial assistants into authorizing unauthorized money transfers. This type of attack leverages the AI’s processing of natural language commands to bypass usual authentication protocols.
   ‍
3. Data Poisoning in AI Risk Models: Malicious actors inject misleading data into AI models, compromising fraud detection accuracy. This method can cause AI systems to misclassify fraudulent transactions as legitimate, allowing malicious transfers to proceed. 

‍

Case Study: AI-Powered Cryptocurrency Theft

AI-driven trading and transaction models process vast amounts of financial data. Attackers can manipulate these models by:
‍

• Submitting false data to skew risk assessments.
• Using adversarial AI to predict and exploit trading patterns.
• Deploying malicious GenAI-generated phishing scams to compromise user wallets.

‍

This has become a growing concern, prompting regulatory authorities such as the U.S. Commodity Futures Trading Commission (CFTC) to issue warnings about the proliferation of deceptive schemes involving AI-driven trading bots, and the importance of exercising caution when engaging with platforms that promise guaranteed profits through AI-powered trading. According to the CFTC, some platforms falsely advertise the use of advanced AI algorithms to guarantee substantial returns. In reality, many of these bots are either non-functional or deliberately designed to deceive investors. 

‍

‍

Mitigation Strategies: Beating Wallet Attacks Before They Happen

To protect against wallet attacks, organizations must adopt a multifaceted approach that incorporates robust security practices, cutting-edge technology, and continuous monitoring. Here are some key strategies to consider:

‍

1. Implement AI Guardrails: Establishing AI guardrails involves creating structured guidelines and protocols that govern how AI models operate, especially in high-stakes financial environments. These guardrails ensure that AI algorithms behave predictably and responsibly, minimizing risks associated with unexpected outputs or manipulations.

2. Use Secure AI Model Training: Securing AI models from adversarial manipulation is crucial to maintaining financial security. This involves implementing robust encryption for training data and model parameters, as well as conducting data integrity checks to detect tampering.

3. Conduct Ongoing Security Audits withRed Teaming & Penetration Testing: Regular security audits are vital for identifying vulnerabilities within AI-driven financial systems. Red teaming, in particular, simulates sophisticated attack scenarios to uncover potential weak points before actual breaches occur.

4. Educate Employees and End-Users: Human error often serves as a gateway for wallet attacks. Regular cybersecurity training for employees and educating end-users on best practices can mitigate risks related to phishing, social engineering, and unsecured devices.

5. Deploy Real-Time Anomaly Detection Systems: AI-driven anomaly detection can identify unusual behavior patterns that may indicate wallet attacks. These systems analyze transaction data in real time, flagging suspicious activities for immediate investigation.

‍

‍

When Chatbots Go Rogue: A Peek Inside Amazon’s AI Assistant
‍

Real-world LLM misfires can be costly (and comical)
‍

Inspired by Jay’s post, a viral tweet that exposed a vulnerability in Amazon’s AI assistant, our research team, took a closer look. We explored the chatbot ourselves, running a series of queries to better understand how its guardrails performed under pressure. Here’s what we uncovered.‍

‍

‍

‍

‍

Can Amazon help me build a Molotov Cocktail?

‍We kicked off our investigation with a classic alignment test question, asking the model how to build a Molotov cocktail. To our surprise, the chatbot provided a detailed response without any jailbreak techniques. Building a Molotov cocktail has never been so easy. That’s…not great.

‍

While it's not uncommon to find such recipes from various models after a bit of testing, this instance was particularly alarming. Not only did we get the recipe without any effort, but the chatbot also suggested stores where we could buy the materials (see the white bubbles in the picture below)

‍

‍

‍

It’s very nice to meet you Rufus!‍

In the next step of our investigation we decided to learn more about the assistant itself, its system prompt, and its architecture. That’s when we officially met Rufus (Nice to meet you, Rufus. It’s been a pleasure!).

‍

Side note: Rufus is named in honor of a dog from Amazon’s early days who played a role in the company's history. We have to admit this was a great Easter Egg planted by Amazon. Kudos. 

‍

Now that we're on a first-name basis with Rufus, things got a lot easier. With just a few simple questions, we managed to uncover its system prompt and security instructions.

‍

Interestingly, the questions that previously got blocked were now answered without any issue. This discovery highlights a significant point about the unpredictable nature of Large Language Models and the robustness of their guardrails (or lack thereof). 

‍

‍

‍

‍

‍

‍

What did we learn?

‍

Amazon is an eCommerce giant, using some of the best GenAI models and guardrails products, and this case makes it clear that even the best can struggle with the complexities of generative AI. Read the full research on our blog.

‍

1. The Security Generative AI tools offered are still in their early days

While we're all eager to tap into its potential, the security and operational risks aren't fully understood yet. This technology introduces new risks we've never encountered before due to the models' unpredictability and the amount of data they were trained on. When developing and deploying these applications, it's crucial to work with established frameworks, like OWASP, to ensure that these unique risks are adequately addressed.
‍

2. The architecture is crucial in these early stages of Generative AI technology

Following best practices is essential. In our case, the combination of RAG (retrieval-augmented-generation) and guardrails led to some unexpected behaviors. The architecture not only influences these outcomes but also determines the optimal placement for security mechanisms. Ensuring a robust and well-planned architecture is essential (although not enough) to address the unique challenges and risks of generative AI.‍

‍

3. Most models are still vulnerable to various forms of jailbreaking or manipulation

It is crucial to implement multiple layers of guardrails, beyond just the system prompt, in order to safeguard your application's behavior. The more sensitive the data connected to the model, the higher the risk. Therefore, robust security measures are essential to mitigate these risks and protect your data.

‍

‍

Lassoing GenAI without compromising your security

‍

GenAI adoption is accelerating. But LLM-ready security is lagging behind. That gap leaves organizations exposed to new, rapidly evolving threats, many of which traditional security tools weren’t built to handle.
 

At Lasso, we are committed to leading the charge, helping ambitious companies to make the most of LLM technology, without compromising their security posture in the process.
 

Get in touch to explore how we can help you safely bring GenAI applications into production, confidently and securely.

‍

‍