Behind

Defending Against DDoS with AWS WAF: Focusing on User Experience

Running a website inevitably means facing abnormal traffic attacks. Especially when aiming for global services, the domain itself becomes more exposed, increasing the likelihood of becoming a target.

To ensure safe website operations, Videostew uses AWS WAF. Before accessing Videostew, you must first pass through the gatekeeper called WAF. After that, Videostew itself employs several mechanisms to verify and filter out incorrect requests.

< Traces of brute force attacks that sneak in as effortlessly as breathing >

Attack patterns are varied, and we defend against various types through numerous settings. In this post, however, we'll focus solely on the indiscriminate DDoS attacks.

DDoS Attack

A DDoS attack is a method of sending multiple requests simultaneously to cause server downtime. The downtime itself is critical, but it can also be exploited for additional attacks through unexpected error messages or settings caused by server overflow.

Several reasons make DDoS attacks challenging to prevent:

At Videostew, we have a two-tier defense setup to block DDoS attacks while minimizing user inconvenience. It's structured with a non-bypassable Stage 1 block with loose conditions, and a bypassable Stage 2 block with sensitive conditions.

WAF's Rate Limit Blocking (Stage 1)

The very first step is to filter out large volumes of traffic at the WAF level. If attack requests reach the actual web server, the burden of verification increases, so it's best to cut off the attack at the WAF.

The issue, as mentioned earlier, is that once a request is blocked at the WAF, it's difficult to handle afterward. Since access itself is blocked at the source, if legitimate customer traffic is mistakenly blocked, it can be somewhat challenging to rectify. Therefore, we set slightly looser criteria (judging that it's truly, truly abnormal traffic).

The number of requests accepted within a specified time is added through [Rules > Add rules > Add my own rules and rule groups].

The important thing is the Rate-limiting criteria, and there is no one-size-fits-all answer. This is because access patterns differ for each service. You can set it tightly and relax it if unintended blocks occur, or approach it the other way around. The time can also be set freely, but if you set it to 5 minutes, you need to increase the Rate Limit accordingly (since legitimate traffic shouldn't be blocked), which means the impact window for a malicious attacker is also extended.

The point here is that the settings are kept very loose. The purpose of this setting is not to block all DDoS attacks, but to initially filter out only the excessively heavy traffic.

Backend & WAF CAPTCHA (Step 2)

This second stage is where malicious attacks and high-traffic users ambiguously mix. A tighter criterion than the first one should be applied (since filtering is still necessary), and at the same time, a means to request removal should be provided since they might be legitimate users.

The setup is simple. Manage a blacklist of IPs using the IP Sets feature on WAF, and instead of blocking the IPs on this blacklist, offer a CAPTCHA feature.

In the backend code, if a user from a specific IP connects excessively within a minute, triggers multiple CSRF errors, repeatedly fails form validations, or continuously requests non-existent pages, the IP is dynamically added to the IP Sets using the WAF SDK.

When this happens, the user encounters Amazon's awkwardly designed CAPTCHA. Until they pass this, the WAF will not allow them to reach the web server. If the user proves they are human, an `aws-waf-token` cookie is created, allowing them access to the web server. As long as this cookie is valid, the user can normally use the service again.

However, the user is still on the blacklist IP. This means that if the allowed token cookie expires or is lost, the CAPTCHA will appear again. Therefore, when a user with an `aws-waf-token` accesses, they are encouraged to "request to unblock the IP" once more through guidance.

< Mistaken for an attack IP but confirmed as human, here's your disassembly guide >

After clicking the guide message, you can verify your humanity once more using Google CAPTCHA. Once verified, an API request will be triggered using the AWS SDK to remove the respective IP.

On the frontend, display the "Manual Disassembly Request" guide message only to users with `aws-waf-token` and without `aws-waf-verified`.

Effect

This is a graph of actual DDoS traffic. DDoS attacks show such spikes because they flood traffic from various IPs at specific times. You can see that a significant portion is filtered out by the set rules. However, some unfiltered traffic still escapes... Tightening the first condition could inconvenience regular users, so enhancing the second condition adjusted in the backend seems necessary.

And More...

Here, we only covered CAPTCHA blocking using WAF's Rate Limit Rule and IP Sets. Our team also uses AWS Managed Rules (blocking methods provided by Amazon). There's no one-size-fits-all in WAF settings, so we believe that customizing it gradually as you apply it to your service is the best approach.