I have discovered and blocked our DoS attacker. He struck again this morning. I’m still trying to figure out whether I can prevent this for the future. It looks like the attacker would do a normal request, which just looks like ordinary traffic in the logs, then initiate a close with a FIN packet, then block the ACK from my server, leaving the apache process in a CLOSED_WAIT state. When I look at the traffic on the internet, the ack packet going out, and then get responded to with an ICMP packet saying the port is unreachable. I guess what I don’t understand is why retry the ack? It seems my server is trying to be too nice. If he blocks the ack it’s his problem. Call close and be done with him.
11 thoughts on “DoS Attacker Strikes Again”
Comments are closed.
Not an Apache guru, but there is likely a timeout value you can ratchet down in the config so the CLOSE_WAIT half-closed sessions don’t pile up. If not in Apache, then possibly the more generic tcp settings can provide the same effect providing you are running the server yourself (can’t remember if you’re using a hosting service)
Furthermore, depending on what firewall you are using in front of the server, you might be able to intercept this type of attack and deal with it there if your firewall does some level of packet inspection. e.g., the netscreen I’m looking at right now has screen options for protocol anomalies like what you are describing.
I can deal with it by limiting the number of TCP connections allowed from any given client. But I’d prefer not to do it that way.
I have all the apache timeouts at a reasonable level. I’m not sure whether this needs to be a timeout in apache or in the OS. Apache’s timeouts are set pretty reasonably.
Upon further reading, you will probably need to set the timeout value in Linux itself, e.g.
/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_close_wait
Make sure it’s set to 3600 seconds MAX for normal operations, or lower if you keep getting hit with DOS.
Hoping some Linux ninja will correct me if necessary…it’s been a while and I don’t know what flavor you are using. Been a network engineer for a while now and my sysadmin knowledge is a bit dusty.
I found that one too. It’s not in my /proc, so it must have been relocated since that was written.
hit me up in private email if you want to talk firewall config and options…I may not be qualified any more to talk Linux. ;)
CLOSE_WAIT timeout may be as low as 60 seconds on Ubuntu, I am reading, so 3600 may be way high for your needs.
I’ve been using mod_evasive for years. Set it and ferget it. It works perfectly.
http://www.zdziarski.com/blog/?page_id=442
Ever consider a service like CloudFlare.com? It ties in nicely with WordPress, caches static content from your site, and helps soak up attacks like that. It also blocks a ton of spammers. Very handy.
/not affiliated with CloudFlare
It’s not really a big enough deal to justify abandoning the benefits of self-hosting. This is only the first DoS attacker I’ve had, and I have further measures I can take if the problem persists. I may take them as a preemptive measure if I can convince myself there won’t be unintended consequences.
I’ve been building and maintaining Linux servers since 1998. Our main websites had over one million unique visitors per month with no problems using the settings below. Without doing packet sniffing and without knowing your compile options, I’m just ballparking here.
httpd-default.conf:
Timeout 20
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
echo “30” > /proc/sys/net/ipv4/tcp_fin_timeout
echo “3” > /proc/sys/net/ipv4/tcp_synack_retries
If I misunderstood or there’s more involved, try:
http://www.frozentux.net/ipsysctl-tutorial/chunkyhtml/tcpvariables.html