Educause Security Discussion mailing list archives
Re: diagnosing possible DOS
From: Randall C Grimshaw <rgrimsha () SYR EDU>
Date: Thu, 5 Jan 2012 20:15:27 +0000
You might look for a non-php way to handle the unresolved page requests. I have apache redirect to a flat.html file on error. Otherwise you run the risk of exhausting system resources. Randall Grimshaw rgrimsha () syr edu<mailto:rgrimsha () syr edu> ________________________________ From: The EDUCAUSE Security Constituent Group Listserv [SECURITY () LISTSERV EDUCAUSE EDU] on behalf of Alexander Kurt Keller [alkeller () SFSU EDU] Sent: Thursday, January 05, 2012 2:56 PM To: SECURITY () LISTSERV EDUCAUSE EDU Subject: [SECURITY] diagnosing possible DOS Hello Folks, Requesting suggestions to diagnose an Apache issue (Ubuntu server 8.04 LTS with Apache 2.2.8 serving custom PHP application with MySQL backend). Server runs normally for a few hours and then Apache locks up, logs entries simply halt. Strace on the Apache processes all look like this: Process 31281 attached - interrupt to quit 0.000000 restart_syscall(<... resuming interrupted call ...>) = 0 0.538531 poll([{fd=15, events=POLLIN|POLLPRI}], 1, 0) = 0 0.000100 gettimeofday({1324400675, 745453}, NULL) = 0 0.000046 gettimeofday({1324400675, 745492}, NULL) = 0 0.000032 gettimeofday({1324400675, 745524}, NULL) = 0 We identified that a website blog function had allowed for significant commercial blog spam to be posted on the site (nonsensical text with lots of links to commercial sites: “Ugg boots clearance”, “Denim jackets cheap”, etc.), those posts have been deleted and the blog mechanism has been secured. Reviewing the Apache logs and Wireshark captures, we see that we have a LOT of traffic trying to get to those unauthorized (and now unresolvable) blog entries. Many of the requesting IPs are reverse proxies and search engine bots who seem to be crawling those spam URLs very aggressively. We have concluded that our site was leveraged for a search engine “optimization” campaign, but now it appears we are suffering from a denial of service condition that may not have been intentional (If we were selling Ugg boots, we would be rich by now). We have some leads on mitigation: blocking aggressive hosts, mod_security, etc., but on a more fundamental level we are hoping to use this opportunity to educate ourselves on what to look for (and how to look for it) when experiencing these sort of events. Any hints on Wireshark log parsing options for diagnosing DOS? Any thoughts on this behavior and the underpinnings of unscrupulous SEO campaigns? I’ll take this opportunity to thank everyone for their contributions to the list in 2011 and offer a toast to an equally productive 2012! Cheers, alex Alex Keller Systems Administrator Academic Technology, San Francisco State University ☛Burk Hall 155 ☎ (415)338-6117 ✉alkeller () sfsu edu<mailto:alkeller () sfsu edu>
Current thread:
- diagnosing possible DOS Alexander Kurt Keller (Jan 05)
- Re: diagnosing possible DOS Randall C Grimshaw (Jan 05)
- Re: diagnosing possible DOS Steven Alexander (Jan 05)
- Re: diagnosing possible DOS Alexander Kurt Keller (Jan 06)
- Re: diagnosing possible DOS Kevin Wilcox (Jan 06)