96MB Down – Again!
It was the first thing I did after getting home from my photography class to check my email since I was away for the whole afternoon (classes itself was three hours and add another 3 hours of commute to it), and I saw this dreaded email from UptimeRobot saying 96MB.com is down.
I was able to SSH into the server although logging in alone took me a minute. I checked SolusVM panel and the VPS was shown as online, only 80MB of memory was used (out of 96MB) and half of the hard drive space was used.
I have rebooted the server and it seems to be up now, I am still trying to read through the access logs and PHP errors to see if it is just because 96MB.com is getting too popular or someone is trying to crash the website.
Having said that though, I think I need to thank Cloudflare to keep 96MB.com up (even though just the text-only version) while the server was barely functioning, and of course, to all my friends who sent me the warnings one way or the other.
Meanwhile, I will download the logs and read them while eating my dinner at the same time, I’ll keep everyone posted.
UPDATE: I think I have found something, someone is running on Joe Dog Siege against 96MB.com at about 5:31 PM EST this afternoon, sending over 200 requests within 1 second and caused the PHP processes to crash (unfortunately with only 96MB of memory, the VPS could not handle that much in one second). IP belongs to WholeSale Internet in Kansas City and unfortunately the domain name attached to it goes to a private whois registration (surprise, surprise!). In any case, the DC has been notified of the matter and hopefully they will be able to deal with the abusers accordingly. Apologies to all my readers and thanks once again for all of you who brought the matter to my attention ASAP!
How much bandwidth do you use per month? We can provide you free CDN via MaxCDN so not only text files can be cached for you when your site is down.
@Rg: Thanks for offering your help!
“I think I have found something, someone is running on Joe Dog Siege against 96MB.com at about 5:31 PM EST this afternoon, sending over 200 requests within 1 second and caused the PHP processes to crash (unfortunately with only 96MB of memory, the VPS could not handle that much in one second).”
Haha! Made me laugh. I guess he wanted to see if you were the real deal or not. 🙂
@William: Sure, if you could offer any help on how I improve my settings so that I can deal with similar situations better in future, please feel free to let me know or comment here. I am not a server set up expert nor I have ever pretend I am one. In fact, the idea of VPS was completely new to me until last November, so if you could teach me a trick or two, I would truly appreciate that.
@William: Thanks for reminding me, I just checked the settings of the PHP file and realized indeed the max_requests settings were missing after I upgraded the PHP there, I have readjusted and now it should be able to handle this better. Thanks for the note.
What exactly did they do to attack 96MB.com?
@lxape: Repeated request sent out so that eventually all the PHP processes gets stuck and used up all the memory in the box…unfortunately 🙁
I don’t know if you have the ram to run it or not, or if you are already doing so but Config Server Firewall aka “CSF” is free iptables based firewall and is fairly easy to setup and run via the command line I run it on serveral of my VPS setups.
@Mike: Thanks for the suggestion. No, CSF is not configured on this VPS, basically I was thinking since 96MB.com is already behind Cloudflare, which has some power of a firewall, I was reluctant to set up more things and use up more RAM, but I can definitely see there is a good point, let me test it to see how the memory usage is, if not too bad I’ll probably add it on 🙂 Thanks for your suggestions!