LayeredTech Hosting

For more than a year, I have been hosting my servers with LayeredTech. These guys are professional, efficient and affordable. Now, the following rant about them means nothing about their quality service; rather, this is about the miscommunication that occurred between the company and myself. The story goes something like this.

December 19th: Sometime in the afternoon, I take a look at the server email logs and notice that the log size has reached 8 megabytes in size. Upon examination, I realized sendmail was being used, resulting in an open-relay for spammers. I quickly delete the binary and symlink sendmail to Exim. I noticed the suspicious logging activity stop. This, of course, was an “error” on my part, as I didn’t then look at the Exim mainlog to make sure the suspicious activity had stopped completely.

At about 6 PM, my dad tells me he is receiving spam from the contact form page we had setup for visitors to download our programs from. I quickly shrugged it off and told my dad “I guess they’re just spamming us through the form, whatever.” Big mistake for leaving it at that.

At about 8 PM, my dad is still complaining, and he told me to come check it out in detail this time. I take a look at realize that the contact form was being used for spamming by a spammer. Without thinking twice, I rush downstairs to my computer and completely delete the form. *phew* I think to myself. I then realize there’s over 10,000+ emails queued up in Exim, ready to be delivered to poor folks. Quickly, I read the documentation on Exim on how to manipulate the queue! I come across a command called “exiqgrep” which allows you to go through the queue. Along with the main binary “exim”, I simply remove all the queue emails, whether they were good or bad. Note: I then realized there was a front-end script in Direct Admin control panel that manipulated the queue without needing for the command prompt.

Now that the queue was cleared, I decided it was time to hit the sack. Little did I know, however, that all this time LayeredTech was sending ME emails, warning about a disconnection that was about to take place in an hour or two. Of course, I didn’t get those emails because my server was smart enough to tag them as junk, resulting them being ignored. It also didn’t help that I was sleeping at the time, so no way in hell would I notice anything suspicious.

A few hours later, LayeredTech disconnects my server.

At 7:30 AM today (December 20th) I get a call from my site partner, only to realize myself that the sites and email accounts were not working. I shoot off an email to LayeredTech asking for a quick reboot, unbeknownst that my server was cut off by the Abuse Department because of the spam. I quickly get a reply saying to contact the abuse department. Shit.

Within half an hour, the situation gets resolved when I let them know that it was the contact form being abused that resulted in the spam. Thankfully, the staff at LayeredTech understood the problem and reconnected my server, stating that the support ticket will now close. All is well I thought.

At about 6 PM, after about 10+ hours the issue was closed, I get another call from my site partner. The sites and email accounts are down yet again. What the heck is it now?

I login the ticketing system setup by LayeredTech again and realize the spamming ticket issue is still open. I skim through the ticket to realize a technician had re-disconnected my server because the issue was still open, and he wasn’t sure if I deleted the contact form or not. Jesus, what do you think I did with the form, announce it to everyone about a vulnerability on my server? Frustrated and furious, I shoot off a few emails stating how I’m considering moving my business to another host since they shut my server down without any proper warnings. “At least call me” I tell them. Minutes later, the technician replies back saying he had sent me 3 emails and that was sufficient for a warning. To make it short, he reconnected my server and told me to check the support ticket portal next time.

This whole thing really ticked me off. It was after this incident I noticed that my server was blacklisting LayeredTech support emails, only because the email they sent me contained various spam terms, like “Viagra” and miscellaneous pharmaceutical garbage. Sigh.

Anyway, matter is done and over with. I told the host to contact my cell phone next time they decide to disconnect my server. I still love LayeredTech, of course.