Personal

The Day After the Attack

Last Saturday, April 5th 2014, I received an email from Google that they believed my site was hacked.

I immediately FTP’d into the hosting service and pulled down the ASP. It was VBScript. I haven’t written VBScript since 2002. I quickly jumped on Google and searched for “Rob Mensching”. My heart sank when I saw my site was labeled with the dreaded, “This site may be hacked.”

My mind was racing. This was bad but how bad was it? I wasn’t worried about my vanity search. I was more concerned about how far the “hacked site” stigma would flow. Would Google start to think less of FireGiant because I link to it from here? FireGiant is still a relatively young site and today a fair bit of traffic here flows there.

Now I was mad. Someone didn’t just attack my personal site with my blog of random value. They attacked something I’ve poured my soul into for a little over a year. I was certain I’d never find the attacker but I was going to minimize whatever damage he or she could cause.

Fortunately, the night before, I scheduled the weekend with my wife as a “working weekend”. After watching the //build/ Conference I had an idea I wanted to prove out. This attack changed those plans. Now I was moving my website. Now I was even more mad at this unknown attacker.

Where to Go?

It didn’t actually take me long to decide where I was moving. FireGiant is hosted on Azure. The experience was awesome. So much so that we did the work to move the WiX toolset there as well.

RobMensching.com would be moving to Azure.

Taking Stock of the Situation

Moving the “root” of my site (everything that isn’t this blog) would be trivial. Over a year ago I converted that part of my website to static pages using DocPad. However, my blog was going to be a bigger problem.

I wrote my own blog engine back in 2009 when I joined the Live Mesh team to learn LINQ, AtomPub and web stuff in general. It was a good experience but the code clearly was written by someone just learning those technologies. It was going to be a challenge to move.

On top of the challenge to move the code, I already knew that I wanted to move my blog to a static site as well. I really wasn’t interested in migrating my old blog engine to Azure only to migrate it to static pages sooner or later. Maybe I could take this bad situation and make it somewhat more tolerable by going static now.

I’d already investigated several static blog generators. DocPad could probably do it but was missing a number of key blogging features. Believe it or not, I had discussed the missing features with the maintainer of DocPad a month or so before he posted he was quitting. Since I don’t know JavaScript/CoffeeScript well, DocPad was no longer a good option.

Getting Ruby running on Windows is more trouble that it’s worth so popular platforms, like Jekyll, were out. I really wanted a .NET implementation but none that I found were mature enough.

I knew what I wanted so I decided to go all in and write my own implementation. The result is [tinysite][ts]. This blog post isn’t really about tinysite, so let me get back to the story.

Getting Back to Google

By the end of Monday I had all my old blog posts converted to static pages and tinysite generating URLs that matched my old blog engine. I was about to throw the DNS switch when my boy succumbed to a double eye, double ear infection. All work halted that night to comfort the toddler. Google could wait.

Yesterday morning, I kicked the tires a little more and threw the DNS switch. My uptime monitors detected no downtime. Hopefully, you didn’t either.

Yesterday afternoon, I contacted Google to inform them that the offending file was removed Saturday and that I was now on a new hosting provider from clean sources. Could they please remove the hacked flag? Their web form informed me it could take several weeks for the manual review to be completed.

I sat there for a few moments quietly. Several weeks? Now what? There were still a few features to implement (for example, comments are disabled at the time of this writing) but the emergency that I could address was addressed. Now I was waiting on Google.

This morning, Google emailed me that they had “processed your reconsideration request”. I jumped on Google and searched for my name. To my relief, my site had returned to the top of the search results and most importantly the “This site may be hacked.” label was gone.

Disaster averted.

Closing Thoughts

These days I always feel time is precious. If I’m not working with the rest of the team at FireGiant to serve FireGiant customers, I better be working on something that improves myself or my relationship with my family. So what did I learn here?

Register with Google Webmaster Tools. I use Bing as my search engine of choice but I have the utmost respect for what Google accomplished here. They figured out that other spam sites were linking to a spam page on my site and notified me so I could fix it. Their tools are useful but this part of their service is invaluable. How else would I have known I was hacked?

Cheap web hosting is cheap. That’s obviously a tautology but I never thought it would affect me like this. However, I should have seen it coming. Over the last six months, there were several issues with my old cheap web hosting that indicated something was going awry. This attack was a forcing function to finally get me address the root issue. Move to a real hosting solution.

Understand the Open Source communities you depend on. Again this is probably obvious but if you depend on software built by a community you either need the means to maintain the software yourself or you need to do something to maintain the health of the community. The situation with DocPad drove this point home for me and gave me a case study I can contrast with the WiX toolset and FireGiant. I may do that in a future blog post.

Ultimately, my site and in particular blog is in a better place. This is the end of this story. Hopefully, you found something here useful or interesting or both.

In the meantime, keep coding. You know I am.