Gotcha of the Day: Auto Notification When Your Site Is Added To The Safe Browsing Watchlist

A few days ago, one of my clients had quite the shock. She found that when customers searched for their site on Google and attempted to click through, they were taken to a page informing them that they were about to enter a malicious site. Essentially, they were getting this page:

Scary enough for ya?

After poking around, we discovered the cause of the problem, fixed it, and went through the process to get the site cleared. All turned out well in relatively short order.

My client had a good question though: how could her IT team have been notified of this problem *before* someone managed to trip over it by accident?

The first order of business was to understand what this warning was all about. Apparently Google offers a service known as Safe Browsing that browsers can plug in to. Essentially, Google will track a list of suspicious sites, and Chrome and Firefox can use this service to query this list. From what I can tell, one way Google adds your site to this list is via Googlebot. Essentially, if it discovers that it can download malware from your site, onto the list you go.

So, to answer my client's question, all I had to do was write a little script that used the Safe Browsing service. Of course, Google provides a slick little API to access this service, so it turns out to not be that tricky. After a few minutes of shell script programming, here's what I came up with:

#!/bin/bash

##
## Use the google API to check for our entry in their safebrowsing database.
##
## Theory being, we'd like to know ahead of time if the site is marked as problematic
##

if [ -z "$1" ] ; then
  echo "Usage: `basename $0` hostname [notify@email.com ...]"
  exit 2
fi

host="$1" ; shift 
client=sbcheck
apikey="PUT_YOUR_API_KEY_HERE"
appver=1.5.2
pver=3.0
url="http%3a%2f%2f${host}%2f"
api_url="https://sb-ssl.google.com/safebrowsing/api/lookup"

res=`curl -sk "$api_url?client=$client&apikey=$apikey&appver=$appver&pver=$pver&url=$url"`
if [ -n "$res" ] ; then
  mesg="$host marked as suspicious: $res"
  if [ -n "$1" ] ; then
    echo $mesg | mail -s "Safebrowsing Alert for $host" $@
  else
    echo $mesg
  fi
  exit 1
else
  exit 0
fi

I then added the following line to one of my server's crontabs:

 5 * * * * sbcheck.sh mysite.com it@mysite.com

Once an hour, this script runs, which checks to see if mysite.com has been added to the Safe Browsing watchlist. If it has, an e-mail alert is sent out.