Threat Intelligence Blog

Posted November 4, 2009

In late September, Google introduced Google Sidewiki. Sidewiki is, simply put, “a browser sidebar that lets you contribute and read information alongside any web page.” Currently Sidewiki is only fully available in Firefox and Internet Explorer but it is expected that Safari and Google’s own Chrome browsers will be supported in short order.

The reaction in the online community to Google Sidewiki has been mixed. The consumers who are aware of Sidewiki seem indifferent or positive about Sidewiki but the reaction among brand managers, marketers, and some webmasters ranges from apprehensive to hostile. Certain industries like pharmaceuticals are watching especially closely as they hash out what types of legal responsibility they may have to report adverse drug reactions that are published in Sidewiki.

Of course, like any place online, if there is a place to present content, spammers will attempt to take advantage of it. As Danny Sullivan wrote, “not all comments are created equal”, and Google is aware that it must dedicate resources to handling Sidewiki “contributions” that are spam or even more dangerous to end users. Sidewiki, like blogs, forums, Twitter, and other harbors of user-generated content online, could be a viable medium for spreading malware online. The impetus is high for Google to successfully determine what Sidewiki contributions are not dangerous to end users.

Sidewiki spam created to drive traffic to pornography websites. Warning: adult language.

Digging a Little Deeper

In an effort to understand the adoption of Sidewiki, Cyveillance began watching the directory on where Sidewiki entries are being archived for users to view even if they do not have Sidewiki installed on their browser. Beginning on October 13, on a daily basis Cyveillance searched the directory where Sidewiki entries are stored by searching, and noting how many results Google said existed for these Sidewiki entries at the top of the page in the statement, “Results 1-10 of about (number)”.

A couple caveats for the experimentally minded: the queries were not made at the same time every day, and were not always performed from the same geographic origin. However they were done in both Safari and Firefox, while logged in to Google and logged out, to see if these made any difference in the results. The query was also performed from an iPhone for good measure. Here is a screen capture of the results for October 24th.

On October 24th, the directory of Google Sidewiki contributions contained 1,130 entries.

The number of results did occasionally differ depending on the browser used and whether the experimenter was logged in to Google when the query was made. However the differences were negligible and can probably be attributed to the query momentarily being routed to a different Google data center that was just a bit out of sync with others.

number of Google Sidewiki entries
Number of Google Sidewiki entries over time when queried from Firefox while logged in to Google. (No query was made on November 17th.)

A couple of interesting details come from the above chart, which displays results returned when the query was performed in Firefox while logged in to Google.

  • The number of Sidewiki contributions appears to actually have decreased over time. This is surprising as the number would be expected to rise while more users contribute more Sidewiki edits.
  • The directory claims to be empty as of October 31. Since October 31, Google has returned the query saying there is nothing in that directory: “Your search – – did not match any documents. “

Why does it appear Sidewiki usage slowly decreased over time? Perhaps there was an initial rash of spammy or low quality contributions that were being culled from the results as Google tweaked its ranking algorithm for Sidewiki contributions. Still, it is surprising that (at least according to those results) there was a net loss of Sidewiki comments.

More importantly, where did the Sidewiki contributions go on October 31? They were not erased completely or put somewhere else like They still exist in the subfolder, as can be seen in this example, this example, and this example.

Did Google remove results from the subfolder by modifying its robots.txt file? A quick check of Google’s robots.txt file from October 31st has no mention of any sidewiki folder, so it is indeed intriguing why a query of the folder states there is nothing in there. (On October 31 there was a Halloween theme to Google’s robots.txt file but nothing excluding URLs from any /sidewiki folder.)

What Does It All Mean to You?

Luckily for brand owners, the surprising results are not likely to be the result of an intentional effort to make Sidewiki contributions hard to find, but rather a reflection of internal shuffling as Sidewiki is fine-tuned. One example of the tweaking that Sidewiki is undergoing can be found on the Sidewiki leaderboard pages, which currently have an “under construction” notice (you can see their earlier incarnation here). The service is just over one month old, and it is unrealistic to think that the way it is offered at the beginning will be the way it looks even six months after release.

In any case, Cyveillance recommends that enterprises be aware of Sidewiki in these early days and moving forward to monitor closely what visitors are saying about your organization. It is one thing for someone to complain about your organization on their own blog, but it is another thing entirely for that person to be able to write whatever they want on what feels like your actual site. For the proactive types, you can also submit product ideas for Google Sidewiki, for example, where the push to make Sidewiki opt-in for websites (instead of automatically available for Sidewiki comments) seems to be a popular suggestion.

UPDATE November 5: It may be that the reason that Sidewiki entries were not appearing in search results for the query was because they added noindex, nofollow to the meta tags of those pages. However it appears they also added Disallow: /sidewiki/entry/ to their robots.txt file within the last 24 hours as well.

Additional Posts

Google Search Results Significantly Poisoned

Hundreds of Thousands of Links Leading to Malware Found in Google Results Cyveillance has ...

A Dangerous Blend of Phishing Methods

In recent phishing attacks targeting Cyveillance and numerous other organizations, cyber criminals ...