domainsystems.com site down?

What's up with domainsystems.com? Their site has been down all day. Not just me; even Netcraft can't reach them.

Update: Day 2, still down. Has this site been down for awhile? Not one that I've used before, but moniker.com is still pointing to it.

VoilaBot, behave!

My websites are in the local advertising space: employment, online dating, business listings. My audience is in the USA, Canada, and Mexico. Because of that, I have no chance of making revenue from traffic from Europe, China, or Nigeria.

So when I see a spider called voilabot, for Voila.fr, pounding my server farm, that's a spider I'd like to disallow. No problem, just add them to robots.txt. Only thing needed is how the bot identifies itself when scanning that file.

Typically, a spider will identify it's useragent and, parenthetically, give a link to info regarding the bot. Voilabot does not, it just points to their homepage:


2008-01-09 05:00:12 GET /robots.txt - - 193.252.149.16 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+fr;+rv:1.8.1)+
VoilaBot+BETA+1.2+(http://www.voila.com/) - - 200 1100 358 546


which redirects to http://www.voila.fr/, which sucks for me since I don't know French. I am able to find a page on their site about robots.txt, and how to block *all* spiders from visiting my site -- no thanks! I heart Google -- but nothing that mentions what user-agent top specify to block Voilabot.

The Google index, interestingly, includes the robots.txt files they scan. Examining these, some people specify Voila and just as many specify Voilabot. Other Google results include rant like mine -- apparently this bot has been in Beta since 2001.

So Voila.fr Webmaster: please tell us how to block your bot. Thanks!

Update: 1/15/08
No response from Voiala.fr so I'm now denying them at the firewall. If you don't want this bot, I encourage you to do the same.

We've seen it from

81.52.143.15
81.52.143.16

193.252.149.15
193.252.149.16

Selenium and CAPTCHA

Selenium is a great tool for the automated testing of web applications. CAPTCHA is a way to prevent automated tools from accessing your web applications. So the two are mutually exclusive, by design.

Selenium can only be used to test a web app protected by CAPTCHA if a human intervenes for the test. So, if you are testing an 8 page transaction that includes a CAPTCHA on page 2, for example, you can still save tons of time by automating everything else with Selenium but have the test pause for human interaction during the specific section that requires a CAPTCHA response.

This example assumes you are already familiar with how to write a Selenium test, in HTML.

... the first part of your test goes above here...



<tr>
   <td>clickAndWait</td>
   <td>login</td>
   <td></td>
</tr>

<tr><td rowspan="1" colspan="3">human tester must enter CAPTCHA...</td></tr>
<tr>
   <td>waitForPageToLoad</td>
   <td>60000</td>
   <td></td>
</tr>
<tr>
   <td>waitForTitle</td>
   <td>Title of Next Page Here</td>
   <td></td>
</tr>

<tr>
   <td>type</td>
   <td>FieldOnNextPage</td>
   <td>Test</td>
</tr>

... continue on with the rest of your test. That worked for me in TestRunner, at least when the speed was set to slow.

If you are only going to be using TestRunner, as opposed to using a tool like ant to run Selenium tests during a scripted build, you could more easily just insert the following Selenium command into your test, forcing the test to wait until you click the Continue button in TestRunner:

<tr>
    <td>break</td>
    <td></td>
    <td></td>
</tr>

Note: if you are a scammer, screw you. Further note to scammers: the preceding code won't help you in your scam pursuits, because it doesn't defeat CAPTCHA, it just allows a human to do what only humans can do and let the bots do the rest.

Vintacom now under Think Partnership?

Relationship Exchange, the online dating white labeler, is Vintacom Media Group. But now they are under the umbrella of Think Partnership. When did that happen?

Google Analytics can't track Javascript support

AJAX is almost a definitive component of Web 2.0. I tend to be an asshole about security so I like my sites to degrade nicely for those users who have Javascript disabled. I was wondering today if I am a relic.

Digg, my favorite web 2.0 site, is unusable without Javascript. You can login and click around your profile ok, but you can't digg any stories. I'd call that core functionality.

Google maps work ok. Maybe not mashups of Google maps, but the site itself will display in street view and text links for zooming and panning. Nice degradation.

But how many people actually surf around without Javascript enabled? I use Google Analytics to measure the meager traffic I get to this blog, so I checked there first. But since Google Analytics measures traffic solely by Javascript, by definition they can't tell how many people on your siet don't have Javascript enabled. Those users are invisible to Google Analytics.

Looks like Omniture has worked around this issue, I'm checking to see if WebTrends via log analysis provides the data too.

I can't complain about a missing feature in a free service, it's just uncommon for Google to miss a trick.

PS: Blogger requires Javascript, as I discover trying to click the "Publish Post" button and get nothing. Time to enable Javascript again.