Cathartic for CTO

10LocalCoupons.com

2011-02-13T12:48:00.000-08:00

I realize I haven't blogged since I un-retired. So here's the story...

We sold RegionalHelpWanted.com in February 2008. After a year and a half off, I founded a new company with my previous partners to see if we could duplicate that success. (Anyone can get lucky once, but if we could do it again, maybe it it was more than luck?) We are using the same exact business model: working with local media partners using unsold inventory to advertise a local website on a revenue share basis. Instead of help wanted ads, or personal ads which is at the root of what Cupid.com was/is, this time around we are targeting coupon advertising. So in Nashville you'll hear ads for 10NashvilleCoupons.com on radio and see them on TV, and in Portland OR the website is 10PortlandCoupons.com, but it is all one website serving local content to you based on how you get there. Hopefully, you'll find your local pizza guy on there soon, or a discount on an oil change nearby.

Not only is the product different, but the software stack is a complete switch. RegionalHelpWanted.com we did in ColdFusion, Cupid.com in .net, both on IIS against SQL Server. All on Windows. 10LocalCoupons.com is done in django, an awesome framework for Python, on Apache behind nginx against postgreSQL, all on Ubuntu. I'm really enjoying the new way things are done. Much of the tediousness of writing control panel type stuff -- record insert, updates, deletes -- for customer service and accounting needs is a gimme with django's admin package, allowing my development team to hit the ground running. The django community has been a great resource to us.

So the software landscape is very different, but the hardware difference between old and new is even more dramatic. In our previous endeavors, we were paying about $1500 a month per web server for managed services. Now, using open source software on Amazon's EC2, we pay less than one tenth of that. It's a running joke every month when I announce our EC2 cost. My guess is about half of that comes from dropping Microsoft licensing fees, the rest is from virtualization efficiencies and dropping the human support.

Not all things are different however. This new project has given us the opportunity to hire back several of the awesome people we've worked with in the past. That has made it easy for me to go back to work.

I'll be blogging more soon about what we are up to, and pointing out things I've learned along the way, but for now know that I am having tons of fun.

Bulk COPY a CSV into PostgreSQL, skipping first row

2010-04-14T13:56:00.000-07:00

Looked for a solution to this yesterday and couldn't find it. Asked my Linux guru Jeremy today and he had a easy solution, so this might be helpful to others.

The scenario is you have a big CSV file, and you want to bulk copy it into PostgreSQL, but the first row of the file isn't data, it's got the column names in it. In my case, the text file is 65 Megs so it's not like you can just edit it in a text editor to delete the offending line. (The data happens to be the combined US and Canada zip/postal code database from ZipInfo.com, fyi.)

SQL Server has a bulk insert GUI that lets you specify a start row. Needed that functionality here.

Solution:

Use wc to find out how many rows are in your file:

$ wc ZCUG.TXT
872135  1871133 69105493 ZCUG.TXT

That first number returned, in my case 872135, is the number of rows in the file. Subtract one and and tail that number, outputting to a new file:

tail -872134 ZCUG.TXT > ZCUG-trimmed.txt

Boom! A new file without the row of column names.

owasp-esapi-python configuration

2010-03-30T13:23:00.000-07:00

I tried to send this issue to the esapi-python mailing list (after subscribing) but it doesn't look like that is a functioning list. So any help with the following would be greatly appreciated.

Hi!

Thanks for your work on owasp-esapi-python! I am trying to integrate it into a project and will certainly spread the word to help drum up support for this as I make headway.

I've run into an issue during configuration:

When I do this at the python 2.6 interactive shell, it returns a single line of output...

>>> from esapi.core import ESAPI
>>> ESAPI.encryptor().gen_keys()
Creating new keys in /esapi/keyring/

The documentation leads me to believe that it will also output an Encryptor_MasterSalt but, if it's supposed to do that here, it isn't for me. Let me know any info I can provide. This is on Ubuntu 9.10.

Thanks in advance,
- Steve

Is http://downforeveryoneorjustme.com Down?!? lol

2009-11-29T11:11:00.000-08:00

Ok so I don't think it's down, because the homepage does serve. But when I'm trying to use it I'm getting an error dump:


Traceback (most recent call last):
  File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 507, in __call__
    handler.get(*groups)
  File "/base/data/home/apps/downforeveryoneorjustme/1.337010419190071564/main.py", line 137, in get
    self.render_down(u)
  File "/base/data/home/apps/downforeveryoneorjustme/1.337010419190071564/main.py", line 103, in render_down
    db.put(downer)
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 1212, in put
    keys = datastore.Put(entities)
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 179, in Put
    apiproxy_stub_map.MakeSyncCall('datastore_v3', 'Put', req, resp)
  File "/base/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 72, in MakeSyncCall
    apiproxy.MakeSyncCall(service, call, request, response)
  File "/base/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 266, in MakeSyncCall
    rpc.CheckSuccess()
  File "/base/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 111, in CheckSuccess
    raise self.exception
OverQuotaError: The API call datastore_v3.Put() required more quota than is available.

I love the irony. It's working for sites that are up, like google.com. But I'm working on my sister's web site dev.huppahs.com which is being hosted by my friend, and his site is down at the moment.

Quicken install error 1721 resolution

2009-10-21T09:39:00.000-07:00

After several days of exchanging emails with Quicken support regarding my installation issues, they mailed me an install CD. When that produced the same result, I did use their Online Chat. The website is lousy: when you go to support, and select the product Rental Property Manager, it doesn't offer chat support. But if you choose the product Quicken for Windows, and the version Rental Property Manager, chat is available. It took 20 minutes waiting in the chat queue, but then I had the complete attention of the support person who was able to slove this issue for me. I'll post the transcript here, in case it helps others (Google lonk bait):

Loading...

Raj: Welcome to Quicken chat support. My name is Raj. Please give me a moment while I review the info you provided.
Raj: hI Stephen
Stephen Bywater: hello
Raj: How are you doing today?
Stephen Bywater: fine t
Stephen Bywater: ty
Raj: Good to know that.
Raj: If I understand you correctly, you are unable to install Quicken 2010 RPM, is that so?
Stephen Bywater: correct
Stephen Bywater: These things I have already tried, as suggested by email tech support:
Stephen Bywater: creating a new admin acct, deleting previous version manually
Stephen Bywater: running install as admin
Stephen Bywater: using the qcleanui util
Stephen Bywater: none of these solved the install issue. it quits reporting error 1721
Stephen Bywater: would you like the relevant part of the install log?
Raj: No Stephen, thank you for elaborating on the issue.
Raj: Are you installing Quicken using CD or Download?
Stephen Bywater: i tried the download. Then as a troubleshooting idea, support mailed me the CD. same error either way
Raj: Do you have the CD with you?
Stephen Bywater: yes
Raj: Lets try once more to clean Quicken once again and install using the CD by following some different steps. Do you have QCleanUI application with you?
Stephen Bywater: yes
Raj: Please run it once more and let me know if you are using HP Computer or Dell.
Stephen Bywater: Dell
Stephen Bywater: qcleanui
Stephen Bywater: sorry wrong window
Stephen Bywater: uninstallation completed successfully
Raj: Now, we need to follow some steps to delete the shared folders.
Raj: 1. Please click on Start button >> Computer.
2. On the Organize menu, select Folder and Search Options.
3. On the View tab, choose Show hidden files and folders.
4. Clear the Hide extensions for known file types check box.
5. Select OK.
Stephen Bywater: done
Raj: In the mean time, are you using 64 bit Vista or 32 bit?
Stephen Bywater: 64
Raj: Okay.
Stephen Bywater: i'm ready
Raj: 1. In the C:\Program Data\Intuit folder >>>> right-click the Quicken folder >>>>select Delete, and then confirm.
Stephen Bywater: done
Raj: 2. In the C:\Users\[your username folder]\Application Data\Roaming\Intuit\Quicken folder, select the Data folder, and then copy it to a safe location on your hard drive.

Note: You might not have a Data folder unless Quicken 2007 or Quicken 2008 is your first version of Quicken.

Stephen Bywater: Access is Denied
Raj: Are you login in as administrator ?
Stephen Bywater: to /{user}/Application Data
Stephen Bywater: yes, my acct is in admin
Raj: Please close all windows except this chat window and try to delete that Quicken folder once again.
Stephen Bywater: ok I was able to copy \AppData\Roaming\Intuit\Quicken\Data to new location
Raj: 3. In the C:\Users\[user]\Application Data\Roaming\Intuit folder, right-click the Quicken folder, select Delete, and then confirm.

Stephen Bywater: note I am deleting C:\Users\[user]\AppData\Roaming\Intuit\Quicken...
Raj: Yes.
Stephen Bywater: Application Data is not accessible, but AppData is. ok deleted
Raj: 4. Repeat steps 4 and 5 for each Windows user.
5. In the C:\Program Files folder, right-click the Quicken installation folder, select Delete, and then confirm.
6. Empty the Recycle Bin.
Stephen Bywater: done
Raj: http://code.msdn.microsoft.com/KB961894/Release/ProjectReleases.aspx?ReleaseId=2067
Raj: Please open this link and run ENU (vcredist_x64.exe)
Stephen Bywater: installed it
Raj: Now, please insert the CD and don't auto run it. just browse the CD.
Stephen Bywater: ok
Raj: Do you see 'Disk 1' and 'Custom' folder there?
Stephen Bywater: yes
Raj: Please copy both folders and paste them to the desktop.
Stephen Bywater: cone copying
Stephen Bywater: done copying
Raj: Now, please open that Disk1 folder and double click on the Quicken 2010. MSI to start installation.
Stephen Bywater: ok... just accept all defaults as normal?
Raj: Please add 2010 just after Quicken and the location will be C:\ Program Files\Quicken2010 then continue the installation.
Stephen Bywater: installing to custom location now
Stephen Bywater: woot! installation is asking to check for updates! never got this far yet
Raj: Good news
Stephen Bywater: does install window disappear checking for updates?
Stephen Bywater: or is it all done?
Raj: Did you get any pop-up, like, done, or Use Quicken, like this?
Stephen Bywater: it gave the option to check for updates at the end of install, i clicked that, and window is gone now
Raj: Please wait for 2-3 minutes.
Stephen Bywater: ok brb
Raj: Okay.
Stephen Bywater: bak
Stephen Bywater: no indication that it is still updating
Raj: Okay
Raj: Do you see a Quicken icon on the desktop?
Stephen Bywater: no
Stephen Bywater: there is a Quicken 2010 folder in start menu, but it's empty
Raj: Please go to Start---->>all programs and check if there is any Quicken 2010 folder there,
Raj: Do you see any Quicken folder there?
Stephen Bywater: a Quicken 2010 folder, empty
Raj: Please go to C:\Program files\Quicken2010
Stephen Bywater: its in (x86), ok
Raj: Okay.
Raj: Do you see other files related to Quicken? or it is empty?
Stephen Bywater: Quicken 2010 dir has many files
Raj: Please keep that Quicken 2010 folder open and be on the desktop and open Disk 1 folder.
Stephen Bywater: ready
Raj: Sorry, please close that Disk1 folder and Open the Custom folder, open the AU_AD_Q folder, and then open the SKU folder.
Stephen Bywater: np got it
Raj: Open the folder for your version of Quicken. For example, if you are using Home and Business, open the HaB folder.
Open the Custom folder.
Stephen Bywater: RPM for me
Raj: Yes.
Stephen Bywater: ok there
Raj: Right click the splash.png or splash file and select Copy.
Stephen Bywater: copied
Raj: Now, go to that C:\Program files (x86)\Quicken2010 folder. Right click there and paste. Paste that splash file there.
Stephen Bywater: pasted
Raj: Now, double click on qw.exe file there and check if it opens Quicken.
Stephen Bywater: it says "There is a problem with your Quicken installation... Please uninstall Quicken, and install again."
Raj: Okay.
Raj: Please click on Start-->Control Panel-->Programs and features (Uninstall program)-->Locate Quicken and remove it.
Stephen Bywater: uninstalled
Raj: Now, try to install Quicken once again using autorun from the CD.
Stephen Bywater: installing...
Raj: Keep every setting default.
Stephen Bywater: it dropped icons on desktop this time
Stephen Bywater: install wizard is downloading latest Quicken updates...
Raj: It seems to be working this time, right?
Stephen Bywater: yes, progress
Raj: Lovely !
Stephen Bywater: indeed
Raj: Yes.
Stephen Bywater: Updating Quicken...
Stephen Bywater: Installation Complete! Launching
Raj: Okay.
Stephen Bywater: yeah. its giving me option to convert data to 2101
Stephen Bywater: 2010
Raj: Wonderful !!!!
Raj: You did it.
Stephen Bywater: yeah! Thanks so much for your help! :D
Raj: My pleasure...
Raj: Were we able to resolve the issue(s) that brought you to chat today?

Stephen Bywater: yes, definitely
Raj: It's always a pleasure to help our customers and I am glad I was able to resolve the issue for you.

Quicken 2010: Intuit still making buggy software

2009-10-16T10:15:00.000-07:00

Quicken sent a promo today to current owners of Quicken Rental Property Manager 2010 (which is essentially the same as normal Quicken with a few added features). I have a need/hate relationship with Quicken -- there is no love involved -- that I've blogged about in the past.

Since I've griped about problems with their software, I wanted to give their latest software a fair shake, and of course I had to pay $129 for the privilege. But the installation failed citing an Exit Code 1721. There is a button on the Install Wizard failure notification for "Online Help" but it only opens a browser to the Quicken homepage. Searching the Quicken website for exit code 1721 leaves no results.

Apparently Quicken doesn't play nice with Vista User Access Control. Even as an Administrator I can't uninstall the old version, or now, the new one. Quicken does provide a utility that cleans up the registry, but it doesn't delete the install directory. And, again even with Admin rights, neither can I.

So now can I not install the new version, I can't reinstall the old version, and I'm out $129 bucks.

Intuit doesn't offer live chat support for this product, so I've open up a ticket via email. They promise a response within one business day. Quicken, why do you suck so bad?

Hopefully Intuit Won't Shelve Mint.com

2009-09-14T09:23:00.001-07:00

Intuit, the makers of Quicken, announced they are buying web 2.0 darling Mint.com. I wrote about my concerns for Mint.com last year, and apparently Mint.com did a good job answering those. Congrats to Mint!

My concern now is that Intuit bought them just to shelve the Mint technology and eliminate a promising competitor. I use Quicken for all my finances, and love it for what it does: give me immediate insight into total net worth, track spending by category, and manage my rental properties. Nothing else out there can do that.

But there are some things about Quicken that just plain suck. They are infamous for regularly launching buggy software. For example, I'm running the latest release of Quicken Rental Property Manager on 64 bit Vista Home Premium, and the software never remembers my preferences. And it says I have 3 reminders, with a "View Reminders" button that, when clicked, brings me to a calendar view with no reminders on it.

I just tried to use Intuit's live chat for tech support for assistance with the above, and after forcing me to pick a support category even though my problem didn't fit any of them, it launched an IE window (even though Firefox is my default browser) that did not connect to anything. Support for Quicken is typically done through its user forums, which can be hit or miss.

So although it is the best thing out there for my specific needs, it is still an unsatisfying user experience, and I'd love for something better to come along. Apparently that won't be Mint.com. Hopefully Intuit will incorporate the best of Mint's quality technology into their own, instead of smugly shelving it.

How to Survive Authorize.Net Outages

2009-07-08T07:00:00.000-07:00

I was inspired to write this by the recent heavily reported Authorize.net outage. Credit card processing is one aspect of web site development that you have no choice but to out-source (unless you happen to be a bank). There are several out there, and Authorize.net is one of the biggest. No matter how robust the system is, there are bound to be periodic problems. Over nine years of business, and multiple processors, we saw many of them. Some outages were over in minutes, some lasted hours. Your customers, ready to make a purchase, don't want to hear that your credit card processor is having problems. It is possible to code your website to stay in business during an outage.

At RegionalHelpWanted.com (RHW), we were doing tens of thousands of dollars a day in credit card sales of help wanted advertising, none of which we wanted to lose during an outage by the processor. In the US, our credit card processor was Authorize.net. In Canada, it was Verisign. (We used separate services because we ran the businesses as separate companies.)

On Cupid.com, the problem was similar: we were selling online dating subscriptions; I think our processor was iPay.

In both cases, we were selling electronic services, but the method we used to survive a payment gateway outage with minimum business interruption is applicable to online businesses that are shipping tangible goods.

On your website, your customer should not receive any kind of error message saying that there is a problem with the payment system. The customer does not care about your problems, even if you are not to blame.

When Authorize.net is down, typically your web application will time out when trying to connect to them to make a sale. In a minority of cases, the connection will not time out but will give an invalid response, one that does not fit the normal specification. I don't remember any cases where a processor outage occurred and the processor sent back a valid response with a valid error code like "We're down! Try agin later!" In all of these cases, your web application should behave the same way:

- Capture the order information, and store it for later processing.
- Give the customer a success message
- Use an asynchronous process to retry these transaction until they are completed.

Capture the order information
On RHW, we let users opt-in to storing their credit card information to speed future orders, so we had already done the work necessary to do this safely and securely. On Cupid.com, the whole business was built on reoccurring billing, so same deal there. Your Terms of Use must allow you to always retain order information for enough time to process that order, even if the user does not opt in to longer information retention. While your at it, log any response that you did get from the payment gateway, scrubbed of any security sensitive info like credit card number, as it might help with forensics later.

Give the customer a success message
Error messages are the opposite of user friendly. Tell the customer that you have the order information, that it typically only takes a moment to process and that they will receive an email its done. And then let them be on their way.

On Cupid.com, we'd let the user start using the site with all the privileges of a paying member. If they sent any messages to other members, those messages would be queued until their order was successful.

On RHW, we would post the help wanted advertising. If it later turned out that the credit card was rejected, the ad would be removed. But this also could have been setup so that the ad was not posted until the credit card was accepted.

Process the transactions asynchronously
Queuing is a great way to handle any work that the user should not have to wait around for. You could use it just when there is a connection timeout or for every single transaction.

Cupid.com was a .net application, so it made sense to use Microsoft Message Queqing (MSMQ) for this functionality. The web application writes a message containing the order ID. A queue runner reads the queue, using the order ID to select the order and payment info from the database. It attempts to process the order. If Auth.net times out or returns an unexpected response, the message is sent to back of the queue. It is trivial to build a time delay into the queue runner, otherwise you may find yourself making hundreds of attempts per minute for the same transaction. And it's not nice to kick your credit card processor THAT often when they are down.

RHW was a ColdFusion application, and now it is very easy to do queuing in ColdFusuion too. Our need pre-existed this functionality, so we used MSMQ. We wrote the queue runner in VisualBasic, and the connection to Authorize.net was done as a CF web service.

Once the payment is either accepted or rejected, do make sure you follow up with the customer. On RHW, if a card was declined after waiting in the queue because of a gateway outage, we would call the customer and try to receive alternate payment before we removed their posting.

The queue runner needs to also delete the payment info unless the customer authorized retaining it.

That's it. Once we had that coding in place, Authorize.net outages (or Verisign or iPay) were non-events to the development team. There was some alerting built in so that we'd know if the queue was building up, otherwise we might not even know that Auth.net was down. That is, until the accounting department came over and complained that they couldn't log into the payment gateway.

Dell still doesn't grok Ubuntu

2009-07-01T12:14:00.000-07:00

I'm selecting a laptop for my kids. Mostly because they keep pestering my wife for her laptop. I'd love to get them Ubuntu because I'd love to teach them computer programming, and I like Ubuntu as an environment for that. I also like if for kids as it is less succeptible to spyware, viruses, and malware in general. My kids play Wizard 101 online, and according to one user, that works well under Wine.

So I'm configuring a laptop on Dell, going through their twenty page configuration process. On page one I specify Ubuntu. Eight or so pages later, Dell is asking me if I want Norton Internet Security 2009, Computer Associates Internet Security Plus 2009, and QuickBooks Pro 2009. Hello, none of them run on Ubuntu as installed. I'd love it if Intuit offered Linux software, especially Quicken which I use religously. So unless Dell is going to be including Wine configurations for each of these Windows applications they are selling with Ubuntu, they really are doing a disservice to their customer. An uniformed purchaser is going to assume the apps run on the computer they are purchasing, right out of the box. After purchase, do you think Dell is going to let the customer "return" this software?

Audible.com Web Site is Lousy (To Be Polite)

2009-03-25T07:01:00.000-07:00

My wife was surprised this week when I told her Audible.com has been charging her $14.95 every month for almost a year. She checked into it and realized she had signed up for a monthly subscription without even know it. That's a problem right there. Before closing the account, she wanted to redeem six book credits her account had accumulated. We decided to divide them evenly.

When it was my turn to select books, I was expecting an experience similar to Amazon.com. First of all, the Audible.com web site is very slow. Not snappy like Amazon.com, or any other website I use by choice for that matter. Seconds lagged each time I clicked a link.

Secondly, the category navigation is not very intuitive.

Thirdly, it looks like the site is quite different depending on if you are signed in or not. The site is more appealing when they are trying to lure you into registration, and frankly it's faster too. Tolerably fast. Audible.com offers fewer categories, perhaps to remove clutter and reduce load times.

Once you log in, its the slow site. Clicking on "Science Fiction & Fantasy" now and you can feel the delay before the page loads. You get to a page that displays, among other sub-categories for fantasy etc:

# Sci-Fi: Classic (217)
# Sci-Fi: Contemporary (555)

That is not a great number of titles. It gets more frustrating after clicking around twenty pages or so, the amount it took me to figure out "Contemporary" means anything since 1980. There is no way to browse with multiple filters, for example all "contemporary" sci-fi that was also on the New York Times best sellers list. The closest thing Audible.com offers is all 555 titles, sorted by how often they are sold on Audible.com.

I clicked on that link, got frustrated at how long the page was taking to load and was able to type "I'm composing this" before the page loaded.

Suffice it to say, Audible.com: now that Amazon.com has entered the audio book market place, you web site needs a huge overhaul to be competitive:

1) Speed. The competition is a click away.
2) Breadth of content: I understand sci-fi may be your weak area, but if you don't have what I want for all my audio book needs, I'll go elsewhere.
3) Navigation. Browsing a book store is easy and pleasurable. You're website needs to be even easier. Let me go to sci-fi, and then narrow down to New York Times best sellers. Then let me sort by newest first. By letting me narrow in, and showing me what path I am on, you would let me feel like I am getting closer to the perfect purchase. Otherwise I'm lost in the woods, and leave.

No ESAPI in Python yet?

2009-03-20T16:38:00.000-07:00

I'm considering Python for my next big project. I've been doing ColdFusion for the last 10 years but am liking the karma from open source. So I've been digging into the Python documentation, finished reviewing the Python Standard Library and the Django tutorials.

Next, I wanted to look at some reference implementations in Python as a way of further familiarizing myself with coding best practices. I had stumbled across an implementation of OAuth in Python last week but wasn't ready yet. Specifically, I wanted to look an a reference implementation of ESAPI, the Enterprise Security API from the brilliant folks at OWASP.

Surprisingly, all OWASP offered was the referece implementation in Java, plus some "under construction" pages for .Net and ColdFusion. Elsewhere on the web I found an implementation in pHp, but nothing in Python.

So I know the whole idea of open source depends on the community effort, but I'm not thinking ESAPI can be my first Python app. I'll work on gaining l33t Python skills, but in the mean time it'd be great to see ESAPI in Python. Django community, I nominate you guys!

Google, why do you hate SEO so much?

2009-02-04T06:13:00.000-08:00

See the post on GetClicky blog re:Google's new Ajax-powered search results breaks search keyword tracking for everyone.

In a nutshell, the referrers you get from Google SERPs may no longer have the keyword phrase on the query string. So good luck trying to figure out how your customers are finding you!

Web Developers: Don't Reinvent the Non-secured Wheel

2009-01-27T07:38:00.000-08:00

I'm thinking about beginning another web project. Before one gets coding the fun part of any web application, though, there is tons of core code that needs to we written: login, user management, session management, user registration, logout etc. But writing secure applications can be tricky, and any attempt to roll your own is likely to have security flaws. Open source can solve both of these problems: the code is already written, letting you get onto the fun stuff; and if it's been vetted by a large developer community already you get the security benefit of past mistakes fixed.

One might think that reference implementations would be readily available for these in all web languages, and that we would all be using them by now.

Kudos to OWASP for developing it themselves, in their Enterprise Security API (ESAPI) Project. It details all the functions that a secure application needs. Much more, though, they also offer reference implementations in Java.

I'd love to see the web development community support this project by developing reference implementations in pHp, ColdFusion, and .net. Implementations in each of the popular frameworks would go a long way toward making the web a safer place, and would make the development of every new web application that much easier.

Journalspace.com dies instantly, for lack of a smart CTO

2009-01-03T10:05:00.000-08:00

Hearts out to anyone who blogged on Journalspace.com. The service is dead now, in a heartbeat, because they did not back up their data. Ever, apparently.

Hard to believe a website can remain popular for six years, whilst its IT team merrily whistle through their work day without once stopping to think about data backup.

Maybe I'm myopic, but I've seen this happen with companies started by business and marketing people without a technical stakeholder, albeit no implosion has been so instantaneous. Not everyone can be technically minded, but if you aren't, and you are starting a dot com, better hire someone who is, give them a stake in the company, and listen to them about things like contingency planning.

So, what would you do if your data was lost? This question applies to home users and business people alike. As a CTO, this question should keep you up at night, in many different manifestations:

- What if a HD in the database server goes?
- What if the whole database server blows up?
- What if your web server blows up?
- What if your data center goes off line?
- What if the CEO looses his laptop?
- What if someone hacks into the development environment?

There are hundreds of variations on this theme. Good sleep is for the naive, and the retired, and those that have worked very hard for high availability, disaster recovery, and security.

When does 9 mins = 0 mins? When it's Google Apps SLA

2008-12-04T08:26:00.000-08:00

TechCrunch covered this. In a worst-case-scenario nutshell, Google Apps can be down 90% of the time and be considered 100% up, if it is "up" for at least one minute (or to be pedantic, instant) for every nine minutes of "down."

Commentors at TechCrunch surmise that is not a real-world scenario where a web site can be up for one minute then down for nine for a persistent amount of time.

I wish that were true. I recalled working for iWon.com during the original dot com bubble. Version 1.0 of that site was written against Vignette CMS. The cost for that was rumored to be around $1M per processor (that was the talk around the office) but we had piles of money to burn (and give away).

The application was so unstable that, by the time I left, we had about 24 web servers in the cluster, and each was rebooted every 7 minutes. The fine folks at Vignette gave us that "work around" with a straight face. My brother was still with iWon and helped them move to open-source Tcl.

Since then, I've made sure I don't even hold any mutual funds that have Vignette stock.

Songbird to relase your music from iTunes

2008-12-03T13:37:00.000-08:00

With the release of Songbird 1.0, I get one step closer to moving from Windows to Ubuntu. If Apple won't make a Linux client, then I'd love to drop iTunes. Ars technica does a nice writeup.

Other items anchoring me to Windows, for now:
- Quicken
- PC Games. Steam really needs to make a Linux client. And of course, the games themselves. I'm playing only Enemy Territory: Quake Wars these days, but am close to moving on to something new. Unless WoW or EverQuest drag me back in.

Ask.com Top 10 Search Terms for 2008

2008-12-01T12:24:00.000-08:00

Ask.com announced their top 10 search terms for 2008, and Tech crunch covered it succinctly.

Its not surprising to anyone who has examined user behavior that it is common to use search navigationally. That is, they type in Google the search box of Yahoo and,apparently, Ask.com, to get to google. Ask.com top term "dictionary" should be seen as a navigational search; Ask.com owns dictionary.com.

TechCrunch lauds Ask for being more honest than the other search engines who scrub their lists so much the results are meaningless.

In my opinion, Ask.com and others should cite their methodology. Do they remove terms that are navigational to their competitors? Do they remove porn terms? Its fine and expected that they do, but they should say that.

Raw data is one outcome I hope to see in the future, and why I laud projects like solr that help make open search a reality.

MSNBC.com's double links fatal flaw

2008-11-21T07:48:00.000-08:00

For 10 years now cnn.com has been my primary source of news. But they've been dedicating more and more of their homepage to ads that *look* like articles. The banner ads I never mind. I understand they need to make money. But when they had "stories" on their home page that were links to CareerBuilder ever-green content, I got fed up and left (I was competing against CareerBuilder at the time).

I've been using MSNBC.com since then. Their coverage is as good, sometimes notably better. For example, an article can sit on the tecnology section of CNN's homepage all weekend, because they don't have enough content to keep it fresher than that. On MSNBC the churn rate for technology articles is much greater.

But once you dig into an msnbc article, they use those very annoying double underlines links, that if you mouse over an you navigate around the page popup a distracting ad bubble that is completely irrelevant to the story. The double links don't appear on every article. A quick perusal shows them to be on only a minority of articles at the moment. But it's enough of a distraction to be a fatal flaw. When I'm reading, I want to focus on the content, not traversing a minefield of distraction.

Looks like the newest iteration of the cnn redesign doesn't include CareerBuilder fluff, and the ads are clearly marked, so it looks like I'll be switching back for now.

Primerica Sucks

2008-10-09T06:37:00.000-07:00

In 2000 I setup IRAs for the two children that I had at the time (I still have those two, and an additional three lol). I put token amounts in for them, the minimum possible to open an account. I use a Primerica agent, and never setup regular contributions.

My financial situation has changed a lot since then, and I'm trying to get everything accounted for in Quicken. With the financial upheaval, I'm making sure I have my ducks all in a row.

So today I go to the http://www.shareholder.primerica.com website to manage the account. But it's been eight years, so I need to call their customer service to establish login credentials. 1-800-544-5445. They pick up the phone fast enough; so far so good.

But the CSR quickly determines that she cannot help me, as my name is not on the account. Huh? The account is in the name of my eleven year old. Nice. He's off school today, so I interrupt his Mario Karts and get him on the phone. He correctly identifies himself. He states his date of birth. He gives our previous address, which is the one Primerica still has on record. He correctly gives our old phone number, and gives the phone back to me.

So I ask the CRS "can you help me now? Am I authorized to manage the account?" She explains no, since he is not the only name on the account. "So you want me to get my nine year old daughter and have her go through the same process?" The CSR says as long as she can get through the whole identification process without being prompted by me. Because she could hear me in the background helping my son. She wants me to call the agent who I setup the account with eight years ago. Whom I haven't spoken to since.

There has got to be a better way.

Gizmodo site down

2008-09-23T11:21:00.000-07:00

wtf? Gizmodo site is down? How am I to goof off now?

My sister got hosed by MS Antivirus

2008-09-04T18:40:00.000-07:00

[Wow, I haven't blogged for awhile. Been enjoying retirement, I guess. This pic is now way out of date. My hair got much longer and is now way shorter than my blogger headshot :P.]

First, kind reader, be fully confident: Microsoft does *not* make an antivirus product. There is a malware program going around that calls itself MS Antivirus. I know, because I just installed it for my sister.

Her laptop was hosed, she told me. I thought she meant that the hard drive was shot, but then she continued and described all the problems her computer was having, and it was obviously infected with spyware.

So I took her laptop home, did a low level reformat, and installed Windows XP. Got all the patches loaded on, and of course gave her Firefox.

She sent me info on some antivirus software she had just purchased, including the activation code, and wanted me to make sure I loaded that on since it was bought and paid for. I'm a good brother, so I did.

D'Oh!

Spybot Search and Destroy cleaned the resulting mess up, except for MS Antivirus itself. To kill that, I Ctrl+Alt+Deleted and killed the program, then deleted the install directory.

So, a couple tips:
- Whenever I want to fix someones computer, I always download the utilities I need from download.com. Don't do a Google Search for the stuff you need, since any ads may be compromised. I don't know who the hell makes Spybot Search and Destroy but for years I've gone to download.com to get it; it's always one of the most popular downloads.
- An the new corollary I will pass on to my sister: If you suspect your computer is infected with spyware, don't click on any of the pop-ups said spyware produces looking for a cure.

Props to PC Mag for info on MS Antivirus.

And finally, here are the emails my sister got, in an effort to let any other victims know that they've been had.

From: eSafeBill Transaction 
Date: Wed, Aug 27, 2008 at 5:29 PM
Subject: Your MS Antivirus License purchase
To: my sister

Thank you for making a purchase with eSafeBill!

Transaction information:
Amount: 77.9 USD Including SCHD Bundle chosen
Activation Code: 873465112334272
Transaction ID: 
Order Number: 
Item: MS Antivirus License + System Cleaner and Hardware Doctor Bundle
You have chosen to purchase your software along with System Cleaner and Hardware Doctor bundle offer. Please download the installer for the additional software: http://222.73.37.203/i.php?tks=1
Quantity: 1
Date: 08/27/2008 12:21:40
Download source: http://222.73.37.203/i.php?l=msantivirusxp&c=c

This purchase will appear in your credit card statement as "Spyware-shop4..com".
Total amount of 77.9 USD will be charged to your credit card.
If you are not completely satisfied with this purchase, please do not hesitate
to contact us using SUPPORT REQUEST APPLICATION at http://222.73.37.203/i.php?l=secure.esafebill&c=c
Please do not dispute this charge as doing so may affect your credit rating.

===
ACTIVATION

Please download the software from the following link if you
do not have it already installed.
Download source: http://222.73.37.203/i.php?l=msantivirusxp&c=c

Please activate the program by entering the following
code when prompted.

Make sure you enter your activation code correctly.
Just copy it and paste into the activation code box with no changes.
The code consists of 15 characters.
Your code is: 873465112334272

The product is activated now.
In case of any difficulties,
please do not hesitate to contact us.

===
REFUND POLICY

If you are not completely satisfied with this purchase, please do not hesitate

to contact us using SUPPORT REQUEST APPLICATION at http://222.73.37.203/i.php?l=secure.esafebill&c=c
Please do not dispute this charge as doing so may affect your credit rating.

===
THIS IS AN AUTOMATED MESSAGE. PLEASE DO NOT REPLY.

Ok, lets note that dead giveaway too: if it says "Don't contact your credit card company to dispute the charge, it will affect your credit rating" then please call Visa or Mastercard right now and do just that. Their operators are standing by. :)

Google AdPlanner

2008-07-02T04:25:00.000-07:00

Why buy a search term and link it to a page that is unreachable? No prob, if you a) have more money than Yaweh and b) you are the guy from whom we buy search terms.

A search for AdPlanner, Google's new publisher tool, does just that. Organic results include links to Toyota affiliate terms of use etc, which means this blog just might be a linkbait success.

I'll dig more into AdPlanner once I find a link to official public info, but Valleyway is already blogging about what it means to Comscore and Nielsen -- ok, they say its no threat. But maybe they were just being ironic. Oh, I can hope.

LinkedIn valuation leads to site downtime?

2008-06-18T08:31:00.000-07:00

LinkedIn is really lagging this morning. I've never experienced lag on LinkedIn before, so maybe its the news of their 1 billion valuation. Maybe LinkedIn will now be hip and trendy, and downish, just like Twitter!

Downtime killed Friendster when it's valuation was through the roof.

So LinkedIn, take some of that new money and go shopping.

Google Street View for Real-estate Investors

2008-05-26T11:41:00.000-07:00

A real-estate agent who I trust a lot happens to be in North Carolina a lot this year, and is on the look out for investment properties for me. She found a great single family home in Hillsborough that came on the market that morning, and it sounded like it was priced right. I went to Google maps, hoping the satellite view would give me some sense of the neighborhood, since I'd be making an offer site unseen. Not only did I find the satellite view, but also a Google Street View of the front on the house. This is a very residential area, so I was quite surprised. Within a few clicks I had checked out a dozen houses on the street and felt very comfortable with what I saw. We made an offer and it was accepted.

Google Street View will certainly become such a useful tool for us we will take it for granted. Until then, it'll be amazing to see how it changes the way we live and do business.

URL spoofing

2008-04-21T19:56:00.000-07:00

I love it when web sites allow you to enter anything you want on the url, because silly people like me are easily amused. Here's fark.com's example.

Too high brow? Here's my first mod.

Odysseus in Firefox

2008-04-14T12:29:00.000-07:00

I've been blogging here for 361 days. :D My first blog was about using Odysseus as a proxy server with IE for testing web pages. I shared that blog with some colleagues recently, and the balked because, you know, IE sucks. But you can use this great tool with Firefox or any HTTP tool, also.

First, download Odysseus, and run it so you can see the little Roman helmet in your system tray (Windows users). Its got a red ... feathery-decoration thingy on top. To turn that green, and thereby turn on this proxy server, right click on it and select "Interceptor."

To configure Firefox to temporarily use Odyssesus as a proxy server so that you can see, and edit, outgoing GET and POST requests, and to view the REPLY from the web server with all the headers, in Firefox choose Tools > Options... Advanced, Network, "Settings" button, select "Manual proxy configuration:" and enter "localhost" as the server and "50000" as the port. Confirm the dialog boxes.

Now, when you load a web page in Firefox, an Odysseus window will pop up letting you see (and edit) the HTTP data. Test away!

When you are all done, go back to your browser and turn the connection option back to normal.

Wikia Search dead?

2008-03-26T10:48:00.000-07:00

search.wikia.com was released January 7, 2008 as an open source search engine enhanced by user submitted content. But looking at http://search.wikia.com/wiki/Search_Wikia, the wiki about the project, there has been no news in over a month. And the forums look dead too. This could be a really interesting project, but seems like the open source community hasn't embraced it yet. Maybe we're tired of Jimmy? Maybe he's tainted?

If anyone is interested in working on the concept of open search, I recommend Lucene which is the engine Wikia Search is based on, or solr which is an enhacement of lucene. I've done a solr proof of concept in the past several months, and I love what solr can do. I think I'll be spending time their. Google can't be the final solution to search.

chili's hates vegatarians

2008-03-19T18:48:00.000-07:00

My wife and I ate at the new chili's here in town. You expect the service to suck at a new restaurant, but they canned our waitress mid-meal and her replacement was great. No worries there.

But I could not find a single vegetarian entree on the menu. I'm not a vegetarian, but I like to eat healthy, and try to be environmentally friendly. I've learned you can conserve more water than forgoing 1 lb of beef than if you forgo showering for an entire year. And I'm trying to loose weight as my New Years resolution. Plus I love veggies, and that's what I was craving tonight.

The sucky waitress didn't know what they offered for vegetarian choices, but after discussing with the manager said there were some things they did. So I asked for the manager and confirmed that, in fact, there were no vegetarian entrees on the menu. But she was a vegetarian herself and suggested a couple of things she liked. So I took her recommendation and got a fajita with lots of veggies and a bean pattie. It was fine, and I enjoyed it. The manager was awesome, and I made it a point to tell her our new waitress was great, the other one was lousy, and thats when she told us that the other one was "no mas."

We won't go back there though. My wifes peach margarita was way too sweet she said, and the menu choices were just not good. There was nothing healthy.

So chili's.... why not a single vegetarian entree on the whole menu?

Compete Aquired

2008-03-05T11:19:00.000-08:00

Mashable reports Compete Acquired by TNS for $75 Million. Compete offers free public web metrics, like Quantcast. Unlike Alexa, in that it doesn't suck.

I predict that in five years either Compete and Quantcast will be purchased by Google, and the new company will *replace* Nielsen/Net Ratings and comScore. Information wants to be free, and the industry desperately needs a trusted third party to step in so we can all start comparing traffic using the same measuring stick. Mergers and acquisitions rely on this data, and beyond the top 10 sites, Nielsen and comScore methodologies can't see through the data storm.

The survivor will combine data that works by getting anonymized data from a significant portion of ISPs, like HitWise does, and data gathered from javascript that can be copied and pasted onto your website -- does this part sound like Google Analytics to anyone? Google also has the peering relationship with ISPs to get the first part done. And they have the "preventing fraud" chops to get it done right.

Nielsen will not be trusted any more then it currently is (not a lot) and it's Golden Age has past. comScore hasn't shown it can do a better job. Long live {insert successor here}!

iTunes: why my wife must get Vista instead of Ubuntu on her new laptop

2008-02-15T14:05:00.000-08:00

Being a geek, I buy gadgets. For Valentine's day, I bought my wife an iPod. Yeah, yeah, dinner too etc.

For our imminent anniversary, I'll be getting her a laptop. She wants mobility just around the house, and her desktop will be taken over by the kids.

I've been eyeing OLPC's offering, thinking that's what I could get for our 10 year old too and benefit from the networking things that device does so nicely. But since they are not offering the "Buy One, Give One" anymore, that options is gone.

I was thinking about the ASUS EEE PC too, since it's received good reviews and its ultra small. Her fingers are smaller than mine.

But the deal breaker is iTunes. She *must* have iTunes -- it's the thing she's really wanted. Not because it's the best, or cheapest. But it's what she knows, what her friends use, it's easy. Any laptop that doesn't support iTunes will disappoint her. And, fact is, she doesn't need a laptop, the pleasure she will get from having one and using it with her new iPod is the whole point. Everything. Not a "kinda cool, with concessions cause my husband is a geek, and thats really cool, but it comes with caveats*" outcome, I'm looking for here.

Yeah, I know, geek-readers, yo've been using your iPod on Debian or whatever for 2 years without a hitch, or your alternate-MP3-player-that-stores-2x-more-and-cost-.5-less on CentOS, but please see above. It's my wife, dude.

So, unless Apple comes out with an iTunes client for Ubuntu in the next couple days, I'll be paying Microsoft for an OS I'd rather do without. Yuck.

onTargetjobs Announces Acquisition of RegionalHelpWanted.com

2008-02-09T12:02:00.000-08:00

Here's the press release.

Google for Error Messages

2008-02-07T11:22:00.000-08:00

I'm a total geek -- if you are, you may, like me, find this page completely hillarious: http://extension.usu.edu/?q=error

I stumbled upon that while exploring the problem of displaying error messages. To often people forget to turn of debug output in their production environments, revealing *way* too much to the public about their innards.

So, take the default error message for the code of your choice, and Google it. Find a lot of sites at their worst, if Googlebot happened to be spidering them then.

Google hearts Cupid

2008-02-05T08:00:00.000-08:00

In the past I've blogged about Cupid.com running afoul of Google's mystical algorithm. That's a thing of the past, now it's the first search result for the term "cupid."

Our business model of advertising on local radio means we don't have to play the PPC game or get crazy worried about our Google search results, as long as we do well for branded search. For example, if you search for "online dating", the competition to get on that page is fierce/expensive, and we just haven't engaged in that pursuit.

Today I noticed, however, if you search for "online dating" or even "dating," Google's new "Searches related to" section includes "cupid.com" or "cupid." Very cool. Nice to be finally getting our props, just in time for Valentine's day.

Google, Cupid.com hearts you too.

Web Analytics software compared

2008-01-29T13:54:00.000-08:00

One of my favorite quotes, I don't know who said it originally but I got it from a high school math teacher "If you change the questions, the answers will change accordingly."

Public kudos to Stone Temple for their 2007 Web Analytics Shootout, a head-to-head comparison of how several different web analytic packages running concurrently on the same sites reported different outcomes. A must-read for anyone who produces or consumes data like site page views, unique visitors, and conversion rates.

Too bad WebTrends and Omniture were not included in the broader study, each one was only used for one site in the study. It'd be great to see if the findings were true on more heavily trafficked sites.

Mint.com resuccitates a failed concept

2008-01-27T06:01:00.001-08:00

Mint has been getting a lot of good press, kudos to them for that. But the idea is not new. It was tried during Web 1.0 by OnMoney.com, a division of Ameritrade. I worked there for a couple months.

OnMoney tried to make a business as a website where you got manage all your financial data: banking, insurance, investments. Total net worth, with articles from experts advising of all of this.

After I left, I heard they got the results of an $80k usability study: "You've got some great tools that *no one* knows how to use."

OnMoney went on to blow a big wad on a Super Bowl commercial. Not too long after, they closed shop.

Hurdles Mint will need to overcome:

- consumers don't know they need these tools. They aren't fun and sexy. You going to give me a facebook app that lets my peeps know "Steve has maintained a positive balance for 6 weeks in a row!" People -- not the Ivy school grads that fund and launch startups like this, but real live real-world people -- don't talk about finances to their friends. So how go you get a community, how do you spread the word?

- all my financial data in one place? No way I'm going to trust anyone with that. Mint needs to prove to the world that they have a 100% bullet proof infrastructure that the NSA and hackers all over the world agree is insurmountably robust. New code just can't do that. So I'll wait 10 years and if they are still around, then I'll assume their code is good. Can they wait that long?

They could put all their code open-source to address the security concerns, but I'm not sure their business model can support that.

Huppahs.com launched

2008-01-26T13:52:00.000-08:00

My sister Maria and I launched her web site this week (on her birthday, in fact). It's very basic php and for now the only method of payment is thru PayPal -- kudos to Maria for that idea and for coding hooks for their cart.

The final version will be fully LAMP. My friend/employee Jeremy Price is hosting for us on his Linux box. Apache (duh), pHp 5 and mySQL on its way, after we get the credit card processor account set up.

So check out Huppahs.com, even if you not currently planning a Jewish wedding.

Eclipse 3.3.1 in Ubuntu

2008-01-25T18:20:00.000-08:00

If your Eclipse install under Ubuntu just took a crap like mine did due to a bad update (shhhrrrr, it got rolled back), you may be looking to upgrade to the latest and greatest version of Eclipse, and don't want to wait until it finally gets added to the official repository.

Many thanks to Ivar Abrahamsen for writing How to install Eclipse in Ubuntu. He posted it to the Ubuntu wiki as Eclipse Web Tools so find an up-to-date version there.

domainsystems.com site down?

2008-01-21T13:23:00.001-08:00

What's up with domainsystems.com? Their site has been down all day. Not just me; even Netcraft can't reach them.

Update: Day 2, still down. Has this site been down for awhile? Not one that I've used before, but moniker.com is still pointing to it.

VoilaBot, behave!

2008-01-11T06:25:00.000-08:00

My websites are in the local advertising space: employment, online dating, business listings. My audience is in the USA, Canada, and Mexico. Because of that, I have no chance of making revenue from traffic from Europe, China, or Nigeria.

So when I see a spider called voilabot, for Voila.fr, pounding my server farm, that's a spider I'd like to disallow. No problem, just add them to robots.txt. Only thing needed is how the bot identifies itself when scanning that file.

Typically, a spider will identify it's useragent and, parenthetically, give a link to info regarding the bot. Voilabot does not, it just points to their homepage:


2008-01-09 05:00:12 GET /robots.txt - - 193.252.149.16 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+fr;+rv:1.8.1)+
VoilaBot+BETA+1.2+(http://www.voila.com/) - - 200 1100 358 546

which redirects to http://www.voila.fr/, which sucks for me since I don't know French. I am able to find a page on their site about robots.txt, and how to block *all* spiders from visiting my site -- no thanks! I heart Google -- but nothing that mentions what user-agent top specify to block Voilabot.

The Google index, interestingly, includes the robots.txt files they scan. Examining these, some people specify Voila and just as many specify Voilabot. Other Google results include rant like mine -- apparently this bot has been in Beta since 2001.

So Voila.fr Webmaster: please tell us how to block your bot. Thanks!

Update: 1/15/08
No response from Voiala.fr so I'm now denying them at the firewall. If you don't want this bot, I encourage you to do the same.

We've seen it from

81.52.143.15
81.52.143.16

193.252.149.15
193.252.149.16

Selenium and CAPTCHA

2008-01-04T11:34:00.001-08:00

Selenium is a great tool for the automated testing of web applications. CAPTCHA is a way to prevent automated tools from accessing your web applications. So the two are mutually exclusive, by design.

Selenium can only be used to test a web app protected by CAPTCHA if a human intervenes for the test. So, if you are testing an 8 page transaction that includes a CAPTCHA on page 2, for example, you can still save tons of time by automating everything else with Selenium but have the test pause for human interaction during the specific section that requires a CAPTCHA response.

This example assumes you are already familiar with how to write a Selenium test, in HTML.

... the first part of your test goes above here...



<tr>
   <td>clickAndWait</td>
   <td>login</td>
   <td></td>
</tr>

<tr><td rowspan="1" colspan="3">human tester must enter CAPTCHA...</td></tr>
<tr>
   <td>waitForPageToLoad</td>
   <td>60000</td>
   <td></td>
</tr>
<tr>
   <td>waitForTitle</td>
   <td>Title of Next Page Here</td>
   <td></td>
</tr>

<tr>
   <td>type</td>
   <td>FieldOnNextPage</td>
   <td>Test</td>
</tr>

... continue on with the rest of your test. That worked for me in TestRunner, at least when the speed was set to slow.

If you are only going to be using TestRunner, as opposed to using a tool like ant to run Selenium tests during a scripted build, you could more easily just insert the following Selenium command into your test, forcing the test to wait until you click the Continue button in TestRunner:



<tr>
    <td>break</td>
    <td></td>
    <td></td>
</tr>

Note: if you are a scammer, screw you. Further note to scammers: the preceding code won't help you in your scam pursuits, because it doesn't defeat CAPTCHA, it just allows a human to do what only humans can do and let the bots do the rest.

Vintacom now under Think Partnership?

2008-01-02T09:45:00.000-08:00

Relationship Exchange, the online dating white labeler, is Vintacom Media Group. But now they are under the umbrella of Think Partnership. When did that happen?

Google Analytics can't track Javascript support

2007-12-12T10:31:00.000-08:00

AJAX is almost a definitive component of Web 2.0. I tend to be an asshole about security so I like my sites to degrade nicely for those users who have Javascript disabled. I was wondering today if I am a relic.

Digg, my favorite web 2.0 site, is unusable without Javascript. You can login and click around your profile ok, but you can't digg any stories. I'd call that core functionality.

Google maps work ok. Maybe not mashups of Google maps, but the site itself will display in street view and text links for zooming and panning. Nice degradation.

But how many people actually surf around without Javascript enabled? I use Google Analytics to measure the meager traffic I get to this blog, so I checked there first. But since Google Analytics measures traffic solely by Javascript, by definition they can't tell how many people on your siet don't have Javascript enabled. Those users are invisible to Google Analytics.

Looks like Omniture has worked around this issue, I'm checking to see if WebTrends via log analysis provides the data too.

I can't complain about a missing feature in a free service, it's just uncommon for Google to miss a trick.

PS: Blogger requires Javascript, as I discover trying to click the "Publish Post" button and get nothing. Time to enable Javascript again.

The Aesthetics of Hex Rounding

2007-11-20T19:35:00.000-08:00

~~ In which I coin a phrase, "hex rounding," as applied to color selection for the web ~~

A web project underway is for a Floridian audience, and needs to have a local feel. In choosing a color palette, the obvious first choice is "orange." But you can't use the Orange and Blue of the University of Florida fans w/ offending Seminole sensibilities. Similar issue for Orange Green of University of Miami. Can't use Miami Dolphins colors either. Note: I lived in Orlando for seven years, and graduated from UCF.

I was inspired by rainbow sherbet: the orange and green from that, plus a very light yellow, seemed Floridian to me. Please don't argue that with me, blogoshpere; I know its subjective, it's not the point of this blog, just the setup.

Working with a very talented web designer, we google-imaged "rainbow sherbet" and got enough results to color sample a few. Dropping them into the page layout and tweaking for readability netted the following palette:

orange #FB9E03 (251,158,3)
green #B7DF40 (183,223,64)
yellow #FEFF95 (254,255,149)

I have no training in color theory, but I grok the basics of color families. Color families can be colors that have the same hue, in different saturations. Or different tints. Whatever, you change some values but keep others the same and you'll get colors that "go well together." In computers, these colors are expressed in numbers that share similarities. In hex, base 8. Math aesthetics, you see?

And I remember my days of teaching "web design" back in ~1996, before CSS, when we had to worry about "web safe colors". Web safe colors were in the 256 color "Windows palette" of 8 bit graphics. Web safe octets were 00,33,66,99,CC, and FF. In decimal, that is 0, 51, 102, 153, 204, and 255. So the first round of my "hex rounding" netted:

orange #FF9900 (255,153,0)
green #B7DF00 (183,223,51)
yellow #FFFF99 (255,255,153)

So my orange and yellow were now "web safe", and the last octet of my green was "safe". But I'm not really caring about web safe these days. I'm just using it as the most convenient direction to push them, mathematically, into the same color family.

I like working in http://www.colourlovers.com for web palettes, only because it is easy to share successful results with remote people. And this is color picking, not just math. But COLOURlovers doesnt let you input RGB, only hex. I imagine they think people are mostly clicking and dragging around the color picker.

I continue to work on the green. 183 is between my 153 and 204. I test pushing it all the way to 204, and the color works for me. Somewhat lighter. RGB = colors of light, so bigger numbers = "more lights on" and (no pun here, thats where the word comes from:) lighter color.

223 I push to 255 but now my color is yucky bright lime. Lets add blue: from hex 00 go to 33, still yuck, 66, 99 oops too far. I'm now at pistachio ice cream and I'm still aiming for lime sherbet so lets split the difference. Translating to decimal, I need a number between 102 and 153. 127.5 splits the difference. 7F. Check it, yes that works.

orange #FF9900 (255, 153, 0)
green #CCFF7F (204, 255, 127)
yellow #FFFF99 (255, 255, 153)

And here is my palette, hex rounded. Not necessarily better, just mathematically based. Sorta:

Rackspace SLA = 99.9%. Penalties kick in.

2007-11-14T06:56:00.000-08:00

Rackspace had a serious outage. Valleywag scooped the actual police report.

I think most Diggers are being too kind to Rackspace. I just checked with my hosting provider about this scenario:

"There are two Central Hudson utility transformers that feed our building. They are located behind locked fences at the Central Hudson sub station on the perimeter of our campus --- far from any roads.

Additionally, there are receiving and distribution transformers within our building on both the second floor of our mechanical plant and our data center. Again, these devices are not located near roads or traffic.

Lastly, our emergency generators are located in protective, sound enunciator houses to protect them from the environment / external elements. These generators are also not located anywhere near traffic flow."

I host with Cervalis btw. I ckecked with Rackspace's live chat attendant this AM. Their SLA is for 99.9% uptime, so penalties kicked in for affected customers.

Their penalities are a percentage off monthly fees equal to the portion of the montly outage. So if your company was down for .5% of the month, about 3.5 hours, you'd "save" .5% of your Rackspace fees. How crazy would you be if your business was SOL for 3.5 hours?? This SLA is lousy imho.

Loving Ubuntu 7.10

2007-11-07T17:25:00.000-08:00

When my sister asked me to help her with a web site in April, I took the opportunity to expand my professional skills by developing it in LAMP: Linux, Apache, MySQL, and pHp. I'd dabbled with Cygwin when I knew shell scripting was best suited to the task at hand, but that was the extent of my open source skills.

So I took a laptop that was past its prime -- the network card only worked intermittently, and the graphics capability meager -- and installed Ubuntu Feisty on it. It's worked pretty well, and I've been able to muddle through the learning curve to get the project underway.

I'm an update-aholic, so when Ubuntu 7.10 "Gutsy Gibbon" was released, I was edgy and feisty (lol) to get it. There were several kinks: first attempt failed because duh! I ran out of battery power. Second attempt: my network card went on vacation again. Sigh.

I tried the Live CD, then got back on track with the Alternate version. Luckily plenty of other computers around to allow for sneaker-netting.

The Apache upgrade was another hiccup: I had to re-configure pHp and chmod permissions.

I brought it into work and one of my IT guys wrestled with the wireless NIC to get that on our office network. I never could get that done under Feisty. But now I can easily roam, connecting to my home or office network without intervention whenever I'm in range.

But I confess: I love Ubuntu 7.10 for the eye candy. Transparent terminal. Clearlooks, Darklooks depending on mood. Transparent panels top and bottom. I'm a sucker for transparency, it seems.

Jobster gets blocked

2007-10-22T14:07:00.001-07:00

We're having performance issues on RHW lately, and one finding is that Jobster is scraping our jobs. No surprise there. They do attribute us, but I never liked their model of just scraping instead of asking for a feed.

We're seeing this in our error logs:

DateTime={ts '2007-10-22 14:47:22'}, Template=/BrowseAds/index.cfm, RemoteAddress=12.129.9.203, HTTPReferer=, Diagnostics=The request has exceeded the allowable time limit Tag: CFSTOREDPROC
The error occurred on line 223., QueryString=SN=103&M=100&RP=datedesc&D=summary&R=1, Cookie=, Browser=Mozilla/4.0 (compatible; +http://www.jobster.com/indexing.html)\

But the link their robot reports, http://www.jobster.com/indexing.html, is 404. For shame.

So, quick ARIN lookup on the remote IP 12.129.9.203 gives this:

AT&T WorldNet Services ATT (NET-12-0-0-0-1)
12.0.0.0 - 12.255.255.255
CERFnet ATTENS-SEA1-1 (NET-12-129-0-0-1)
12.129.0.0 - 12.129.63.255
Jobster, Inc. ATTENS-011426-005621 (NET-12-129-9-192-1)
12.129.9.192 - 12.129.9.223

So I'm blocking them at the firewall, until they decide to play nice.

In the press

2007-10-04T06:52:00.000-07:00

It's kinda weird when non-technical people refer to me as a "geek guru" but I guess in small pond like Poughkeepsie that works.

As a company, we almost never get press in local dailies because we compete with their classified revenue so directly. They hate us.

Here we are in a local business weekly, and that adversarial relationship is explained:

Radio the the Web and Back

Monster.com Spam

2007-10-02T16:39:00.000-07:00

Ha seems Monster.com (or at least, someone who works for them) got caught spamming, and then buried themselves deeper by trying to deny it. I saw this on the first page of Digg.com tonight:

http://www.mneylon.com/blog/archives/2007/10/01/monsters-irish-office-think-spamming-is-acceptable/

PurplePages search box

2007-09-06T13:59:00.000-07:00

Hey, now see that awesome PurplePages search box on the right of this page! Just added it there for testing but it looks good so I'll keep it. :D

Add a PurplePages.com search box to your site :D

quantcast.com for public web analytics

2007-08-31T11:00:00.000-07:00

Researching a competitor I found their "profile" at quantcast.com. Never heard of them, so I dug in, checking out regionalhelpwanted.com and cupid.com.

They do a "panel" analysis of web site traffic, like Alexa, ComScore, and Neilsens. But unlike those last two, they publish their results for free.

Alexa does that too, but I've always seen there data as heavily skewed. By nature, they skew heavily towards webmasters. Matt Cutts from Google pointed this out in his blog. I've also heard Alexa skews heavily toward young Koreans. Whatever.

Quantcast looks cool: they exact material that they present is curious: what web sites the users of a given domain have an affinity for. What magazines they read.

Another cool thing is that they let you put their Javascript on your site, so they can report on you with 100% accuracy. I'd *love* for all their competitors to do that!

But the don't make their methodology or sample size public, so it's all hogwash for now. I'll stick with Hitwise.

Update: looks like I'm musing in good company here.

New code? Bah.

2007-08-31T10:13:00.000-07:00

I found myself telling one of my developers yesterday "We've been doing this for years now. If you find yourself coding anything from scratch, you're probably doing something wrong."

In that vain, props to the War of Stuff blog for the code necessary to add Digg, del.icio.us, reddit onto my blog.

Duplicate content

2007-07-11T06:13:00.000-07:00

We have over 300 local web sites, each advertised on the radio as a local domain. But they all redirect to regionalhelpwanted.com because of 1 thing: Google.

If you have separate domain names that all have significantly similar content, especially if they are all hosted at the same IP, Google's algorithm infers that you are trying to spam the search engine, and all your domains get banned from Google. The broad rule is "duplicate content."

It does not matter that for our business model, separate domain names are completely legitimate. Google has no mechanism to make exception, and no venue to argue the merits of our case.

We learned this the hard way with Cupid.com. Can you imagine: we had tonsvof radio play, but when someone went to Google and searched for Cupid, we were not listed. Not on page 1, or 1000, or anywhere in between. It took us awhile to figure out what they had a problem with, and of course they
don't tell you, but even after we changed our system, it took 9 months for Google to list Cupid.com again.

I guess that's the downside of Google owning the search space: they make the rules and everyone else follows or sits out.

Trip to Colorado turns into Trip to Lake George, Part 2

2007-07-04T19:27:00.000-07:00

Having just email blasted a link to Part 1, I realize I haven't gotten around to writing Part 2, about how Lake George turned out to be our best vacation ever.

I'm too tired at the moment to write about Lake George. Hosting a barbecue this rainy day has wiped me out. A picture is worth 10k words? Here's 40k or so: ttp://www.kodakgallery.com/I.jsp?c=ycksfne.1c73twdy&x=1&y=-38ysc3

Trip to Colorado turns into Trip to Lake George, Part 1

2007-07-03T03:41:00.000-07:00

My family (a wife and five kids, you know) planned to be in Colorado this past weekend, for an informal memorial service for Mandy's grandmother who passed away last year. Mandy's dad organized the service, including arranging and paying for our flights. He'd be traveling with us. We'd be met in Denver by other extended family traveling from various parts of the country.

We agreed to meet at Stewart Airport, minutes from our house, between 5 AM and 5:15 for a 6 AM flight.

We arrive together. I drop off Mandy, the kids, and a pile of luggage at the curb, then find my way to Overflow Parking and back. I've flown out of Stewart a dozen times and never known Overflow even existed; its just not that busy of an airport. But a bus is waiting for me right behind my car, the driver reminds me to turn off my headlights, and speeds me back to the terminal.

Ever travel with five kids? I get the diaper backpack on Rose, age four, the toy backpack on Seth, six, the laptop and gadgets backpack on Meg, eight, and the snack backpack on Rourke, 9.9. Extend luggage handles, Meg pulls the small, Rourke the medium, I throw two toddler car seats over my shoulder by their straps and clasp large luggage. Mandy sees the plan and picks up what's left: Shane, 2.

John is waiting in line to check us all in by the time my pack and I get in. He's brought in two booster seats from our pile. Cool.

The line is huge, for Stewart: maybe thirty people at the airTran check in, and John says that they had just opened up a 2nd line and he had jumped into it, so was only four people deep.

There are four "self check-in kiosks" but I hear the attendant explain that people need to go one at a time because even though it is "self check-in" they still need to go through her. The wait is about fifteen minutes; one of the attendants called out while we were next in line: "Is the Bywater party here?" "Yes!" I called back. "Is the McLaughlin party here?" next. "Yes!" again; Mandy's maiden name. Cool, they know we're here. I'm feeling better.

So we finally get to the counter, and an attendant starts to check us in. It's now 5:30. At first she says that the flight was just closed but she can claim she was already checking us in and re-open it. But there is a problem, she doesn't have tickets for all of us. She reads off the names that she has tickets for, but doesn't have Rose or Rourke listed, and only has one Margaret (missing either Mandy or Meg, that is). I hand her back the ticket that she had just given us with Rose's name. My guess, in hindsight, is that they are in the process of giving our seats away upstairs at the gate as we are trying to get checked in downstairs.

She calls over a supervisor for assistance, and is truly trying to help us. She tells us that she is not going to be able to get us on, and even if she is they are going to close the doors ten minutes before the flight, and we still need to get through security. I ask if it's worth trying to get the kids rushed through security, the supervisor nods, so I bolt that way.

Mandy has already had the kids in line, and made it to the first security guard, who doesn't let our gang through because we don't have boarding passes. I explain that they are being printed, our flight is about to go, and the attendant had recommended I get moving through security since we have five kids in tow. The security guard bluntly states that we can't get past her without boarding passes. Grrrr. I bolt back to the check in.

Things there are over. No way we can get on the flight with eight of us, as it has been "way oversold, and closed." No way is there anyway to get us there before Saturday. Not with eight people. Ouch. John decides to bag the trip.

Regroup. Meg is crying because we aren't going on an airplane. Mandy and I put our heads together and decide to bring the brood to Perkins for breakfast while we figure out a new plan. On the way, Mandy calls Rocking Horse Ranch to see if they have rooms, but their Reservation Desk doesn't open until nine.

Ok, Lake George then.

Keeping scammers off online dating sites

2007-06-05T10:01:00.000-07:00

Mark Brooks, my former co-worker, did an interview with Nelson Rodriguez of LoveAccess regarding keeping scammers off online dating sites. Nelson is a great guy, and from our past-discussions I consider him an expert in this area.

The idea of the online dating industry sharing resources to fight scammers was discussed at the iDate 2006 conference in Miami; Mark and I were on that panel. I publicly stated that I would support that effort, and suggested Mark be a good person to spearhead the effort. Seems like his recent interviews are a step in that direction. Kudos, Mark!

I'll include here my comments on Mark's interview with Nelson:

We talked about shared effort at iDate Miami 06. I still think it has merit -- somethings are easy to share and don't get into proprietary info.

At Cupid.com, we only advertise in the US and Canada, so we block IPs known to be from any other foreign country. We use http://ip-to-country.webhosting.info/node/view/6
for this data.

Also, I have no need to allow anyone who wants IP anonymity on my site, so we also block all IPs known to be associated with TOR, The Onion Router.

Commercial anonymizing services will gladly block their members from access your web site if you send them a request.

IPs are easy to change. MAC addresses can be spoofed but that is a good idea I hadn't thought of. But as soon as you block someone, you alert them of the need to change. The only thing that seems to limit scammers is time. So when considering a scammer strategy, devise methods that let them waste time without letting them know that they've been identified. They will learn that your site doesn't convert for them, and they will be motivated to try elsewhere.

Netgear router = impossible to configure

2007-05-29T17:16:00.000-07:00

I wasted another couple prime Friday evening hours this week at my brother's office trying to configure his Internet connection. I set it up once months back but he lost connectivity and called Verizon instead of me. The tech had him wipe out my configuration, and then couldn't help him get back online. Grrrr.

The hassle is his DSL router and his Netgear Wireless router both want to be a DHCP server, and both want to be IP 192.168.1.1. I can configure the DSL router however I want, it's got a nice interface. So I disabled its DHCP and gave it a different IP. So far so good.

The Netgear router has a "configuration wizard" that you cannot escape. After a hard reset, it redirects all requests to 192.168.1.1, and writes a route aliasing the domain "routerconfig" or whatever to that adress. Then it tries to sense the Internet connectivity. If it doesn't have a upstream link, you can't proceed any further in the wizard. Nor can you bypass the wizard, Grrrrr! If it's plugged into the DSL router, it detects that it'll get a dynamic IP and beckons you to continue. But "poof!" it doesn't work. 404 for the next screen in the wizard.

I've concluded that the best way to "solve" the problem is to replace the Netgear wireless router with a different one, form *anyone but Netgear.* I talked to Jeremy, my Senior Systems Administrator, and he said that about a year ago Netgear switched their interface around and now they are total crap spo he's stayed clear.

I recently replaced my Netgear MR814 Wireless Router at home because it no longer allowed wireless connections. Most often it wouln't broadcast anything, not even its SSID. Most I could get out of it was 5 minutes uptime wirelessly. It guess that's a "security feature" since it couldn't do WAP either.

The replacement is a Belkin Wireless G Plus router. WAP, plus an interface I can do anything with. I sense I'll be buying one in the near future for my bro. Much better than losing yet another Friday evening.

Tragic follow up: High school poets

2007-05-18T18:46:00.000-07:00

Last Friday I blogged about listening to high school students recite poetry at Barnes & Noble; their teacher is another parent from my daughters soccer team. Our daughters played together while I listened to her students' poems.

This week when I arrived for the end of soccer practice, the eight year old girls were all gathered around a picnic table working on a group sympathy card for a team mate. I learned that, tragically, her father had died suddenly over the weekend of a heart attack. He had been at work. This was the same family who I had run into at the book store.

I was flabbergasted. I cannot imagine. This is a young family; he was not an old man. He was my peer.

It made me assess: am I doing everything I can do the keep my heart healthy? I made it to the gym this week twice, I need to go more. I've been eating healthy. What would my family do? I cannot imagine.

Highschool poets

2007-05-09T17:52:00.000-07:00

My 7 year old daughter -- D'oh!: 8 year old daughter! -- Meg and I both had birthday money burning a whole in my pocket tonight, and it needed to get spent at Barnes and Noble. While I was paging new paperbacks Meg's soccer coach asked if I'd come over and listen to her high school students present poetry. I applaud, no: snap, their willingness to stand up in public -- the closest thing we have now to a town square -- and read their own work.

It reminded me of a poem I wrote in college, especially appropriate for the young audience of tonight. I wish them all well.

"Bite Big"

Bite big!
Grow, puppy, grow!
Don't be afraid to choke.
Show canine smile around raw red meat.
Aggressively chew, aggressively eat
and Grow!

"The Diamond Age" by Neal Stephenson

2007-04-30T17:50:00.000-07:00

Neal Stephenson must really hate glass. In "Snow Crash" (yeah, I know book titles are supposed to be underlined, not quoted like an article, but Blogger doesn't allow for underline because now underline = link).... (Begin again:) In "Snow Crash" he has skateboards equipped with sonic glass-destruction blasts. And in "The Diamond Age" nano-extruded diamond has replaced glass because its cheaper and stronger. He highlights the quaint distortion of perspective through glass. Ok, Neal, we get it: glass is bad. (I'll need to re-read Cryptonomicon to see if it contains glass bashing.)

This isn't a review, just some of my thoughts after reading the book. I loved the book and will continue to read everything Neal Stephenson writes. Without going through his trash, I mean.

In "The Diamond Age" Neal explores the possibilities of nanotechnology. My brain is stuck on Neal's (can I call you Neal? Thanks.) posit that things built in a vacuum can be hollow, thereby contain a vacuum, yet be strong enough to maintain integrity when exposed to normal atmospheric pressure, and thus are lighter than air. I love the premise. Its elegantly simple but my brain rejects it. I want to see if it really works. I don't get to; thats why its called fiction.

Diamond is pure carbon, simple. But doesn't it take enormous pressure to get carbon to align itself in a crystal lattice? Neal, I wanted you to explain where all that energy comes from. Just a paragraph or two lay that piece out, please. Energy is everything, really. Once energy is solved, all physical things like starvation go away.

I enjoyed the node logic of the drummers. It fits my world-view. All are nodes, all have a value. The whole exists as nodes add and delete. The equation never completes.

New rule

2007-04-27T11:35:00.000-07:00

Lets just all agree, officially, that drinking Kahlua and coffee counts as multitasking.

Ok? Thanks.

How to tell if your website uses 301 redirects

2007-04-18T17:02:00.000-07:00

I was at Search Engine Strategies in NYC last week so I'm in an SEO mood. It was a good conference... 90% reenforcement and 10% new stuff. Most of the people there were marketers, not technologists, which is good because they did most of the small talking for me.

One thing Search Engine Marketing people stress is using 301 redirects instead of META REFRESH or 302 "Moved temporarily" redirects. 301 and 302 are HTTP headers. Most web pages send a 200 for normal status. 404 is standard for missing page ("Help! I've lost my 404 page!"). Google and, presumably, other search engines prefer 301 redirects. Why? I guess it intuitively makes sense... they don't want to bother putting something in their index if they think it's not going to be around for awhile.

I know I've made sure both RegionalHelpWanted.com and Cupid.com use 301 redirects. We have a new product, PurplePages.com in beta, though, and I wasn't sure if we coded that correctly. So I needed to check. And I realized most SES attendees probably wouldn't know how to check if their sites use 301 redirects and would have to either learn or trust their coders, so therefore I decided to blog it. It's good for marketing people to not have to rely on coders. Good for all.

You can't just "view source" of a web page to see if it's doing a 301 versus a 302, because by the time you have a web page you've already been redirected. You need something that will interrupt the process. An intermediary browser. A proxy!

In the past I've used Achilles proxy server for this. But I grimace every time I install it or suggest someone install it because it really is crappy. Free software doesn't have to be crappy. Achilles would run for a couple minutes and hang. Awful interface. But it was enough to get the job done. Barely.

Searching for Achilles today to install it on my new laptop, I found that it still hasn't been update but someone else had made a new proxy. Odysseus is the successor to Achilles. I love it-- wish it had come out years ago.

Ummm... if you are using a Mac ( and lots of marketing people do, right?) this isn't going to help you. Feel free to read on anyways. Like I could stop you anyway.

Download it (it's free). Install it. Start it... in your Windows System Tray right click on it and choose IE Settings > Odysseus. Now all your Internet Explorer requests will go through Odysseus. Right click on Odysseus again and choose Configuration. Check Intercept Reply. You can deselect Intercept Request unless you really want to click on the Ok button a lot (although that is great to leave on when use are security testing your site). Right click on Achilles one more time and select Interceptor.

Now navigate to whatever page you want to test. If you left Intercept Request checked then you'll need to click "Ok" for every pages and cookies along the way. Notice that the window title bar will say "Interceptor - REQUEST" for every item you request, and "Interceptor - REPLY" for your response from the web server. To see the status number, click on the Raw tab of a reply. The first line with be something like "HTTP/1.1 200 OK" for a normal page.

For me, I went to HudsonValleyPurplePages.com and got the status "HTTP/1.1 302 Moved Temporarily" as it redirected to http://www.purplepages.com/?osn=1 D'oh! I immediately assigned it to a Web Developer to fix. Then as a sanity check I went to HudsonValleyHelpWanted.com and thankfully it was "HTTP/1.1 301 Moved Permanently" to http://regionalhelpwanted.com/home/1.htm?SN=1&. Whew!

When you are all done, just right click on Odysseus again and deselect Interceptor to turn it off to resume normal browsing.