The specifics of the data are not important here. 37s has a real knack for creating a continuous stream of positive buzz using status updates like these in addition to their blog posts. It's less about their average customer rating or whatever increasing by 3% this year, and more about knowing the company is still alive and doing well with a good product and happy customers. What's particularly smart is that, excluding their books, their articles and reports aren't long or comprehensive, but have the minimal amount of information to offer enough value to keep them relevant and in a positive light -- like when you kick your feet periodically in water to stay afloat. It's not easy by any means. I think it's quite clever.
It's also a marketing moat because it raises questions for customers when competitors don't have similar report cards -- what are they hiding? The low-pressure copy and honest voice are a big help, too. Very clever indeed.
I think transparency is a huge competitive advantage. And like you said, it works great as a marketing tool AND as a tool for accountability. There are several ways you can do this but probably the easiest is to just sign up for a service like Pingdom (which is great b/c it injects a 3rd party measurement into the equation) then use their status page tool. We did ours using a twitter widget, pingdom's status page, and some CSS, and I think it turned out OK: http://www.getadministrate.com/status/
I am not sure if average page load times are a good way of measuring site performance. I wish they published numbers like median, 75th percentile and 95 percentile load times.
More numbers == harder to read. The point is not to give an accurate picture (as someone else pointed out, they certainly have more detailed internal metrics). The point is to make potential customers feel that their apps are fast and responsive. Hard numbers make that sound plausible. More detailed numbers would make the point diffuse and confusing.
I am not sure if either of us can accurately say what is the point "they" are trying to make. The point I was trying to make is that a large number of developers who look at 37signals for technical leadership could be dangerously misled into using average as a performance metric.
Leaving aside the technical issues even from a customer point of view using average to measure performance is pernicious. There are scenarios where the average remains the same while almost half their customers are experiencing load times that have doubled. What should the report card say in that case?
"92% gave us the highest Customer Service rating."
This seems somewhat disingenuous, considering that the rating scale was only out of three, and the "highest" level is used to show any satisfaction whatsoever. Their copy implies some sort of rigor to their rating process, and suggests that the "highest rating" is somehow significant and indicative of inordinate quality, as would be implied by a five or ten out of ten. A three out of three when the bottom two ratings indicate dissatisfaction or displeasure isn't really saying much. I'd think that it's fair to assume that the majority of most customer bases like the product in some way. A more realistic description would be to say that "92% of our customers were pleased with our service."
It's like saying that 92% of students in a pass/fail class earned the highest grade. It might technically be correct, but it doesn't mean anything.
While there has not been much downtime, I have noticed a big spike in their Twitter account apologising for some bug or other.
In fact, for last month or so, every second and third day, I see a tweet from them in my timeline saying "Sorry for disruption in ________service of <product>"!
They continue to be a great example of a company that tries to communicate when something is an issue (or is going great).
It seems like the critics of which metrics are highlighted are missing the point... the average person who uses their products or plans to sees a company who is willing to communicate how they are doing.
If you need to dig into more detailed stats, they're not doing a good job with how their service performs. The whole point of their services is so that you don't need to worry about those things :o)
I think one useful metric to track would be similar to the stock market's "52-week high." As an end user, comparing to the last month isn't very useful. But knowing, is this the top mark they've had in the last 12 months? That provides richer context.
At https://starthq.com we are working on producing similar report cards for all the web apps out there. For the web app developers out there: is there any other information that you would like listed?
Good to see a mix of quantitative and qualitative measurements in there. While uptime and customer satisfaction are probably correlated, there are probably more factors influencing the latter.
I thought this might be a survey or request for users to fill-out a report card, not an informational page. We tried Basecamp once. It seemed like an expensive implementation of something we could easily setup ourselves. Not sure why everyone is so 'ga-ga' for 37signals...
I'm curious about how you'd easily setup this for yourself? As for the expensive, you can get away for a small team with something like 20$/month. 20$ is a fraction of what a software developer earn per hour!
I believe people are so 'ga-ga' (to use your term) because they keep it to the bare minimal (but good luck figuring out what that minimal is!) instead of adding thousands of features. We've been using basecamp exhaustively for a couple years. Is it perfect? No. But it's the best good-enough I've found to get shit done.
