Actually, I may be the last person to learn about Andrius Kalikauskas and MS Labs. Andrius is a passionate guy with undergraduate degrees in math and physics and a physics PhD. Andrius was introduced to Xpertweb by Mitch and Flemming as a person who is interested in reputation systems. Since they are Xpertweb’s secret weapons, their recommendation ensured that he would be involved.
Andrius’ larger concern is thinking itself. Most people don’t realize that thinking is a tool so they don’t treat it with the objectivity they might use when shopping for a computer or Sawzall. The tag line of MS Labs is “Do you care about thinking?”
Flemming riffs on thinking:
The presence of that quote in this post is an example of Andrius’ work. He organizes and juxtaposes important points of view among his MS Labs members. I’d seen the quote before, but might not have found it again without seeing it in the MS Labs members’ feed. Logically enough, the MS discussion group is called “Thinking Relevantly.”
Relevance is the first service Andrius is performing for Xpertweb. Another is to help organize willing hearts and minds to help with the coding and rollout. Andrius is pressing Flemming (actually, himself and Flemming…;-) to have working code on May 23, for demonstration at the BlogTalk conference in Vienna. That’s what Andrius does, he catalyzes innovation.
Andrius and Flemming are both driven to take the broadest view possible when starting a project, questioning every assumption and bias. Combined with Mitch’s dogma-killing crusade, Xpertweb is reasonably protected from blowing smoke up its own ass. All three of them know that the most dangerous force in developing a new protocol is one’s own assumptions and biases.
Half a Loaf
Here’s an example of how Andrius is questioning our assumptions:
When you go to the references link, you see entries like this:
You’ll find 14 sets of these kindwords on the page, demonstrating why Andrius is so valuable. I picked a series of three at random, and discovered something about non-explicit reputation systems when I rea
Like blogs, kindword/frustration/evaluation comments can be helpful but they share a failing we’ve discovered about blogs, which is why people are trying to leverage them into knowledge management systems: non-explicit, anecdotal text is quotable but otherwise unusable. Here’s a report just in from Jason Calacanis on how his company’s venture capital database overtook the previous market leader, which was a blog about venture capital:
Reputation is too important to be a hobby.
If I’m looking for a Java programmer for a particular solution, and I need it by Tuesday, how do I use the accumulated kindword entries to find the perfect programmer? I want the kind words, of course, but I need quantitative ratings and average rating reports and numerical comparisons that act as pointers to help me get to the these little blocks of text. All of that info might be available elsewhere, but surely it will be captured on the site of the person with the reputation, (or in an RSS feed) so, that I can click on a link to an order form as soon as I’m satisfied that I’ve found what I need. The Xpertweb approach is to require the buyer to provide the kind (or not) words and a number grade (01-99%). Then we can look at all the Java programmers who have sold n or more tasks involving, for example, graphic representation of numerical data.
The operative word is require. A reputation system is worthless if it captures ratings only at the whim of the buyer, or worse, at the whim of participants in a forum, as in this example, where the comments do not necessarily relate to a particular task. They can be too vague to benefit the next customer. Therefore it’s imperative to require that a grade and comment be recorded within a specified period after presentation of a completion report or invoice.
Pulling the String . . .
Have you ever had one of those projects which seemed simple, but once you got into it, you discover non intuitive requirements embedded in your initial enthusiasm? Such a discovery is like pulling a string out of a sweater. Actualizing the Xpertweb meme is a little like that.
Here’s what happens when you think seriously about a useful reputation system:
Fair enough, those forms could be designed in an afternoon. But there are other considerations. Once completed, where should their data be stored? Today, such information is kept by the seller. Naturally, the seller will yield to the temptation to excise the unflattering remarks. The data could be kept on a central server, but then what happens to reputations built through blood, sweat and tears if the central servers go out of business? It’s not like the W3C is gonna store this info for us. Just as bad, any centralized system may not scale as needed or worse, is corruptible, as described in the HumanTech Parable.
The only answer left standing is that both the buyer and the seller must keep the information, which must be identical to be valid. That means that both parties must have a web site with space and programming for the reputation system. Ratings so mirrored are demonstrably valid. If there’s any divergence, the ratings cannot be presumed to be valid.
The ratings are only useful if subsequent users can access the reputation data. Conventional wisdom says the data should be a mySQL data base with a CGI. Then, of course, each user would need an XML-RPC or SOAP routine to access reputation data from all the other sites. That’s a load which is sure to overload the requirement for user-maintained data. (Visionary doesn’t have to mean stupid–there are some experts who think Xpertweb is silly enough already!) The obvious but counter-intuitive answer is to post all data as pure XML in plain sight on each user’s Xpertweb site with known paths to the data.
And it doesn’t stop there. Our little Xpertweb engine (the little engine that might) needs to help its users describe specific products, like a customized PHP-MySQL shopping cart or mowing a 20,000 sq. ft. lawn. Once described, that product data must be accessible as a product page, and the product itself must have a reputation built around it. When someone needs their drain fixed, they’re not really looking for a plumber, they’re looking for a fixed drain.
That’s almost the end of our little string, so I’ll spare you the rest.
Freeing the Horse
As Doc reported on Thursday, these requirements seem implicit if you’re serious about a useful reputation system, sort of like seeing the horse in a block of marble and removing all the marble that doesn’t look like a horse. In fact, as Doc also related, a useful reputation system seems to me to be implicit in the XML spec. Though enterprises seem to be using XML primarily as a serialization routine (like SOAP) to connect legacy data systems, XML is fine as a data format, if you’re willing to live with its verbosity.
As a data format hosted on a web server, XML is readable by search engines, a skillion parsers and certainly by a thin-client purpose-built script like the one we’re building for Xpertweb. We’re even on the cusp of a promise dormant since the spec became a recommendation in February, 1998: An XHTML page can contain explicit links to bits of XML data and, without any programming display linked data when the page is opened. XML is truly data for the rest of us, because it frees us from CGI programming and the hidden data that only CGIs can talk to.
All those moving parts seem obvious and necessary if you’re serious about a useful reputation system. If there are any shortcuts, we’d love to hear about them. We have recurrent internal debate on whether all these moving parts are necessary, typically when a new teammate signs on, as in Andrius’ case.
I see a reputation engine as a kind of internal combustion engine. Even if it’s a two-banger, you still need quite a few moving parts to get it to turn over. I think we have a pretty good design and built-in means to re-engineer it while it’s running. That’s why I welcome dogma slayers,