What Quora Can Learn from Wikipedia: Real-Time Editing
January 28, 2011 § 17 Comments
By: Christopher Lin
The hot topic for Quora these days (and ever) is the challenge of maintaining high content quality in the face of the great unwashed masses who don’t understand the site’s norms for rigor and eloquence. Solutions abound: Forced tutorials! Sub-Quoras! Make everybody a reviewer! Pay for full-time reviewers! PeopleRank!
Implicit in these solutions is the assumption that the old model of human administration and crowdsourced wisdom in the form of up- and downvotes are insufficient to address problems of the scale that Quora faces. Although this is true in the sense that throwing more warm bodies at the problem isn’t going to do much, I think it is incorrect to assume that a crowdsourced system of moderation is fundamentally unscalable. We know from projects like Wikipedia and large online forums that relying on a system of moderation driven by a small team of users with special powers and privileges can actually be very effective for encouraging desired behaviour and filtering out disruptive users. Rather than abandoning the attempt at human moderation wholesale1, I think it is more instructive to figure out why such a system has worked well for other projects but does not seem to be scaling well for Quora.
This will therefore be the first in a series of posts about lessons I think Quora can learn from other successful online collaborative communities. The goal is to examine the differences between Quora and these other projects, and how these differences affect the efficacy of Quora’s current approach to moderation; and to suggest ideas for how Quora can change to fit the conditions that make such an approach so effective for other projects.
Real-Time Editing: Real, Immediate, Visible Effects on Content
When you make an edit on Wikipedia, that change takes place immediately—the next person to visit the article will see what you’ve done. It’s a simple notion with important consequences, chief among them that vandalism is very easy and its prevention virtually impossible. But it also means that when moderators see that vandalism, they can swiftly fix it—a change that also takes place immediately and is visible to the very next visitor. Administrators always have actual final control over what appears on the site at the next moment.
Quora is different. Despite the real-time nature of every other interaction on the site, answers can’t be changed without the author’s permission; other users, including administrators, can only suggest an edit. This makes editing practically less effective since many users either refuse to or neglect to accept the suggested edits, and psychologically less effective as well, since there’s no immediate feeling of accomplishment after you’ve suggested an edit—the answer still looks as shitty as it was before. There’s no way of knowing how long it will be until the answer author gets around to accepting the suggested edits or how many people will show up in the intervening time and read The fundamental property of a crowdsouced edit has had the rug pulled out from under its feet.
The trade-off, of course, is that the clear attribution of Quora answers to their authors is what motivates people to write those answers in the first place. Moreover, the identity of an answer’s author is extremely important for interpreting the veracity and perspective offered by that answer. Identity is baked into the system for good, both in the sense of permanence and beneficence.
Unfortunately, the rapid rate at which the site is being assaulted with bad content increasingly evinces the fact that making edits immediate will be necessary in order to keep question pages clean. The high volume of crap has been particularly instrumental in demonstrating how differences in the immediacy of edits compound into broader differences in how easy it is to maintain high-quality content and keep users happy. For Wikipedia, the road from mediocre, half-baked content to polished encylopedic article can be very quick and painless as dedicated users take the kernel of substance in the initial contribution and provide better context, style, and tone. Moreover, because the ease of editing makes the content so amorphous, users contribute with the understanding that their content can and will be altered. As a result, moderators can erase bad or mediocre contributions without too much personal offense. On Quora, meanwhile, the fruitlessness of edits and the touchy social ground they tread—Do you really want to correct a stranger’s spelling, much less that of a famous CEO? And what if you end up seeing mistakes multiple times and keep bothering that person with repeated suggestions? Far better to just let it go, dude—make progress a much fuzzier task. Because you just don’t know exactly what you’ll get out of the time you spend making each edit, the entire enterprise seems more burdensome. And as more and more crap sticks to the pages, unfixable through intimidation or delay, the norms of content begin to suffer.
But if edits can be made in real-time, the ability of administrators to swiftly and effectively make drastic changes to visible content improves enormously. In particular, I think two specific types of real-time edits would be highly effective for cleaning up the answer sections on the site.
First, give administrators the power to turn answers into comments or posts when appropriate. By far the largest problem with the influx of new users has been each user’s initial confusion about the differences between these three types of content on Quora and what’s appropriate to write in each one. Right now, the only response that can be made is to mark the answer Not Helpful and write a comment explaining what the user did wrong. But this offends the user by hiding his or her content, potentially hides thoughts that are useful despite not being an answer, and leaves the page vulnerable to the answer getting lifted out of the collapsed region through upvotes from people who agree with the sentiment expressed. Instead of reducing administrators to such toothless pleading, just let them dothe right thing. Some users might still get angry, no doubt, just as some get mad now when their answers get buried, but at least now their content has been preserved without posing a continued threat to the cleanliness of the question page.
Second, allow administrators (and possibly other users who have displayed good judgment) to directly make edits to answers as they appear on the question page. Rather than showing the “old version” of an answer with a Suggestions Pending notice and a link to view the “new version”, the system should immediately begin displaying the new version of the answer with a notice that the answer has been edited and a link to view the old version. The attribution tagline of an answer would have to change from, for example, “Robert Scoble” to something like, “Robert Scoble, with edits by Tracy Chou (reasons for edit: capitalisation, grammar, gratuitous photograph).”
This second suggestion especially requires a lot of finesse in determining the precise extent of administrator powers, as well as in designing how these changes are presented in the interface and how any resulting controversies can be discussed and resolved. For starters, Quora will have to rejigger its terms of service to create separate conceptions of user-owned content and community-owned content so that such editing was legal. It should also take extreme pains to ensure that editing is not to be used for changes that alter the information or viewpoint presented, but rather for cosmetic fixes like spelling or grammar, or clarification fixes, like adding summaries of linked content.
It’s worth noting that the ability of administrators to make real, immediate, visible effects on content will remain an issue for Quora no matter how many machines are thrown at automatically classifying bad answers or how many administrators are thrown at them to review and suggest edits. Short of setting much stricter standards for answers and collapsing anything that remotely whiffs of mediocrity, successfully moderating answers will require giving administrators the power to make actual changes to the content instead of waiting around for the users to follow their advice. It’s a very hard issue to deal with, but one that is worth figuring out for the sake of maintaining Quora’s norms for content quality.
1 Although don’t get me wrong, I love Quora’s approach of using innovative technology to augment the ability of human moderators to do their job, and I definitely think those efforts will be very important for maximising content coverage. It’s just that when it comes down to it, a lot of work is going to need human supervision, at least in this decade, and figuring out how to maximise the efficacy of human administrators is just as important as writing classifiers to automatically guess which topics certain users shouldn’t be trusted to answer.
Christopher Lin likes data science, user interaction design, writing, the lulz, and being ever so presumptuous as to tell Quora what he thinks they should do. He is a contributor and reviewer on Quora, and maintains the list of Quora answers that have the largest number of upvotes despite being marked Not Helpful. He generally sympathises with the Rebel Alliance, but kind of wishes they had TIE Fighters instead of those god-awful alphabet soup starfighters. This is one of the poorer examples of his “quirky, self-referential bio blurbs.” Sorry.