The Problems of Scaling and Scoble at Quora

January 26, 2011 § 31 Comments

By: Dan Kaplan

You knew it was a new era for Quora when Mashable made a list – “9 Ways to Get More Out of Quora.” The writer in question, a public relations blogger named Heather Whaling, informed the Cashmore constituency that Quora offered rich land, fertile with opportunities to Expand Your Network and Establish Your Expertise. “You may notice,” she wrote “that people will begin to request your input.” The Quora Gold Rush was officially on. 

In the weeks since the influx began, the distribution of brilliance in the Quora feed has precipitously thinned. When I first joined the site, in February 2010, it was like gulping from a firehose of awesome. In January of 2011, it’s like sipping it through a narrow straw: the awesome still ekes through, but I have to work harder to taste it.

Under the pressure of new users, Quora’s answer voting system is also breaking down. Always vulnerable to cliquishness and groupthink but mostly held in balance by the icy standards of the early community and the Admin police, the up- and down-voting mechanism is increasingly unable to guarantee that the best answers will float to the top.

Indeed, it seems that in these transitional times, the best way to amass the most up-votes on prominent questions involves inserting photos of dubious value and composing highly digestible morsels of mass appeal. Giving yourself credit for the photo you inserted and tweeting your answer to a swarm of loyal fans who followed you to Quora from Twitter certainly doesn’t hurt.

The change in the site’s nature has aggravated some of Quora’s most dedicated early users. In backchannel discussions, members of this group have bemoaned the encroachment of Social Media Experts and the tangible decline in the quality of discourse across the site. They worry that amidst the cacophony of mediocre voices, the elites (whose answers are responsible for the majority the attention Quora gets from the press) will increasingly neglect or even abandon the community.

The danger of losing Quora’s elite core is not lost on its founders. A week after the appearance of the Mashable list, co-founder Charlie Cheever wrote the second post in as many weeks discussing the team’s efforts to maintain content quality in the face of onrushing Social Media Expertise:

The most intriguing of the three steps he outlined – which also included boosting the ranks of admins and better educating newcomers on how shit goes down in the hood – is “an algorithm to determine user quality…somewhat similar to PageRank” but not quite, “since people are different from pages on the web.”  Shervin Pishevar called this concept “PeopleRank,” and why not.

A reputation system that consistently elicits brilliance from the elite core yet doesn’t alienate new users would be a marvel of engineering and interaction design. But the PeopleRank concept raises a veritable fuck-ton of issues. Some of these issues are Google-hard Computer Science problems: how do you teach a machine to recognize legitimate expertise across a theoretically infinite range of topics and questions? Others are epistemological, because what defines legitimate expertise, anway? And who gets to decide? Both the technical and philosophical questions get at the heart of what Quora is and what it could become.

Quora as it currently stands is an increasingly chaotic but utterly democratic bazaar of brain output, where every up- or down-vote counts the same, regardless of the topical sophistication or motivation of the user clicking the arrows. My vote on an answer about scaling MySQL databases carries the same weight as Adam D’Angelo’s, despite the fact that I know negligibly more than squat about SQL and D’Angelo is a technical wizard.

An accurately PeopleRanked Quora could solve this problem. It would also be inherently undemocratic and elitist, potentially hindering Quora’s ability to penetrate the mainstream. But if the goal is to populate a database with brilliant answers to an ever-expanding set of questions, perhaps elitism is a legitimate position for the short and medium term. After all, the lasting value of Quora’s database hinges on the site’s ongoing ability to attract the planet’s brightest and most articulate people on any topic, extract their best insights and make them rapidly accessible to the curious of mind.

This is a worthwhile goal, but achieving it with PeopleRank means revealing to the hoi polloi precisely where they stand.

Dan Kaplan is a Quora aficionado. These views are entirely his own. You should follow Dan on Quora.

§ 31 Responses to The Problems of Scaling and Scoble at Quora

  • Quora says:

    The Problems of Scaling and Scoble at Quora…

    You knew it was a new era for Quora when Mashable made a list – “9 Ways to Get More Out of Quora.” The writer in question, a public relations blogger named Heather Whaling, informed the Cashmore constituency that Quora offered rich land, fertile with o…

  • […] This post was mentioned on Twitter by Todd Branchflower, Chris McCoy. Chris McCoy said: The Problems of Scaling and Scoble at Quora « The Quora Review – http://bit.ly/huFbCP […]

  • Thanks for your insight. I am Quora fan but do not spend enough time on it to understand all the dynamics that are unfolding.

    Regards,
    Cameron Chell

  • Joshua Russo says:

    One easy thing to do is that answers from people who have accumulated highly ranked answers could weigh higher than others. I’m sure they have thought of that though.

  • Vivek Wadhwa says:

    Dan, you did a much better job than I did in describing key issues. Very interesting.

  • […] of Scaling and Scoble at Quora (Dan Kaplan/The Quora Review) Dan Kaplan / The Quora Review:The Problems of Scaling and Scoble at Quora  —  You knew it was a new era for Quora when Mashable made a list – “9 […]

  • Really interesting post – though I’m not sure I agree with your characterisation of Quora’s current system as “utterly democratic”. There’s a fair amount of obscure and arbitrary moderation going on, with admins able to override community consensus with one click. Strikes me as an uneasy mix and a potential problem to be solved.

  • JFF says:

    Quora has a number of problems beginning with the fact that they do almost nothing to set expectations, tone, some rough “rules of the road.” People who are not in the technorati elite get there and have no idea whether they’re suppposed to jump in with half-baked opinons, unsourced facts or general jibber-jabber. You answer something and then wait to see if you get bitch-slapped, maybe with an explanation for the bitch-slapping, maybe not. Making matters worse, it’s becoming evident that a growing number of people are voting down and collapsing answers because they don’t “like” what they read, or took a dislike to someone’s name or pic. Great.

    There also seems to be a fundamental flaw in a business model that is going to be based on restricting users to an elite core. How many elites are there in the world? Do elites really need another little club to hang out in and kill time reading questions and answers? If so, can the company make any money on a niche market?

    In short, I’m not really sure what the basic proposition of the site is, and I’m not sure the founders know either. If they do, they might tell people what it is. Life 101 teaches that if you don’t ask for it you don’t get it. But I suppose that tells us something about Quora’s attitude at the outset: if you don’t know what this site is about before you get here then you don’t belong here. It’s not a good plan. That’s ok; all plans require revision … but you have to know where you’re going … somwhere between Yahoo Questions and Wikipedia is not enough definition.

  • Nerd Uno says:

    Quora is about 7 parts Soup Nazi, 2 parts gossip, and 1 part good tech info in the time we’ve spent there. Clique doesn’t begin to describe the problem. And soon the clique probably will be all that remains.

  • […] Kaplan / The Quora Review: The Problems of Scaling and Scoble at Quora  —  You knew it was a new era for Quora when Mashable made a list – “9 […]

  • steve248 says:

    Hey bud, its called building reputation systems. You can read about them here:

    http://buildingreputation.com

  • Pete C says:

    PageRank is named after Larry Page not Web Page , so if he’s going to draw a comparison mr clever should call it PishevarRank! Just sayin’.

  • Another related problem, but one that is limited only to the China topic on Quora, is the presence of propagandists who work for state media in the PRC or in internet companies controlled by the propaganda regime.

    For example, if a question is asked about Tibet, human rights violations, or any other myriad of topics which are unflattering to China’s image, the question will be watered down or answers will be voted down.

    There is only a relatively small group on Quora that take part in this activity and they all have a common thread. Many of them work for Chinese state media concerns such as Xinhua News Agency or China Radio International, or they work for internet companies in China which do censorship work for the Beijing propaganda regime indirectly.

    The small group is able to vote down answers and keep any sort of reality based discussion of China human rights and political freedoms discussions off of Quora.

    Examples and more discussion are available here:

    http://chinacensorshipwatch.org/tag/quora

  • […] Read more here Posted in Uncategorized , interesting, science, tech | No Comments » […]

  • scoble says:

    What’s with the scoble in the headline? It never appears in the body. Link bait??

    • Dan Kaplan says:

      While I don’t deploy Scoble’s name in the post, I do link to a prominent Quora answer of his and explicitly criticize some of his up-vote getting tactics.

  • Diana S says:

    @scoble:

    The term ‘scoble’ is used as a generic reference to a pernicious species of social media maven. This species is as common as the bedbug, as unwelcome, and as difficult to eradicate.

    • Dan Kaplan says:

      Diana, that’s not true. The problem of Scoble that I point out (i.e. Scoble using his wide off-Quora influence and off-topic photographs to garner up-votes) is actually distinct from the larger problem of social media mavens. The two issues are connected, but not the same.

      Also, Scoble is not a bed bug. He’s done a lot of solid work in the tech blogosphere.

  • Nerd Uno says:

    Funny… but far from accurate in the case of Robert.

  • kariobrien says:

    Think about Quora like the internet as a whole. Or even, a library! There are lots of authors, some with PhDs and years of experience. Some anonypluses. Non-mammalian semi-participants.

    On Quora and in the library you can read the little author bio, Google their name, find out how many copies were sold on Amazon…but at the end of the day, you choose to take that book home. You choose to read it. Every idea, you choose to accept or dismiss.

    The problem isn’t how many votes an answer gets, or whether Scoble’s minions vote it down (collapsed answers are still viewable to the curious), the problem is how smart is the person evaluating the answer. Are they really going to take a five to ten line home at face value?

    Thanks to the internet there are infinite “authors” and “books”: Alternative answers found outside Quora. My guess is this amazing resource which is crowdsourcing our collective knowledge as Google once did, will still be subject to requests for “a second opinion”.

    My Quora eval via Groundswell principles: http://bit.ly/hU63cy

    Note:
    Mashable didn’t start the “age of Quora”. Neither really did Scoble. It was Twitter. Google: Twitter Snowball Effect

    • John says:

      Your statement is great regarding the intelligence of the peorson collapsing comments. My experience has been (see comment below) that they do not research the relevance of an answer before collapsing. When I did rebuttal their actions they apologized (three different posts). Still do I want to spend my time defending my comments to a nameless admin? No. Like others in my network I will stick to books and blogs where I know the experts and comments are not heavily moderated.

  • John says:

    This past week I finally gave Quora a try. I was at first thrilled to see a space where so much great content was being shared as well as the opportunity to chime in where I had expertise. But it all turned sour when the “elite core” as you so well called them began to hide my comments. You can read the rest of my experience here. In fact I still have not found anyone yet that thinks Quora is still relevant. http://flurrycreations.com/theblog/2011/01/25/my-first-two-days-on-quora-scolded-and-collapsed/

  • @JoeHobot says:

    Just wait for the spamsphere to arrive, been on quora for some time now and the more I visit the more I dislike some of the questions.

    The idea is great, I might write about them thing or two but not yet when the blogsphere is buzzing how awesome it is… Myspace was awesome too :) back in the days…

  • Quora is reminiscent in many ways of Friendfeed. When first launched, great conversation by interesting people. Then the popularity, and then the sale to Facebook. Not quite the same place. Regardless of user controls, Quora will have their hands full keeping Quora … well … Quora. I have been mostly off line in January and find new Quora followers every day in my e-mail. Potentially too fast of a hit?

  • crowdsourced wisdom vs anointed experts – there are problems with both; on the one hand digital collectivism and lowest common denominators,on the other who recogises the experts (other experts so it gets recursive). solving either problem at scale would be quite the achivement…

  • Emad says:

    quote: “”” A reputation system that consistently elicits brilliance from the elite core yet doesn’t alienate new users would be a marvel of engineering and interaction design. But the PeopleRank concept raises a veritable fuck-ton of issues. Some of these issues are Google-hard Computer Science problems: how do you teach a machine to recognize legitimate expertise across a theoretically infinite range of topics and questions? Others are epistemological, because what defines legitimate expertise, anway? And who gets to decide? Both the technical and philosophical questions get at the heart of what Quora is and what it could become.””””

    This is such an embarrassing indictment of the author, demonstrating his fundamental misunderstanding of PageRank.

  • […] va-t-il détrôner Facebook ? » ROFLCOPTERDTC ), on a droit aujourd’hui aux articles des early adopters qui se plaignent de la détérioration du site depuis que tous ces nouveaux pékins sont […]

  • Great post. I found it while googling for others who have had issues with the arbitrary nature of Quora’s voting system and abuse of administrative authority on the site.

    After a steady slow climb of interest in the site and limited followers but highly voted up answers, I challenged an answer by a user that I then discovered was an admin. I found this out because after correcting this user’s answers in the comments, all my comments were deleted and I then found past answers of mine collapsed.

    This continued for weeks. I finally wrote about the issue and that only resolved the problem for under one week. I then found all my answers collapsed again and at this point harassment from users/admins began.

    Upon my complaint of this abuse and harassment, I was blocked. I was told that I was not to engage with personal conflicts with administrators.

    I have no recourse at this point. But if it is okay, I wrote the following post about my response. If you don’t want it here, I understand if you delete due to length:

    Here is the post that shares my feelings on this. I have been blocked from contacting any admin or moderator or user. I have been blocked from using the site and yet, Quora has no system in place for assuring users can delete their content for privacy concerns. Alas. (Forgive the spaces and breaks, this was cut/paste):

    It would appear that Quoras’s response to my criticism of abuse of authority by an admin (Shannon Larson) and harassment from a user
    (yesterday two emails were sent with links to the harassing content) is
    to block the person who brings this issue up.

    They have blocked my account without notice and without recourse.

    Criticism of Larson’s abuse of authority cannot be defined as personal
    conflict, as Larson is an admin and therefore represents Quora at all
    times. Nor can challenging the inaccurate answers Larson has sometimes
    given. Nor can challenging inaccurate answers from other users on the
    site.

    Incorrect answers should never be upvoted or encouraged. Regardless of
    who makes them. And any user should have ability to challenge these
    inaccuracies and correct them without fear of penalty, regardless of the
    popularity of the falsehoods or myths.

    By blocking a user who addresses the abuse of admins or challenges
    inaccurate/incorrect answers, Quora has demonstrated again their
    priority and desire to be more of a mass appeal network/site and not necessarily one for intellectual integrity.

    There is evidence of arbitrary and personal collapsing and downvoting of
    answers by me for no just cause. There is also evidence of user
    harassment directed personally to me.

    That users are blocked without just cause and without recourse from
    Quora is further evidence that the site cannot be trusted for objective
    moderation.

    Perhaps there is a remedy and solution, but as of today, it doesn’t
    appear that appealing to Quora administration for solution is advisable.

  • Heidi Klum says:

    Wow! This can be one particular of the most useful blogs We have ever arrive across on this subject. Actually Fantastic. I am also a specialist in this topic so I can understand your hard work.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

What’s this?

You are currently reading The Problems of Scaling and Scoble at Quora at The Quora Review.

meta

Follow

Get every new post delivered to your Inbox.

Join 261 other followers

%d bloggers like this: