Skip to main content

Please enter a keyword and click the arrow to search the site

Crowd-judging on two-sided platforms: an analysis of in-group bias


Management Science


Management Science and Operations

Authors / Editors

Kwan A P;Yang S A;Zhang A H


Publication Year



Disputes over transactions on two-sided platforms are common and usually arbitrated through platforms’ customer service departments or third-party service providers. This paper studies crowd-judging, a novel crowd-sourcing mechanism whereby users (buyers and sellers) volunteer as jurors to decide disputes arising from the platform. Using a rich dataset from the dispute resolution center at Taobao, a leading Chinese e-commerce platform, we aim to understand this innovation and propose and analyze potential operational improvements, with a focus on in-group bias (buyer jurors favor the buyer, likewise for sellers). Platform users, especially sellers, share the perception that in-group bias is prevalent and systematically sways case outcomes as the majority of users on such platforms are buyers, undermining the legitimacy of crowd-judging. Our empirical findings suggest that such concern is not completely unfounded: on average, a seller juror is approximately 10% likelier (than a buyer juror) to vote for a seller. Such bias is aggravated among cases that are decided by a thin margin, and when jurors perceive that their in-group’s interests are threatened. However, the bias diminishes as jurors gain experience: a user’s bias reduces by 95% as their experience grows from zero to the sample-median level. Incorporating these findings and juror participation dynamics in a simulation study, the paper delivers three managerial insights. First, under the existing voting policy, in-group bias influences the outcomes of no more than 2% of cases. Second, simply increasing crowd size, either through a larger case panel or aggressively recruiting new jurors, may not be efficient in reducing the adverse effect of in-group bias. Finally, policies that allocate cases dynamically could simultaneously mitigate the impact of in-group bias and nurture a more sustainable juror pool.


Crowd-sourcing; Crowd-judging; Platform governance; Platform operations; Two-sided marketplace; Bias; Experience; Learning

Available on ECCH


Select up to 4 programmes to compare

Select one more to compare
subscribe_image_desktop 5949B9BFE33243D782D1C7A17E3345D0

Sign up to receive our latest news and business thinking direct to your inbox


Sign up to receive our latest course information and business thinking

Leave your details above if you would like to receive emails containing the latest thought leadership, invitations to events and news about courses that could enhance your career. If you would prefer not to receive our emails, you can still access the case study by clicking the button below. You can opt-out of receiving our emails at any time by visiting: or by unsubscribing through the link provided in our emails. View our Privacy Policy for more information on your rights.