The Good, the Bad and the Ugly

Why Crowdsourcing Needs Ethics

This position paper for the workshop CrowdWork 2013 discusses some of the ethical implications of crowdsourcing in general and of contest-based crowd design in particular, especially in regard to the question of fair payment. The paper establishes four different categories of crowdsourcing with separate ethical challenges and argues for the crowd work industry to develop a code of ethics from within, in order to counter the exploitation and abuse that it often enables.


Crowdsourcing has been a loaded term right from the beginning when it was coined by Jeff Howe in 2006 1. By building on the notion of outsourcing, the term has a lineage of cheap labour and globalised exploitation written into its DNA. Furthermore, as I have argued elsewhere 2 3, the crowd as a term refers to other people, it is always them not us who are supposed to do the work… In his original article for Wired, Howe left no doubt that crowdsourcing would be a disruptive and potentially destructive force for various professions, not only but also the creative industries. By using the example of iStockphoto, Howe was able to show the changes in the field of photography, where the price of a stock image was brought down by the power of the amateur crowd from $100 to just $1. With an economic impact of such magnitude, it is no wonder that crowdsourcing has quickly evolved from yet another buzzwords into the engine of a whole industry, that claims to double its workforce every year 4. Due to its rapid growth and its applicability to all kinds of tasks, crowdsourcing has become a relevant issue for society at large. It has the potential for collaborative problem solving and contributions to the commons of unprecedented scale, but it also completely undermines hard earned and fought for standards of fair labour 5. This holds especially true when we talk about crowd work, where people laboriously try to make a living, while platform owners and clients try to maximise their profit. When crowd work is not just an occasional pastime but becomes the reality of a daily workplace, an ethical debate about what conditions we regard as appropriate or acceptable becomes more pressing. It is important to think not just about the Good, but also about the Bad and Ugly side of crowd work and about how to improve it.
There is a huge gap between those like Don Tapscott and Anthony D. Williams who paint a picture of crowdsourcing as the holy grail of innovation that will ‘fix a broken world’ through ‘social production’ and that will help ‘create a new civilisation’ 6 7, and those for whom crowdsourcing is outright ‘evil’ 8 9. The important point here is that crowdsourcing as a method has the potential for both, with many shades in between. An ethical debate is therefore necessary in order to help crowdsourcing to evolve for the better.

Crowdsourcing Definition and Categories

When it comes to what constitutes crowdsourcing and its subcategories, there are many rival terms, such as collective intelligence, open innovation, common based peer production, human computation, or cloud labour, which are overlapping but all have slightly different meanings. Attempts to create an integrated definition just for crowdsourcing alone can already get unwieldy 10. For the sake of simplicity, I use crowdsourcing as the overarching term of the phenomenon, because it is the one most widely used by the industry itself and I also stick to Jeff Howe’s original definition: ‘Crowdsourcing represents the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in form of an open call’ 11. The term crowd work, in my understanding, is a subcategory that puts emphasis on the labour aspect of many crowdsourcing tasks, in contrast to those that are perceived as just play or game or are not perceived by the crowd at all. As for the subcategories of crowdsourcing, I build on a classification briefly outlined by Alek Felstiner 12, but I suggest to further simplify it into just four distinct categories, by focussing only on the type of incentive being used to engage the crowd:

Crowdsourcing with a monetary incentive:

  • cognitive piecework
  • contest-based crowd work

Crowdsourcing without a monetary incentive:

  • volunteer crowd work
  • disguised or epiphenomenal crowdsourcing

The first two categories both use monetary rewards to motivate contributors. In cognitive piecework, a term coined by Lilly Irani 13, the workers are paid micro sums for every micro task they perform. The most widely discussed example here is Amazon’s Mechanical Turk with its Human Intelligence Tasks (HITs), starting for a cent a piece. In contest-based crowd work, the workers can potentially earn larger sums, but have to compete against each other in order to get paid at all, for work that they have already completed in advance. This is the form of crowdsourcing most commonly used for design tasks, a typical example here is 99designs. In volunteer crowd work, participants donate their workforce, often for a perceived greater cause, the most prominent example here is Wikipedia. And in disguised or epiphenomenal crowdsourcing, contributors are either not aware that they are the source for someone or something else, as is often the case with people using reCAPTCHA, or they just don’t care because the harnessed product of their intellect is just a side effect, an exhaust, of an action they primarily do for themselves. The term epiphenomenal in this context originally comes from Jonathan Zittrain 14, but I understand it in a wider sense that also allows to include the affective labour of social networking and and the user-generated content that emerges as a side effect in computer games such as Spore.

Not a crowdsourcing category in its own right, but an important incentive mechanism beyond money that can be found especially in the last three categories is gamification. It is the introduction of points, badges, leader boards and virtual achievements that function as a rewarding pseudo currency. It is similar to money in the sense that contributors ‘earn’ something for the time and work they invest, but quite unlike money in the sense that these points can neither be spent, converted or extracted from the platform they stem from. They mainly serve as feedback and for the display of social status. Gamification is a crucial element of crowdsourcing as was already described by J.C. Herz in 2002 in Harnessing the Hive (even before the terms crowdsourcing and gamification existed) 15. Because of its manipulative potential, Ian Bogost has suggested to better speak of gamification as ‘exploitationware’ 16 17.

In the four categories suggested above, I exclude crowdfunding altogether. Even though this practice is sometimes described as crowdsourced fundraising, it follows a different logic and demands a separate discussion. Here, I am concerned with the crowd as a source for labour, not as a source for money.

The Thorny Subject of Exploitation

Due to its success, crowdsourcing today probably polarises more than ever. The website, which describes itself as a hub for the industry, already lists over 2000 crowdsourcing platforms. The huge diversity of the platforms makes it difficult to discuss the ethical issues that are connected to crowdsourcing without falling into sweeping statements. Still, there is one reproach that sticks out, an issue that appears to be inherent to crowdsourcing across the board, it is the accusation of exploitation. One of the strongest voices among the crowdsourcing critics is, interestingly enough, Jimmy Wales. He describes crowdsourcing as a ‘vile term’ and rejects the underlying business model as a way to ‘trick people into doing work for free’ 18 19. How come Wikipedia is usually not accused of exploitation, even though it is built on free labour? Most people tend to have an intuitive grasp of what exploitation entails, but it is tricky to pin it down precisely. Especially when the potentially exploited click on ‘agree’ from the safety of their home, without being physically coerced into work, or the free labour is predominantly perceived as a fun pastime, even though someone else makes a huge profit from it. The most thorough academic analysis of exploitation was of course done by Marx, but it is also so fundamental and technical that it leads away from this intuitive grasp for fairness that we all have. In What’s Wrong With Exploitation the philosopher and political theorist Robert Mayer takes a fresh approach to define the problem 20. He writes that exploiters inflict losses and thus harm by failing to benefit their victims as fairness requires, even when their interactions are mutually advantageous. Mayer makes clear that even if the workers are, in absolute terms, a little bit better off – for example through the approximately $1,50 they might earn per hour on Amazon’s Mechanical Turk instead of earning nothing – they are exploited when they do not get a fair share of the value they create. Coercion, according to Mayer, is a separate wrong and not a necessary precondition; it therefore doesn’t have to be forced labour in order to justify the term exploitation. Treating people unfair is enough and what can be considered as fair must be negotiated. Robert Mayer also points to the thorny problem of legal measures. An abolishment of exploitative working conditions can easily take away the little income that people make from getting exploited and therefore on short term can worsen their situation instead of improving the conditions step by step towards more fairness. A concern that is brought forward by crowd workers quite often, when legal measures against crowdsourcing are discussed.

While the problem of exploitation certainly is the elephant in the room, for a wide range of crowdsourcing practices, there are also very category-specific ethical issues that I will briefly discuss in the next section. Given that this is a short workshop paper, I will mainly focus on crowd work with a monetary incentive and especially on contest based crowdsourcing in design. The reason for this particular focus is, that I have a background in design myself and the research that I am conducting as part of my PhD is an ethnographic study of crowdsourcing in design.

Category-specific Ethical Challenges

Cognitive Piecework

In the paper Working The Crowd from 2011, Alek Felstiner has meticulously analysed the legal situation for cognitive piecework in regard to fair labour laws and minimum wage standards in the US by looking closely at Amazon’s Mechanical Turk. After weighing in the advantages that workers certainly get through crowdsourcing, especially the freedom to work from their home in their own time for whom they want, Felstiner unequivocally makes clear the downsides:

‘Crowd workers tend to receive extremely low pay for their cognitive piecework […] They usually earn no benefits and enjoy no job security, and in fact the vendors may seek to prevent them from doing so. […] Crowd laborers do not enjoy true legal protection on the job, and the cyberspace in which they work remains essentially unregulated for employment and labor law purposes. In addition to these fundamental drawbacks, crowd workers also encounter problems with information asymmetry, deception, and privacy‘ 21.

There are now a number of attempts that try to tackle these problems in commercial crowdsourcing from the outside. In California, Christopher Otey has filed a class action law suit against CrowdFlower, one of the worlds largest platforms for cognitive piecework. The lawsuit is set to answer the far-reaching question if crowd workers are free contractors or if they are actually employees eligible to the minimum wage under the Fair Labor Standards Act 22 23. For the crowd workers in the US the case could decide wether they will continue to earn on average between one and two dollars per hour or the minimum wage of $7.25 per hour. For the US crowdsourcing industry, the case questions the feasibility of their business model as such. However, seen from a global perspective, crowdsourcing has not much to fear from national labour laws. In this context, worker initiatives that try to create solidarity and counter information asymmetry seem more promising. An example for that is Turkopticon, a browser plug-in, developed by Lilly Irani to give the so called ‘Turkers’ on Amazon’s platform a voice and allow them to monitor ‘report and avoid shady employees.’ As the site claims:‘We’re watching out for the crowd in crowdsourcing because nobody else seems to be. Turkopticon helps the people in the ‚crowd‘ of crowdsourcing watch out for each other—because nobody else seems to be.’ 24 25.

Contest-based Crowd Work

As already mentioned above, contest-based crowdsourcing is particularly endemic in the field of graphic design. The most problematic issue here, in addition to the forbiddingly low hourly  income rates, is the systemic waste of work and the uncertainty for those who contribute their work if they will ever get paid. Compensation for the time consuming skilled work resembles a lottery and it turns out that graphic design lends itself to this mode of crowdsourcing partly because there are so many underemployed designers, partly because the necessary tools of the trade are so easily available and also because the practice itself is for many people already intrinsically rewarding. There will always be a crowd willing to design for free, something that is less likely for the often repetitive drudge of cognitive piecework. However, many of the designers who do crowd work are neither amateurs nor do they want to work for free. The motivations vary greatly, but common reasons for participation are the search for experience and exposure and the belief to do better than the next person and therefore have better chances to win the price. Globalisation also plays a huge role, many contributors come from countries with very low wages. Young designers trying their luck in crowdsourcing are in ample supply. A particular large crowd design platform,, claims to be ‘the fasted growing design market-place in the world’. It has more than 200,000 registered designers and it already conducted over 180,000 design contests. Even though the site boasts many numbers, the pricing scheme of 99designs is very opaque. It is not directly visible that the site takes a share of 40 to 45 per cent. From the initial $300 that a client is paying for a logo contest, the platform takes off $120 right away. The client gets on average 116 logos, which leaves the designers with a chance of 1 in 116 to eventually getting paid $180. The average renumeration comes down to about $1.50 per logo design, before taxes. The design of a logo usually takes several hours of research into the clients identity concentrated design work. While the crowdsourcing platform always wins, all the risk, liability and workload has to be carried by the contributing designers. Drawing from my ongoing participant observation as a designer on various crowd design platforms, I can state that these websites are geared in a way that entices designers to put in far more hours than would be economically sensible. An important reason for this are the individual design portfolios that automatically build up on these platforms over time. In contrast to the more anonymous cognitive piecework, the design portfolios stick with the online persona of the designer and can aesthetically be judged by anyone also much later. Designers tend to be very aware of this exposure and are therefore inclined to invest even more time. Others, who seem to be doing the maths, frequently steal ideas and even existing designs from elsewhere on the internet to produce competitive work within a few minutes. This again unsurprisingly frustrates those putting in the extra hours and they in turn try to reveal the cheaters to the client. The exploitative mechanism of these competitive crowdsourcing sites for creative tasks therefore creates a poisonous and ugly working climate. There are initiatives such as No!Spec ( that through education try to prevent designers from taking part in commercial crowdsourcing contests, but the number of platforms in this area and the number of contributors participating is still growing quickly. There are also notable exceptions in this field, platforms  such as, that are more balanced and fair and also consider the interests of the designers. Jovoto tries to foster a sense of community, the projects are for big brands, they are more complex, the prize money is often ten times higher then on the standard crowd design platforms. Most importantly, the community has a say in who wins a share of the prize and the most successful designers here have over time made up to €30.000 with their ideas. But for that, they have also contributed hundreds of designs, of which one out of ten might win something. Even the most successful and ambitious designers on the fairest contest-based crowd design platform make on average only between 100-200 Euro for elaborate finished design concepts for big brands. While it is questionable if this counts as exploitation, it is certainly not a sustainable business model for the participating designers, but a bargain for the companies.

Volunteer Crowd Work

A great deal of volunteer work by the crowd online is done for the greater good and not for profit. It is contributed by the many for the benefit of everybody and Wikipedia is the poster child of this form of commons-based peer production, as Yochai Benkler calls it. As mentioned above, Wikipedia is generally not seen as exploitation even though it is sustained by unpaid labour. Is it therefore better not to pay your contributors at all than to pay them too little? This seems to be the conclusion of OpenIDEO, a crowd design platform that only offers gamification points, experience and maybe exposure for those who help solve complex design problems for IDEO’s clients. Even though the projects on this platform generally tackle issues that seem beneficial for the common good, it is not transparent for the contributors who  actually benefits to what extent. For example when the city of Singapore hires the large for profit design agency IDEO in order to let the crowd on OpenIDEO develop ideas that foster sustainability and a sense for community in the city. Without doubt, IDEO invests a lot of time and expertise on the management of its creative voluntary crowd, but for the participants, it is not at all open what IDEO gets for that service.
Another ambiguous example is Al Gore’s project Reality Drop, a crowdsourcing site that uses gamification as an incentive to ‘Spread Science about Climate Change, Global Warming.’ The site serves as an armoury for human spam bots who are sent out to ‘spread truth’ and ‘destroy denial.’ Members of this crowd gather points for copying and pasting pre-formulated short arguments into online discussions that then link back to the longer arguments on the ‘Reality Drop’ site. It is especially designed to attack articles that question climate change. Even though the cause might be right and noble, this form of organised propaganda war in the trenches of the comment sections shines a light on the manipulative potential of crowdsourcing.
Volunteer crowd work is also increasingly used against perceived wrongdoing or even crime. There is  the German site Guttenplag that engages the crowd to reveal plagiarism in doctoral theses, preferably of high profile politicians. There is the London Metropolitan Police that tries to find rioters through a specific crowdsourcing app called Facewatch ID and in the US, a company called BlueServo has pioneered realtime crowdsourced surveillance, that allows everyone to become a virtual sheriff, watch the Texas border and report illegal immigrants. Finally, in the UK there is Internet Eyes, a company that even charges for giving the crowd access to CCTV cameras in shops and then ‘gamifies’ the catching of shoplifters. While crowdsourcing might be efficient to battle certain unwanted behaviour, this certainly brings up all kinds of ethical questions regarding the motives of those watching and reporting, the effect on those who should be paid doing these jobs and regarding those being named and shamed, potentially innocently, by a vigilant cyber mob.

Disguised or Epiphenomenal Crowd Work

Crowd workers, for example on Amazon’s Mechanical Turk, are often so detached or alienated from their work, that they have no chance to figure out what will be the final sum of the small parts they are producing. They neither necessarily know who their employers are, nor if their goals are justifiable. Already in 2008, Jonathan Zittrain has pointed out in Ubiquitous Human Computing that ‘disembodied human intelligence tasks can deprive people of the chance to make judgments about the moral valence of their work. […] Whether captcha-solving is deployed to scan an old book for a university library or to support the work of a particular religious institution or political party or spammer, the captcha-solver ought to know. Such knowledge will help develop and reinforce norms about what kinds of borrowed mental cycles are acceptable’ 26.


In order for the crowdsourcing industry to become more respectable and sustainable for all stakeholders, it has to openly address these ethical issues from within. But the underdeveloped ethical debate in particular in regard to the problem of exploitation is striking when one looks at how the crowdsourcing industry presents itself at conferences, in publications and online. A notable exception is a recent paper by Kittur et al. 27. In The Future of Crowd Work, a group of authors that are all intimately involved in the development of crowdsourcing tools, ask the crucial question: ‘Can we foresee a future crowd workplace in which we would want our children to participate?’ By framing the problem in this way, they very effectively cut through the long and complicated history of moral philosophy and instead address the core of the issue in the most direct and engaging way. The question makes it evident right away, that all is not well in the world of crowdsourcing at the moment, especially when it comes to fairness, respect and economical sustainability for the contributors. To acknowledge that is the first step to establish new ethical standards for the future of crowdsourcing.

No doubt, there will always be exploitation in crowdsourcing, it is to some extent inherent to this mode of production because, for example in contest-based crowd work, the financial resources for payment are limited while the number of participants in  the crowd is not. As soon as the product of the crowd is privatised and transformed into profit while not being beneficial at least to those in the crowd, one can argue that exploitation emerges. The criteria which constitute fair use of crowd work have to be developed in negotiation between all stakeholders, those who make the tools, those who use them, those who work in the crowd and those in politics who have the mandate to protect the rights of the people and make sure that this new mode of production does not annihilate hard won achievements of fair labour standards. One way to think about this could be along the lines of a Fair Trade Mark. In order to live up to its huge potential and ambition to solve global problems on a massive scale, crowdsourcing first has to figure out for itself, how to be less of a global problem.



  1. Howe, J. ‘The Rise of Crowdsourcing,’ Wired, 2006
  2. Schmidt, F. A. ‘Crowdsourcing Design: For a Fistful of Dollars,’ A Peer Reviewed Newspaper About #BWPWAP, (Digital Aesthetics Research Center, Aarhus University in collaboration with transmediale 2013). Article can be found here.
  3. Schmidt, F. A. ‘For a Few Dollars More: Class Action Against Crowdsourcing,’ A Peer Reviewed Journal About #BWPWAP. Article can be found here.
  4. Massolution, Enterprise Crowdsourcing Research Report,, 2012
  5. Scholz, T., Digital Labor: the Internet as Playground and Factory (New York: Routledge, 2013)
  6. Tapscott, D., Williams, A., Wikinomics: how mass collaboration changes everything (London: Atlantic, 2008)
  7. Tapscott, D., Williams, A., ‘From Crowdsourcing to Kony 2012: Macrowikinomics: New Solutions for a Connected Planet’, YouTube video.
  8. Howe, J. ‘Is Crowdsourcing Evil? The Design Community Weighs In’, Wired Business, link.
  9. Fuller, M., Evil Media (Cambridge, Mass: MIT Press, 2012)
  10. Estellés-Arolas, E., González-Ladrón-de-Guevara, F , ‘Towards an Integrated Crowdsourcing Definition’, Journal of Information Science, 38 (2012), 189–200
  11. Howe, J. ‘Crowdsourcing: A Definition
  12. Felstiner, A, ‚Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry‘ (2011). Berkeley Journal of Employment and Labor Law, Vol. 32, No. 1, 2011.
  13. Irani, L. ‘Tweaking Technocapitalism’ (2009)
  14. Zittrain, J., ‘Ubiquitous Human Computing’, University of Oxford Legal Research Paper Series, 2008
  15. Herz, J. C., ‘Harnessing the Hive: How Online Games Drive Networked Innovation’, Esther Dyson’s Release 1.0, 18 October 2002
  16. Bogost, I., ‘Persuasive Games: Exploitationware’, Gamasutra, 2011
  17. Bogost, I., ‘Gamification Is Bullshit’, Ian Bogost, 2011
  18. Wales, J. ‘As Wikipedia moves to S.F., founder discusses planned changes,’ 2007
  19. Think Quarterly – the Open Issue, Google 2012
  20. Mayer, R., ‘What’s Wrong with Exploitation?’, Journal of Applied Philosophy, 24 (2007), 137–150
  21. Felstiner, A, ‚Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry‘ (2011). Berkeley Journal of Employment and Labor Law, Vol. 32, No. 1, 2011.
  22. Mack, E., ‘The Lawsuit That Could Help Undo (or Cement) Crowdsourcing in the U.S.’
  23. Cushing, E., ‘Amazon Mechanical Turk: The Digital Sweatshop’, Utne Reader, 2013
  25. Hodson, H., ‘Crowdsourcing grows up as online workers unite’, New Scientist, 2013
  26. Zittrain, J., ‘Ubiquitous Human Computing’, University of Oxford Legal Research Paper Series, 2008
  27. Kittur, A. et al. ‘The Future of Crowd Work’ (presented at the CSCW Conference on Computer Supported Cooperative Work and Social Computing, San Antonio, Texas, 2013)