How to Increase the Accuracy of Crowdsourcing Campaigns

Crowdsourcing is a new approach to performing tasks, with a group of volunteers rather than experts. For example, the Geo-Wiki project [1] aims to improve the global land-cover map by crowdsourcing for image recognition. Though crowdsourcing gives a simple way to perform tasks that are hard to automate, analysis of data received from non-experts is a challenging problem that requires a holistic approach. Here we study in detail the dataset of the Cropland Capture game (part of Geo-Wiki project) to increase the accuracy of campaign’s results. Using this analysis, we developed a methodology for a generic type of crowdsourcing campaign similar to the Cropland Capture game. The proposed methodology relies on computer vision and machine learning techniques. Using the Cropland Capture dataset we showed that our methodology increases agreement between aggregated volunteers’ votes and experts’ decisions from 77% to 86%. \n \n[1] Fritz, Steffen, et al. “Geo-Wiki. Org: The use of crowdsourcing to improve global land cover.” Remote Sensing 1.3 (2009): 345-354.