Winners: Best Use of Crowdsourcing for Description

Next, the spot light turns to the winners of the Best Archives on the Web awards in the category Best Use of Crowdsourcing for Description. This is the definition of the category:

Whether through Flickr, wikis, blogs or allowing users to comment on descriptions in their online catalogs, many archives are starting to harness the power of their regular researchers as well as experts around the world to help augment or create descriptions for their collections. This award will recognize crowdsourcing efforts that have resulted in a significant exchange of information for the institution.

The judges selected one winner and singled out one nominee to receive an Honorable Mention. And they are . . .

Winner: Nederlands Instituut voor Beeld en Geluid (Netherlands Institute for Sound and Vision)

The description of the project from the nomination statement:

“To explore the impact and success criteria of social tagging in the audiovisual heritage domain, a large-scale video labeling pilot, Waisda?, was launched in March 2009. The goal of Waisda? (which translates to ‘What’s That?’) is to collect user tags that can help bridge the semantic gap, to collect time-related metadata, and to offer people a new way of interacting with television programs, thus creating a connection with the television archive. Waisda? is the world’s first operational video labeling game in the cultural heritage field.

Waisda? invites players to tag what they see and hear. They receive points for a tag if it matches one their opponent has entered within a time frame of ten seconds. The underlying assumption, based on the ‘Games with a Purpose’ by Luis von Ahn, is that tags are most probably valid if there’s mutual agreement. Waisda? introduced three innovations: Using gaming as method to annotate television heritage, actively seeking collaboration with communities connected to the content, and using curated vocabularies as a means to integrate tags with professional annotations.

From the launch in March 2009 to November 2009 (period of the evaluation, the website is still operational, see the WebScience paper by Oomen et al. for more information), over 340,000 tags were added, of which 40.3% consists of matching tags (added by different players within the ten second time frame. In total, 42,068 unique tags have been added.

Waisda? was executed by the Netherlands Institute for Sound and Vision and KRO Broadcasting (Dutch public broadcasting organization). The Business Web & Media Group of VU University Amsterdam performed additional research on topics such as game play and tag quality. (They carry out research in light of their involvement in the PrestoPRIME European research project.) The software company Q42 built the application.”

Waisada? received a lot of love from the panel of judges: “Looking through the site I just wished that I knew Dutch, so that I could play. In some ways it reminded me of the Google Image Labeler game, but its application to video content was novel. Based on the nomination form and the accompanying papers, it appears that the data gathered through the game has in some cases been very useful to enhance the description of the videos. I also appreciated the work that the project team had gone through to market the site to their desired audience, including their use of social tools such as Twitter.” The rigor of the evaluation and documentation, as well as the sheer fun of the project, were key in helping snag the win for Waisda?. Also, it’s not every nomination that gets this response from a judge: “I also very much enjoyed watching the Dutch reality show about the farmer.”

Resources in English:
– Background on the game and an English summary of the evaluation can be found on the Images for the Future blog.
– Also, two papers on Waisda? were presented at the WebScience conference this year in Raleigh, N.C..

Honorable Mention: PhotosNormandie on Flickr

Longtime readers may remember that I wrote about the PhotosNoramandie Flickr group back in April 2009. Then, as now, the group exists because of the volunteer efforts of two people with the talent and the interest to make it possible-Patrick Peccatte and Michel Le Querrec-and because of the flexible and popular platform that Flickr provides. The purpose of PhotosNormandie is simple–to make archival images of the Allied invasion of Normandy more easily discoverable by more users and to attempt to correct and supplement their existing metadata. The fact that this takes place entirely outside the archival context makes it both more interesting and perhaps more threatening. Patrick and Michel represent no archives, but rather the kind of passionate amateurs who choose to devote their time to advancing knowledge about archival materials. The lack of a connection back to the original archival collections troubled the judges, but they noted that “this project does a lot of things right– in particular harnesses an existing community and tech infrastructure rather than trying to reinvent the wheel or try to get people to a website where they wouldn’t regularly go.”

And so, congratulations to our two notable European examples of using crowdsourcing for description!

Be Sociable, Share!

3 thoughts on “Winners: Best Use of Crowdsourcing for Description”

  1. Pingback: Waisda? wint prijs

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.