Accessibility of the Web with a Computer Game

Images on the Web present a major accessibility issuethis responsibility off their hands. We set our goal to
for the visually impaired, mainly because the majorityassign proper descriptions to arbitrary images. A
of them do not have proper captions. This paper"proper" description is correct if it makes sense with
addresses the problem of attaching properrespect to the image, and sufficient if it gives enough
explanatory text descriptions to arbitrary images oninformation about its contents. Rather than designing a
the Web. To this end, we introduce Phetch, ancomputer vision algorithm that generates natural
enjoyable computer game that collectsexplanatorylanguage descriptions for arbitrary images (a feat still
descriptions of images. People play the game becausefar from attainable), we opt for harnessing humans. It is
it is fun, and as a side effect of game play wecollectcommon knowledge that humans have little difficulty in
valuable information. Given any image from the Worlddescribing the contents of images, although typically
Wide Web, Phetch can output a correct annotationforthey do not find this task particularly engaging. On the
it. The collected data can be applied towardsother hand, many people would spend a considerable
significantly improving Web accessibility. In additionamount of time involved in an activity they consider
toimproving accessibility, Phetch is an example of a"fun." Thus, like the ESP Game, we achieve our goal
new class of games that provide entertainment inby working around the problem, and creating a fun
exchange forhuman processing power. In essence, wegame that produces the data we aim to collect.
solve a typical computer vision problem with HCI toolsWe therefore introduce Phetch, a game which, as a
alone.side effect, generates explanatory sentences for
The Web is not built for the blind. Only a small fractionrandomly chosen images. As with the ESP Game, we
of major corporate websites are fully accessible toshow that if our game is played as much as other
the disabled, let alone those of smaller organizations orpopular online games, we can assign captions to all
individuals . However, millions of blind people surf theimages on the Web in a matter of months. Using the
Web every day, and Internet use by those withoutput of the game, we mention how to build a system
disabilities grows at twice the rate of the non-disabled .to improve the accessibility of the Web.
One of the major accessibility problems is the lack ofDesign of a Useful Game
descriptive captions for images. Visually impairedA traditional algorithm is a series of steps that may be
individuals commonly surf the Web using "screentaken to solve a problem. We consider Phetch as a
readers," programs that convert the text of akind of algorithm. Analogous to one, Phetch has
webpage into synthesized speech. Although screenwell-defined input and output: an arbitrary image from
readers are helpful, they cannot determine thethe Web and its proper description, respectively.
contents of images on the Web that do not haveBecause it is designed as a game, Phetch needs to be
descriptive captions. Unfortunately the vast majority ofproven enjoyable. We do so by showing usage
images are not accompanied by proper captions andstatistics of a oneweek trial period. Because it is
therefore are inaccessible to the blind (as we showdesigned to collect a specific kind of data, Phetch's
below, less than 25% of the images on the Web haveoutput needs to be proven both correct and sufficient.
an HTML ALT caption). Today, it is the responsibility ofWe prove this through a specifically designed
Web designers to caption images. We want to takeexperiment.