PandoraBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitants at DigiHuman Lab Rutgers University

PandoraBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitants is a site-specific installation that uses computer vision and interactive software to track the music choices made by local feeder birds. The project is in collaboration with the computer scientist Dr. Ahmed Elgammal and the The Art and Artificial Intelligence Laboratory at Rutgers University, which is a platform for the artistic uses of computer vision and machine learning.

Aims and Objectives
Our current project involves creating one outdoor, sound-emitting, interactive sculpture. This sculpture or “listening station” plays music in a dedicated genre: classical, country or rock. Lifted into the air on a 10ft. post, the stations will features a directional sound cone in the form of a hood (PandoraBird design, figure 2). This structure covers an external speaker and a customized mp3 player running a music application that has been specifically created for wild birds. This audio hood is mounted over a tray of wild bird food. A small camera, mounted at the tray level uses computer vision to quantify the number and species of birds feeding during each musical interlude. A solar panel and rechargeable battery pack provid each listening station a contained, renewable power source. The PandoraBird project is a mobile, self-sustaining learning system that can be exhibited at museums, sculpture parks and other outdoor venues that may be visited by wild feeder birds. The computer interfaces with the iPod app, effectively allowing each avian species to identify which tunes it prefers in a given genre, and to build a species-specific database of favored music. The station invites viewer participation on the ground and has a webcam that allows humans (all over the world) to watch and listen to the birds’ musical choices.

Background
In 2007, Demaray collaborated with the videographer James Walsh on Listening Stations for Birds, That Play Human Music (art images, figure 1) Created for the woods of the Abington Art Center in Pennsylvania, a forested sculpture garden that is surrounded by the suburbs of Philadelphia, the piece considered the way that life forms, human and otherwise, may interact in a shared environment. Set along a secluded nature trail, this installation was comprised of a series of four sound-emitting sculptures, each playing a loop of either classical, jazz, country, or rock music and offering a tray of wild bird food. While this early work aimed to see what kind of music birds might like, its primary motive was to get viewers out into the woods, to interact with other species and to consider the impact of our presence on other life forms. Observations on the musical preferences of the feeding birds were noted by park visitors on worksheets. While the piece was popular with park visiter and local birds alike, data collection proved inconclusive, largely due to issues with bird observation and identification.

For Demaray and Walsh, the most difficult part of this ambitious installation was however having to choose the four or five tunes that were played at each listening station. How does one select what songs a bird might like best? The team ended up choosing works that they felt might be considered human masterpieces in each category. They concluded that if they were going to give a gift to these other life forms—if this was in fact an act of trans-species giving—it should be the best that humans have to offer.

With advents in the field of computer vision, The PandoraBird Project is now be able to create an interactive system that may actually be able to identify which specific tunes individual birds prefer. Elgammal’s work group in the Department of Computer Science specializes in using computer vision for fine-grained recognition, which is the problem of recognizing subordinate categories. In a study titled Write A Classifier: Learning Fine-Grained Visual Classifiers from Text and Images (NSF #1409683), his group is investigating algorithms for automatically recognizing localized body parts. This study supports the PandoraBird system by creating algorithms that automatically recognize bird species from images based on text descriptions of these species

Rationale
There is ample circumstantial evidence that many avian species pay attention to human sound. In the US Mid-Atlantic Region, cat birds and mockingbirds replicate noises made by people. In Australia, the lyre bird even learns human tunes and teaches them to successive generations of its young. Utilizing a computer vision system, PandoraBird may allow us to better understand, and ultimately communicate with, the other life forms in our shared environment.

The PandorBird Project Design, figure 2

Collaborative Design
The offerings in each genre of music are chosen using standard criteria from web-based “music-discovery services,” using melody, harmony, rhythm, form, and composition. The system will initially begin with a small database of different musical compositions in its defined genre. When a bird feeds during one of these melodies, PandoraBird uses computer vision to record its species and length of stay. If the feeding continues to the end of a piece of music, the system will select another melody with similar qualities. The presence of an individual bird at the feeder is logged by the listening station as a “thumbs up” or “thumbs down”, for the piece of music currently playing and this feedback refine the system’s playlist.

The Significance of the Project in Its Field
PandoraBird may be the first example of a computer vision system dedicated to identifying the musical choices of feeder birds. The novel algorithm for species identification and interactive system that the project represents may be used for a wide range of future purposes. We plant to share the data base of preferred human songs in real-time during the project’s installation. In a more immediate context however, the authors of this project maintain that if we are to bombard other life forms with noise, we should begin to consider which types of noise our companion species might prefer.

Implications for Future Project Design Collaborations
Pandora Radio for Birds may be the first project that utilizes new technology to identify which specific tunes individual birds may like. Using this system as a pilot project, we may ultimately begin to create an interactive system that allows birds to make human-type music choices themselves.