PandoraBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitants at DigiHuman Lab Rutgers University

PandoraBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitants is a site-specific installation that uses computer vision and interactive software to track the music choices made by local feeder birds. The project is in collaboration with the computer scientist Dr. Ahmed Elgammal and the The Art and Artificial Intelligence Laboratory at Rutgers University, which is a platform for the artistic uses of computer vision and machine learning.

Aims and Objectives
Our current project involves creating one outdoor, sound-emitting, interactive sculpture. This sculpture or “listening station” plays music in a dedicated genre: classical, country or rock. Lifted into the air on a 10ft. post, the stations will features a directional sound cone in the form of a hood (PandoraBird design, figure 2). This structure covers an external speaker and a customized mp3 player running a music application that has been specifically created for wild birds. This audio hood is mounted over a tray of wild bird food. A small camera, mounted at the tray level uses computer vision to quantify the number and species of birds feeding during each musical interlude. A solar panel and rechargeable battery pack provid each listening station a contained, renewable power source. The PandoraBird project is a mobile, self-sustaining learning system that can be exhibited at museums, sculpture parks and other outdoor venues that may be visited by wild feeder birds. The computer interfaces with the iPod app, effectively allowing each avian species to identify which tunes it prefers in a given genre, and to build a species-specific database of favored music. The station invites viewer participation on the ground and has a webcam that allows humans (all over the world) to watch and listen to the birds’ musical choices.

Background
In 2007, Demaray collaborated with the videographer James Walsh on Listening Stations for Birds, That Play Human Music (art images, figure 1) Created for the woods of the Abington Art Center in Pennsylvania, a forested sculpture garden that is surrounded by the suburbs of Philadelphia, the piece considered the way that life forms, human and otherwise, may interact in a shared environment. Set along a secluded nature trail, this installation was comprised of a series of four sound-emitting sculptures, each playing a loop of either classical, jazz, country, or rock music and offering a tray of wild bird food. While this early work aimed to see what kind of music birds might like, its primary motive was to get viewers out into the woods, to interact with other species and to consider the impact of our presence on other life forms. Observations on the musical preferences of the feeding birds were noted by park visitors on worksheets. While the piece was popular with park visiter and local birds alike, data collection proved inconclusive, largely due to issues with bird observation and identification.

18_BirdStacolor

Listening Stations for Birds, that play human music, Elizabeth Demaray and James Walsh, Abington Art Center, Jenkintown, PA 2007

For Demaray and Walsh, the most difficult part of this ambitious installation was however having to choose the four or five tunes that were played at each listening station. How does one select what songs a bird might like best? The team ended up choosing works that they felt might be considered human masterpieces in each category. They concluded that if they were going to give a gift to these other life forms—if this was in fact an act of trans-species giving—it should be the best that humans have to offer.

With advents in the field of computer vision, The PandoraBird Project is now be able to create an interactive system that may actually be able to identify which specific tunes individual birds prefer. Elgammal’s work group in the Department of Computer Science specializes in using computer vision for fine-grained recognition, which is the problem of recognizing subordinate categories. In a study titled Write A Classifier: Learning Fine-Grained Visual Classifiers from Text and Images (NSF #1409683), his group is investigating algorithms for automatically recognizing localized body parts. This study supports the PandoraBird system by creating algorithms that automatically recognize bird species from images based on text descriptions of these species

Rationale
There is ample circumstantial evidence that many avian species pay attention to human sound. In the US Mid-Atlantic Region, cat birds and mockingbirds replicate noises made by people. In Australia, the lyre bird even learns human tunes and teaches them to successive generations of its young. Utilizing a computer vision system, PandoraBird may allow us to better understand, and ultimately communicate with, the other life forms in our shared environment.

The PandorBird Project Design, figure 2

PandoraBird

Collaborative Design
The offerings in each genre of music are chosen using standard criteria from web-based “music-discovery services,” using melody, harmony, rhythm, form, and composition. The system will initially begin with a small database of different musical compositions in its defined genre. When a bird feeds during one of these melodies, PandoraBird uses computer vision to record its species and length of stay. If the feeding continues to the end of a piece of music, the system will select another melody with similar qualities. The presence of an individual bird at the feeder is logged by the listening station as a “thumbs up” or “thumbs down”, for the piece of music currently playing and this feedback refine the system’s playlist.

The Significance of the Project in Its Field
PandoraBird may be the first example of a computer vision system dedicated to identifying the musical choices of feeder birds. The novel algorithm for species identification and interactive system that the project represents may be used for a wide range of future purposes. We plant to share the data base of preferred human songs in real-time during the project’s installation. In a more immediate context however, the authors of this project maintain that if we are to bombard other life forms with noise, we should begin to consider which types of noise our companion species might prefer.

Implications for Future Project Design Collaborations
Pandora Radio for Birds may be the first project that utilizes new technology to identify which specific tunes individual birds may like. Using this system as a pilot project, we may ultimately begin to create an interactive system that allows birds to make human-type music choices themselves.

The Songs We Sing Amsterdam at the Lloyd

The Songs We Sing Amsterdam is a site-specific installation that may be experienced at the Lloyd Hotel and Cultural Embassy from May 27th till June 7th, 2013. Part of a multi-city project which is currently on display in Europe, The Songs Cycle piece considers the concept of a biotope – small environments shared by humans and other species.

According to The naturalist E.O. Wilson, half of the species currently living on Earth will become extinct in the next 100 years. In this new world, we may have to create fictive environments in order to experience a sense of calm and beauty – especially in an urban context.

The Songs We Sing, the title of the installation in each city, is about connecting our ecological moment with our cognitive/behavioral needs. It ponders the lack of human companion species in a postindustrial Western landscape, and offers a possible solution.

The Songs We Sing--one installation site at the Lloyd Hotel and Cultural Embassy

The Songs We Sing–one installation site at the Lloyd Hotel and Cultural Embassy

The Songs Cycle project attempts to improve the current lack of animal centric sounds in our auditory experience of the natural by collecting bird calls from human volunteers. In Amsterdam, each installation sits features audio of randomly sampled human bird calls that were collected from project participants between May 5th and May 7th at the Lloyd Hotel and Cultural Embassy. Over the course of the exhibition these sampled sounds will be wirelessly installed at multiple outdoor locations in the vicinity of the Lloyd. The project aims to mix these human generated animal noises with the ambient sound of the local environment. Anybody interested in experiencing the installation need only look for the project sign on the outside of the Lloyd building and listen.

Special thanks to the project participants below:
Hugo Bastidas_Fischreiher,
Andrew Cramer_Oehoe, Theadore Dean_Seidenreiher, Elizabeth Demaray_Rohrdommel, Dylan Koomen_Mallard, Nanny Roed Lauridsen_ Grutto, Abdel Patrijs_Grebe, Ellen Rikkink_Pheasant, Brenda Saunders_Heron Crabier, Eva Schepens_Egrit, Renate Schepen_Roodborstje, Suki Verwiel_Porseleinhoen

For more information on The Songs We Sing and other projects at the Lloyd please see: http://www.lloydhotel.com/en/about-us/cultural-embassy

For information on the The Songs We Sing at DADAPost Berlin see: http://demaray.rutgers.edu/2013/05/04/the-songs-we-sing-berlin/

Lichen for Skyscrapers Project

I am thrilled to report that the “Lichen for Skyscrapers Project” was featured as part of New York’s Art in Odd Places Festival from Oct. 1-10 and is currently on view as a site-specific installation on 14th Street between Union Square Park and the Hudson River. A self-guided walking-tour map and downloadable brochure, detailing the installation on 14th Street, can be found at the end of this post.  A Rutgers Focus news release states:

Lichen is a versatile combination of fungi and algae and can grow vertically on pourous surfaces.

In this era of environmental consciousness, many buildings are being outfitted to “go green.” A Rutgers–Camden professor is taking the term quite literally.

Elizabeth Demaray, an associate professor of fine arts, is cultivating lichen on the sides of New York City skyscrapers to counteract the lack of native vegetation found in the city. Her “Lichen for Skyscrapers Project” was featured as part of New York’s Art in Odd Places Festival from Oct. 1-10 and is currently on view as a site-specific installation on 14th Street between Union Square Park and the Hudson River.

“Metropolitan centers figure into local temperatures in an interesting way,” Demaray says. “They are sometimes referred to as ‘urban heat islands’ because they create heat and they trap heat. A large part of this process is due to the materials that we build with and the actual architecture of the buildings that we create.”

Demaray says one of the ways to reduce heat in these cities is to cultivate lichen, which forms a protective barrier, insulating its supporting building from harmful elements. It can lower cumulative temperatures by absorbing sunlight and reflecting heat due to its light color palate while making oxygen and creating green space on the sides of buildings. 

Scientific American Lichen for Skyscrapers post:

http://blogs.scientificamerican.com/observations/2011/11/29/its-a-bird-its-a-plane-no-its-reindeer-chow-a-n-y-c-artist-uses-lichen-as-paint/

Rutgers Magazine post:

http://news.rutgers.edu/news-releases/2011/10/professor-going-gree-20111021#.UoZDERb3CqQ

Lichen Project Walking Tour