I am so please to be presenting PandoraBird at The International Symposium of Electronic Arts in Hong Kong this year. This art work is in collaboration with Ahmed Elgammal and  The Art and Artificial Intelligence Laboratory at Rutgers University. The project premiers tomorrow at City University on the ninth floor terrace of Daniel Libeskind’s Run Run Shaw Creative Media Center. If you are in Hong Kong, please join us!

PandoraBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitents, ISEA presentation at the Digital Ethology Panel, CMC 5, City University , Hong Kong, May 7th, 2016

PandorBird: Identifying the Types of Music That May Be Favored by Our Avian Co-Inhabitants is an interactive outdoor installation that uses computer vision and interactive software to track and then play the music choices made by wild song- birds. This mobile learning system uses a novel algorithm for species identification, plays avian-favored human music, and builds a database of the musical compositions preferred by local feeder birds. Offerings in each genre of music will be chosen using standard criteria from web-based “music-discovery services,” such as melody, harmony, rhythm, form, and composition.

Daniel Libeskind’s Run Run Shaw Creative Media Center,at City University in Hong Kong. the site of the 2016 International Symposium of Electronic Arts. The bit of trees that you see on the terrace to the left if the main tower is the installation site for the PandoraBird project.

The current system begins with a small database of different musical compositions in its defined genre. Every time a bird feeds during one of these melodies, PandoraBird will use computer vision to record its species and length of stay. If the feeding continues to the end of a piece of music, the system will select another melody with similar qualities. The presence of an individual bird at any given feeder will be logged by the listening station as a thumbs up or thumbs down, for the piece of music currently playing and this feedback will instantly refine the system’s playlist.

We are using a Raspberry Pi and its camera module to implement this project. The Pi is programmed to search for a network at startup. Once a network is found, it launches a bash script that starts a Python program that plays music as a background process. Currently, the program plays a fixed number of tracks in a set order, but Aditi Devrajan, a programmer at the DigiHuman lab is currently working on how to use the Pandora API to stream music of varying genres. Currently, the bash script launches a second script that initializes the camera module, takes time-lapse photos at predefined intervals (right now the rate is once every three seconds) and writes them to a temporary folder, overwriting the JPEG file each time. The resolution, picture quality, frame rate, and audio/video options can be adjusted according to requirements and network quality. The photo is then transferred via an MJPG streamer to the localhost, and can be viewed through a browser by any computer on the network.

Elizabeth Demaray plugs internal audio system into PandoraBird’s Raspberry Pi.

If no network is found at boot, then the system simply takes time-lapse photos, labels them with the current date to the second, and stores the stills in a local folder for later extraction. All algorithms are implemented to analyze the birds that interact with the system, creating a user account for each species and looping through genres using algorithms to identify birds’ preferences.

While the system is still in it’s infancy, below are two short clips shot during a test run in Philadelphia before unveiling the system at ISEA. I showed these two short videos during the Pandora Bird presentation at ISEA because they both clips documented unusual trends that we are even beginning to see in this early phase of the project. The first clip shows PandoraBird playing Clair de Lune by Debussy. The striking thing about this clip is that there were actually more birds at the Pandora station after the music started playing.

Demaray mounts PandoraBird’s internal camera and audio system.

The second clip that we looked at during the presentation is PandoraBird playing Metallica’s For Whom The Bell Tools. The astounding thing about playing this song for this population of birds which are primarily House Sparrows and House Finches (the smaller one with the red heads), is that there appears to be a difference in the way that the two species react to Metallica. This clip seems to indicate that the House Sparrow appears to dislike Metallica a bit less than their cousins the House Finch. Stay tuned for project updates. When our current species are logged into Pandora Radio as users, we plan to do a a live web feed, so that you the audience my log into the feed and listen to music that is dictated by the preferences of these other life-forms.

PandoraBird, photographed in Carrie Borgenicht garden in Philadelphia, PA on May 12, 2016

In addressing the nature of a biotope—an environment shared by multiple species where human and animal populations overlap—this project maintains that if we are to bombard other life forms with human noise, we should begin to consider which types of noise our companion species might prefer. The ultimate goal of this project is to create multiple listening stations that play different genres of human music so that local feeder birds can make human type musical choices for themselves.