codeconstruct

While the first article about the creation of the mcgarrybowenoke holiday card focused on how the control files for the site were generated, this article focuses on how we made the site animate faces and play sounds.

The first step in creating the animations was to photograph as many mcgarrybowen employees as we could. Setting up a photo shoot in one of our New York lobbies, we were able to captures videos of roughly 200 employees (and a few dogs) from which we captured frames from singing, blinking, and the occasional random idle event.

The singing and blinking images were added to the main image to create a three panel image that sat within a frame and simply moved left and right to make a person blink or sing. The idle animations worked similarly but were created as separate files that didn’t have to load until called for.

In a similar way, the individual notes were loaded with the page, but the song files—by far the largest files that the page would have to load—weren’t loaded until they needed to be played. Although each song was created from a MIDI file, all of the sounds played on the page—songs and individual notes—were MP3 files.

To simplify the task of adding sound to our page, we used SoundManager2, a JavaScript sound library that uses HTML5 or Flash to play sounds, depending on the capabilities of a user’s browser. Our hope was that the site experience would be similar on a desktop computer and an iPad, but several other limitations led us to rethink the iPad experience.

Unfortunately, the iPad’s limitations went beyond simply not having Flash. In addition, iPad’s version of Safari doesn’t allow websites to preload audio files or play more than one sound file at a time. Even loaded sound files unload when a new file plays. As a result, the desktop version of the site played on an iPad would featured a laggy piano, songs that would stop if a face was clicked, and occasional loading errors. To deal with these issues, we disabled these associated features, accepted that there would be some lag when clicking on a face, and rewrote some of our code to load the songs differently.

With these changes and a few minor tweaks for different devices, we were able to support acceptable experiences not only iPads and iPhones but also Android tablets, Android phones, and all major browsers at least as far back as Internet Explorer 7.

The idea for the installation was inspired by the famous scene from <em>Big</em> in which Tom Hanks and Robert Loggia play songs on FAO Schwarz’s floor piano.  The first step was to order a large toy piano—we used the “HSC SLW988 Super Gigantic Keyboard Playmat.”

Once the piano arrived, we removed its speakers and electronics, finding that the wires from the keys ended in female headers with 0.1″ spacing.  A few pieces of double-length male header let us connect these headers to a breadboard, and a handful of ~200K resistors let us set up the digital pins of an Arduino to read the switches.

By grounding a couple of points on an Arduino Uno, we were able to use DFU programming mode to update the ATMega8U2 chip with firmware to emulate a keyboard.  With a sketch to send the appropriate keyboard presses for each key, our giant floor keyboard was able to interact with our site just as a regular computer keyboard would.

The resulting piece was installed in the lobby of our New York office the afternoon before our holiday party.