The Strong Museum of Play - Bubble Tower

Executive Summary

Bubble Tower is a large-scale interactive installation built for The Strong National Museum of Play. Using a Kinect v2 depth camera and skeleton tracking, visitors see themselves extracted into a whimsical papercut-style animated world spanning approximately 15 vertically stacked displays.

The experience supports up to six visitors at a time, allowing kids to interact with floating bubbles through full-body gestures—popping bubbles, collecting them, pushing them around, and triggering special effects like tornado swirls. A signature interaction allowed visitors to clap above their head to capture a photo that floats upward inside a bubble.

I developed the complete real-time tracking and interaction system in openFrameworks, including gesture detection, bubble physics behaviors, and the compositing pipeline that blended depth-camera silhouettes into the illustrated environment.

Exhibit Concept

The goal of the installation was to create a playful “magic mirror” experience: visitors become part of a storybook-like animated scene, where their motion directly affects the environment.

Rather than presenting a screen-based game, Bubble Tower was designed as a physical spectacle—an interactive tower of motion and animation that could be understood instantly by kids and crowds.

System Overview

Kinect v2 used for skeleton tracking + depth extraction
Real-time visitor silhouettes composited into a 2D illustrated world
One PC drove the full display tower as a single mapped canvas
15-screen output treated as one continuous vertical animation space

Key Engineering Challenges Solved

1. Real-Time Depth Compositing With Clean Silhouette Output

Depth cameras produce noisy edges and flickering artifacts, especially in public environments. I implemented filtering and edge-softening techniques to stabilize the extracted visitor silhouette so it felt visually integrated with the illustrated world rather than looking like raw camera footage.

2. Gesture-Driven Interaction Design (Multi-Visitor)

The installation supported multiple visitors simultaneously and recognized distinct gestures to drive different interactions, including:

clap above head → capture photo bubble
arms extended → collect bubbles onto hands
poke bubbles → pop interaction
overhead swing → trigger tornado swirl
general motion → push bubbles dynamically

This required reliable gesture detection using skeleton tracking while keeping the experience responsive and forgiving for kids.

3. Large-Format Display Mapping + Performance Stability

The entire tower was rendered as one continuous experience across 15 displays driven by a single machine. The system required stable frame rate performance and careful layout/mapping to ensure animations felt cohesive at architectural scale.

Collaboration Delivery Context

Unified Field was brought in as a technical development partner through Northern Light Productions. I worked directly with their project management team, participated in museum testing sessions, and supported onsite calibration and integration to ensure the installation performed reliably in a real visitor environment.

Outcome

Bubble Tower became a high-visibility interactive centerpiece at The Strong Museum, combining large-format spectacle with intuitive gesture-based play. The final experience delivered:

clean real-time visitor integration into the environment
responsive multi-user interaction
a playful, instantly understandable gesture vocabulary
stable performance across a tall multi-display installation

The project won Gold at the New York Design Awards (Digital – Entertainment & Leisure).

Skills Demonstrated

Kinect v2 skeleton tracking + depth processing
openFrameworks (C++) real-time interactive development
gesture recognition design for public installations
real-time compositing + silhouette filtering
large-format multi-display rendering and mapping
physics-driven interactive object systems (bubble behaviors)
onsite calibration and museum deployment support