Select Page
Research

Generative Impressionism

An Interactive Video Wall Installation

Tags

Generative Art, Interactive, GLSL, TouchDesigner, Kinect

Year
2018

Demo of Interaction with Artwork

The Challenge

While continuing my experiments with GLSL, the challenge was to transform static works of art into living, interactive experiences. Inspired by the texture of Pointillism, the goal was to deconstruct a painting and have it “repaint” itself in a fluid, impressionistic style, reacting dynamically to the presence and movement of a viewer in front of a video wall display.

My Solution & Approach

I developed an interactive installation using TouchDesigner and a Kinect sensor. The core of the experience is a multi-pass GLSL system based on a 2D fluid simulation. The first shader pass generates a dynamic vector field (a flow map) that directs the “brushstrokes.” The second pass uses this evolving vector field to “paint” the source image by directionally smearing its pixels in a process known as advection.

The program was designed for a multi panel video wall. The Kinect tracks a viewer’s position, triggering the generative effect on the LED panel when they stand in front of it, creating a responsive connection between the viewer and the art.

Examples of Generative Effect on Pointillism Art

Technical Highlights & Learnings

  1. Kinect-Based Interactivity: The installation uses a Kinect camera to track a viewer’s position and movement, allowing them to directly interact with the artwork being diplayed.
  2. Vector Field Simulation: A GLSL shader generates a dynamic 2D vector field to direct the “flow” of pixels, based on fluid dynamics principles like curl and divergence.
  3. Multi-Pass Shader Architecture: A multi-pass rendering pipeline where separate GLSL shaders handle distinct tasks: vector field generation, pixel advection, and final color grading.
  4. Context-Aware Video Wall: The system was designed for a multi-screen display, with interactive zones that activate the generative effect on specific artworks as a viewer approaches them.

System Diagram Illustrating Kinect Sensor Detection Zones

Demo of Generative Effect

Tools & Technologies Used

Software: TouchDesigner
Programming: GLSL (OpenGL Shading Language)
Hardware: Microsoft Kinect, Multi-screen Video Wall