The project ‘Harmonic Paper’ is a stand-alone interactive interface that helps users explore the relationship between music and drawing. By means of a microcontroller, MIDI codec IC and series of photocells, the interface detects the thickness and location of drawing in a letter-size paper(8.5 in x 10.5 in), and converts them into music note and sound volume. As shown in Figure 1, a user can insert a piece of drawing on the top of the interface(1) and crank the handle(2) in order to roll down the paper. While the paper passes the sensor box(3) that includes 15 photocells, it generates the sounds on real-time according to ink’s location and thickness with color LED light feedback.

The color of the lights are mapped by the order of corresponding sound frequency(i.e. C, D, E, F, G, A, B convert to Red, Orange, Yellow, Green, Blue, Dark Blue, Violet). The software interface is also provided in which users can virtually practice the functions of this hardware. Such combination of software and hardware allows users to create both musically-composed drawings and visually-created music.

** Publication:
- Kang, Laewoo, Tianyuan Gu, and Geri Gay. Harmonic paper: interactive music interface for drawing (CHI 13).
doi : https://doi.org/10.1145/2468356.2468492 / download [pdf]

- Kang, Laewoo, and Hsin-Yi Chien. "He(和): Calligraphy as a Musical Interface.", Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia download [pdf]

Harmonic Paper by Laewoo(Leo) Kang, Tianyuan Gu, Geri Gay (2013)

和 (He) by Leo(Laewoo) and Amy Chein (2010)