This is a bidirectional art and music generator where drawing creates sound and sound creates visuals.
What is Synesthesia?
Synesthesia is a neurological phenomenon where stimulation of one sense triggers experiences in another sense. Some people naturally "see" colors when they hear music, or "hear" sounds when they see colors.
How to Use:
Draw to Create Sound: Move your mouse/finger across the canvas to draw. Different colors create different pitches, speed affects tempo, and position affects volume.
Sound to Visual: Enable MIDI or microphone input to generate visuals from sound. Play music and watch it paint on the canvas!
Choose a Mode: Try different synesthesia mappings (Classical, Kandinsky, Spectrum, Particle) for unique experiences.
Record & Export: Capture your audiovisual performances as video, audio, or image sequences.
Keyboard Shortcuts:
Space - Clear canvas
Ctrl+Z - Undo
Ctrl+Y - Redo
R - Start/Stop recording
M - Toggle microphone
Tab - Toggle controls panel
1-4 - Switch synesthesia modes
Tips for Best Results:
Start with the Classical mode to understand the basic pitch-color relationship
Try drawing slowly vs. quickly to hear how speed affects the sound
Use the Particle mode for explosive visual effects
Enable MIDI and play your favorite instrument to see music visualized
Experiment with different synthesis types for varied timbres