Turning plants into active participants in their own representation

Initial Exploration

The idea grew from a question: What if plants could represent themselves digitally? Rather than designing visuals about plants, we wanted their own biological data to become the creative material. This conceptual shift framed the plants not as subjects but as collaborators.

Choosing the Right Signals

After experimenting with different types of sensors, we focused on two inputs:

• Soil moisture – a slow but steady indicator of a plant's well-being. It became the main driver of colour palettes and textures.
• Movement (accelerometer) – a more dynamic signal, sensitive to subtle vibrations or external interactions. It shaped the distortion and rhythm of visuals and sounds.

This combination gave us both a baseline rhythm (moisture) and moments of surprise (movement), resulting in an installation that felt balanced and alive.

Hardware Layer

Each plant was connected to an Arduino Nano ESP32, chosen for its compact form, Wi-Fi capability, and ability to handle real-time sensor input.

• The soil moisture sensor tracked hydration, generating a range of values that could be mapped to visual and sonic parameters.
• The accelerometer detected micro-movements, often caused by human interaction, air currents, or the plant's own subtle shifts.

Database Layer

A Supabase database functioned as the central hub. It received JSON-formatted data from the Arduino at regular intervals, ensuring that no signal was lost. The database created a reliable bridge between the raw sensor readings and the responsive frontend.

Frontend Layer

A Raspberry Pi acted as the installation's display engine. It continuously queried Supabase for the most recent values and used them to update the visuals and sound in real time. This three-layer architecture (Arduino → Supabase → Raspberry Pi) ensured a smooth flow from biology to digital experience.

The Raspberry Pi ran a web-based interface built with HTML, CSS, and JavaScript.

• A curated archive of visuals represented different states of moisture and movement. The system dynamically selected images based on incoming values.
• To avoid abrupt changes, a fade-in/fade-out transition was applied, creating an organic sense of continuity.
• Movement data introduced distortions and layering effects, making the images feel fluid and responsive.

The result was a display that never repeated exactly the same way twice. Visitors could watch as the plant's current state shaped an evolving digital portrait.

Sound was generated in real time using the Web Audio API. Instead of looping pre-recorded tracks, the system synthesised audio on the spot.

• Moisture values shifted pitch and tonal layers, creating a “mood” that reflected hydration.
• Movement data modulated rhythm, depth, and texture — sudden vibrations could trigger more active sonic patterns.
• Effects such as oscillators, filters, reverb, and delay gave richness and spatial depth to the soundscape.

This approach ensured the audio was not just background ambience but a living composition tied directly to the plant's behaviour.