Meandering River is an audiovisual artwork by Berlin-based digital art studio onformative. It is an installation comprised of real-time generated visuals and music composed by an A.I. The digital artwork makes change perceivable by creating a unique awareness of time. Spanning over multiple screens, the piece reinterprets the shifting movements of rivers by visualizing and sonifying their impact on the surface of the earth.
Over time natural forces gradually shape landscapes. Indiscernible to the bare eye, we only perceive one moment at a time. The fluctuations and rhythmic movement of rivers are a glimpse into the past, as traces provide evidence of the constant transformations that surround us. The audio-visual installation creates a bird’s eye view of a landscape. This orientation conceptualizes a new human perspective on space and time, in an attempt to decipher the unpredictable patterns. By experiencing it from an influential perspective, the real-time visuals demonstrate what is happening on a larger scale.
Based on a bespoke algorithm, the rippling and oscillating movement inherent in the generated imagery provides a vantage point that transforms an understanding of progress, to examine the shifting movements and rhythm of natural forces. As the evolving patterns of Meandering River emerge, the viewer is left with a humbling sense of the unpredictability of change, and the beauty of nature.
With time, flowing river structures shift as water erodes the ground, forms a riverbed, and carries soil deposits to other locations. The effect of depositing and eroding changes the landscape’s color formations depending on the shade and quality of the ground.
By investigating academic and scientific research that examine this natural phenomenon, different algorithms were developed to authentically simulate the unpredictable movements of rivers and reinterpret their organic structures, rhythmic fluctuations and visual materiality.
During the visual research and development process, a bespoke rendering pipeline was designed, allowing an iterative visual exploration and composition. Various shaders and algorithms were applied in real-time as graphical layers to simulate natural appearances such as hydraulic and thermal erosion, oxbow lakes and meander scars. Additional post-processing passes like surface materiality, vegetation and lighting enabled the final landscape compositions to reach the visual granularity and quality intended for the work.
The musical score was developed in collaboration with Berlin-based sound studio kling klang klong with the intention to enhance the emotional, complex and unpredictable nature of the visual aesthetic. Through research of various computational strategies, the musical composition was created using the Google Magenta Performance RNN learning model and a custom-built procedural system. To collect training datasets, piano players were invited to improvise musical phrases based on the diverse visual sceneries of the animation.
The range of auditive interpretations they produced, served as a large-scale polyphonic inventory for the Artificial Intelligence to learn from. To reflect the performative nature of the work, values outputted directly from the river simulation such as the length, curvature and events of connections of the river were analyzed and interpreted in real-time to influence musical parameters like tempo, note strength, intensity and pitch range.
As a result of an iterative process guided by kling klang klong’s valuation, the machine achieved the composition of an original musical score whose harmonic quality reached the standard expected from a human composer and a rich cinematic connection between the visuals and the music.
In July 2018 Meandering River was shown in public for the first time and it has been presented internationally since. Further exhibitions are planned in 2019 including Google I/O Festival and UN Art Center in Shanghai.