Archive notice: This article was originally published on July 10, 2009. Links and embedded videos are preserved as part of the historical record.
“microsia - the sound of a cell”
by Andreas Illiger
“microsia” is a journey into the sound world of the microcosm. It is a computer game without an objective that turns the player into a researcher who dives into an imaginary plant cell to explore the audiovisual behaviour of its individual particles. Once the player has discovered how each particle works, they can experiment with them and repeatedly discover and experience new audiovisual compositions through their interaction.
Music in Every Cell
The bizarre world of the microcosm is probably familiar to everyone - mites that look like gigantic monsters, electrons orbiting an atomic nucleus, or paramecia dividing merrily. But what does the microcosm actually sound like? The interactive microsia application attempts to explore this.
Andreas Illiger created the microsia project as a semester project during his communication-design studies. The assignment was to create a work on “visual music” using the new possibilities of digital technology. The idea of creating an interactive sound game rather than merely a fixed animated film emerged quickly. Illiger aimed to design an application that went beyond a simple experiment and allowed players to discover new sound structures repeatedly.
Instinct to Play
The first task was to find and test various game concepts. Of 14 ideas for different game mechanics, the most promising were programmed as simple prototypes to test how they worked. The Processing programming language was chosen for the prototypes, a Java-based language developed for artists and designers that allowed interactive sketches to be created relatively quickly. Although the graphic design was reduced, testing the interaction concepts was important.
One prototype depicted movable particles that emit a kind of radar. As soon as the radar from one particle hits another, it contracts again and a tone sounds. The tone continues until the radar has fully contracted. Despite its reduced graphics, this prototype had a very organic quality and was reminiscent of microscopic organisms. The game’s subject - sonic exploration of a plant cell - was therefore quickly established. Four of these game-mechanics prototypes, including the “radar particle” described above, entered the final concept. The virtual cell contained four different types of particle, each with a different function and sound. Some particles therefore generate rhythmic sound structures, while others produce melodic or ambient ones.
Graphic Implementation
Once the game mechanics had been devised, work began on the graphic design. Initial sketches were created in Photoshop from a wide variety of references, including images of the microcosm. The game’s final graphics combine edited microscopic photographs with drawn images created in Photoshop using a Wacom Intuos graphics tablet. The design came purely from the imagination because, naturally, no plant cell looks like the microsia cosmos. The aim was not to create a realistic representation, but to transport players into an imaginary world and tell a story. The game’s graphic elements were saved as individual images known as sprites. During the game’s subsequent programming, the program code defined how and where these sprites, such as the image of a cell particle, were displayed on screen.
Programming
Once the graphic design had been established and all the game’s sprites had been drawn, programming could begin. This was carried out entirely in C++ using the DirectX interface. DirectX is an interface developed by Microsoft to enable direct access to hardware in Windows. Graphically demanding programs and games are possible only through direct communication with components such as the graphics card.
The first programming step was to find a way to generate the sound produced by the cell particles. Because the particles’ sounds were intended not merely to remain fixed but also to change, ready-made samples could not be used. The sounds were therefore generated in real time, requiring a suitable form of sound synthesis to be found and programmed. FM synthesis (frequency modulation) was chosen. This form of synthesis could be implemented quickly yet enabled a wealth of different sounds. The simplified basic idea of FM synthesis is to use carrier signals such as sine oscillators and modulate their frequencies with separate modulators. This produces overtones that generate the sound. Depending on how intensely the oscillators modulate one another, different overtones and therefore different timbres are created.
After the FM synthesiser - the heart of microsia - had been programmed, work began on programming the individual cell particles. During implementation, the graphics of the individual sprites were repeatedly refined because all the graphic elements were now interacting for the first time. The aim was to create a harmonious overall image, so the individual elements repeatedly had to be adapted to one another. Some elements of the graphic design did not consist of ready-made images but were generated in real time and were therefore based on formulas, such as the “radar” of the “radar particles” or the “tentacles” of the cell nucleus. This had the advantage that the shape of such an element could change, something impossible with a fixed image.
Thrown Effects
At this stage, the microsia application was already playable. Experimentation led to the further idea of integrating a second type of cell particle into the game. This introduced the “modular particles,” which produce no sound themselves but only modulate that of other sound particles. The modular particles make a sound particle harsher or more metallic, or add noise or an echo effect. The intensity of the effect depends on the distance from the respective sound particle. The modular particles therefore greatly expand microsia’s musical possibilities. They can be “thrown” through the cell so that they move independently and rhythmically modulate the sound particles. In doing so, the modular particles simply influence individual sound parameters of the FM synthesiser.
Final Phase
Once the inside of the cell had been programmed, the surrounding elements were refined. Save and recording functions and a help tool were added. Once the opening menu had been programmed and the microsia logo created, the game was complete.
Because microsia generates visuals simultaneously with the music, the game is also well suited to concerts. A second plant cell from a different leaf was created for this purpose. It contains the same cell particles but generates different sounds. In the game’s current form, players can choose between two plant cells. Because the FM synthesiser has the potential to produce different sounds, further cells may be added to microsia in future. The shareware version can be downloaded at http://www.microsia.de. by Andreas Illiger
