all projects

Cacophonia - Experimental music game using Kinect

Cacophonia from Martin von Lupin on Vimeo.


The goal of the course was to produce a game prototype. There were nearly no restrictions about the game genre or concept. Just the controller was supposed to be neither keyboard nor mouse and we shall not make an ego shooter game.


The idea

Our idea was to develop a game, where you are producing music while playing. The music shall be a reflection of the way of playing.

We used a pretty simple concept for our game and focussed on the interaction and the way of producing music.

So the goal of the game is very easy. It's about sky falling balls which have to be brought into a special area instead of dropping down. You can do so when fixing a pad at a smart position, so the balls are dropping in the right direction.

The music

We wanted to implement a pattern for triggering music. At first we tried to play a tone when a ball hits a pad. The tone shall depend on the y-value. Later we bound an specific audio sample to a certain area of the screen (a graphical cloud). This systems seems to be more intuitive and understandable for the player, as later user tests showed. The audio samples in the game were all self produced.

The controller

For our purpose we needed a controller which is very precise. We decided to use the Kinect, since the exact placing like you have it when using a mouse seems to be most reachable when using your own body.

The background story

We wanted to build a small background story. We decided to use a natural surrounding as metaphor. So the falling balls are seeds falling from the sky and the save area is ground. So when you bring a seed into the ground, a flower is growing.

Early interactive prototype

Cacophonia – early Hi-Fi-Prototype from Martin von Lupin on Vimeo.

This video shows an early prototype. You already get the idea of bringing the balls into a certain area while placing pads. But this prototype mostly focussed on technical issues. Notice the sound pattern. The tone high depends on the y-value of the pad. Later we changed the pattern, since this way leads to too many limitations. Pay attention to the interaction for fixing a pad as well. It's when the user rises one hand into a area at the top. But we realized this move is very unhandy, especially when the other hand is at the bottom.

User tests and issues

Screenshot of the first user test. It's obvious that the interaction was'n comfortable since way too much pads were produced by accident. The music wasn't bound to obstacles yet, so the sound that was playing was realized by the players, but the system behind was mostly not clear. The concept of saving balls by bringing them to the end zone by placing pads was clear after some tries, even without any explanation by us.

The Results from user tests

It was very challenging to combine gaming and producing music. That's what we realized in several user tests. Most users didn't realize any system behind the music they were producing or they didn't care, since they didn't get any benefit (besides the audio benefit) for triggering a sound. They just wanted to finish the game as fast as possible without any seeds falling down. The risk of loosing a ball when taking a harder way just for making a more complex sound didn't make sense at all.

Score system

So we decided to implement a score system which encourage the user to trigger sound samples. We went back from the idea, that the event for triggering a sound is when a ball hits a pad, because the music feedback was hard to communicate.

So we decided to make some areas in the world to sound areas and when the ball passes those, a visual and aural feedback shall occur as well as an increase of the score. Since we wanted to have a small background story we choose the cloud-metaphor for those areas.

Pictures from our three user tests. The test were fundamental for our progress. The feedback helped us in finding a intuitive interaction and a game concept, that doesn't need any explanation or introduction to be able to play.

The interaction

We tried several interactions like placing a pad always at the location of one hand at the beginning. As interaction to fix a pad we tried to use a rising arm as a trigger or a handclap. Actually we decided pretty late to give the opportunity to rotate a pad. We tried to rotate by the distance of the hands and some other parameters as well. But in the end the most intuitive interaction seems to be the placing between both hands the push forward of those when wish to fix a pad. This follows the idea of the real world of holding a stick in your hands and give it away or lay it down.

The final prototype

The video (see top of the page) shows our final prototype. This prototype loops through the same level again and again. Actually when finishing a level you are supposed to reach the next one, which looks of course different. The amount of seeds you have at the beginning are those you saved at the last one. So the goal of the game is to come as far as you can and – as a benefit – while playing you produce an unique instrumental depending on your game strategy.

It's still a prototype

There are still several issues which need some attention. For example when playing the game the first time, the interaction seems a bit hard to handle. Most people don't realize the border of triggering the fixing event when pushing the hands forward. Actually this border is shown by the fill of circles which are located on each hand. When the opacity of the fill becomes 100% a pad is gonna be fixed. Most users didn't even realized the change of the opacity. A better way would be a visual feedback directly from the pad between your hands since this is the point the user is focussing the most.

And of course there a several minor bugs. For example sometimes in the end zone a strange bug occurs, which let the seeds bounce on an invisible obstacle.


The extensive lectures during the course gave a very holistic and detailed view over all types and genres of games. Neither am I a gamer nor have I ever seriously produced a game, so the tasks at the beginning like the scrabbleship challenge, were we had to combine two board games into one or the game analysis, were we had to play a console game for 10 hours and then analyze it, were a very good jump into the world of game design for me.

The course literature »Game Design Workshop« by Tracy Fullerton was a very handy reference during the final project, especially for the user tests.

The final project was very intense and combined subjects about different aspects like game concepting, music and audio production, graphic design or programming.

The combination of playing and music production is very challenging and lead us through a lot of iteration of our game concept. The most impressive part for me was the design of the interaction in combination with actual user tests.

All in all the course gave me a very good overview what it takes to design a game. The course was very instructive, motivating and fun.

This game is the outcome of the course Media Design 3: Speldesign by Marie Ehrndal and Simon Niedenthal at Malmö University, Sweden in Spring 2013. I worked together with Cathrin Malmqvist, Karin Danielsson, Mattias Veileborg and Björn Mauritz.

Impressum // © 2020 - Martin von Lupin