REZ is for me one of the greatest games I have ever played. What makes him unique for me is that it’s hard to explain how you feel when you play it. The words sound hollow: « It’s a TPS where you fire enemies straight out of a cyberspace matrix à la Gibson ». If you see videos, they give the impression of watching an old dated game. Clearly, what makes this game unique is the clever mix of interactivity, music and visuals. And it is only by playing it that the whole thing makes sense. Each element taken separately clearly does not pay tribute to the whole.

Passionate about DJing, game design and addicting VR, I naturally wanted to look into the subject. For me, there are 3 essential elements to make a musical RV game:
– the flow
– the rise
– the VR special: kinesthesia

The flow

The flow theory is immensely documented, I prefer to refer to these articles if you need an introduction (take this as a starting point: php). This notion of flow is essential for music games. The goal is to maximize sensory immersion and to move towards a full awareness of the present moment. If the player is totally focused because the game requires it, he will then live his experience at 100% without feeling forced to.

The rise in intensity

The music follows a rhythm. Not only at the level of a bar, but on the structure of the track. There are powerful phases, calm phases, rise in intensity. Much of the art of DJing is about manipulating the audience to create a veritable explosion of joy.
I remember one of the most memorable DJ sets I’ve ever seen. It was Amon Tobin in Paris at the Rex Club in 2002. In general, all the other DJs arrived at the turntables at 1 o’ clock. By the time the audience got warmed up, they were usually dropping the big drum & bass around 2 o’ clock. Amon Tobin was then one of the biggest DJs of the D&Bass scene. He arrived at the turntables, late, 1:30. He started his DJ set with a surprisingly atmospheric and calm D&B. One hour goes by, always calm, the heads in the audience nod, wondering a little bit where it was going and especially when it was going to take off. Where usually the party is in full swing, he starts to play more energetic tracks, but every time a track starts to come up, he cuts it off, putting the start of another track, a calm one. The audience was going crazy. I’ve never seen that before. People were getting tense, whistling when he was cutting tracks. 30 minutes goes by again, 3:00 in the morning, still nothing. But the tension was rising. Suddenly, at 3:30, Amon Tobin launches a track with a huge rise, the audience is at the edge of its patience. But this time, he chained, tracks upon tracks, with huge beats that sets fire to an overpowered dancefloor. I’ve never seen a DJ set creating such an explosion in my whole life.
What we need to understand from this experience is that the way the song evolves is crucial to providing an experience that takes the player along. It is not enough to take a track and launch it into the system with a little beat detection. You have to craft a real visual, auditory and sensory experience that follows a path and brings the player along.
The procedural approach can give at best an initial basis of work but cannot give a functional experience, it is only able to counterpoint the musical analysis. It deprives it of all the finesse that creation can bring: to furnish a moment of calm with stunning visuals, to pass in a psychedelic mode to black because a part of the track is darker.
The musical experience must be composed with visuals and interactivity in mind to be great. For me, it’s like hoping that by asking a drummer, a guitarist and a singer to blindly compose each on their own, you can hope to assemble their work and have a song worthy of interest.

The kinesthesia

It is really necessary to work on the harmony between movement and music, and the intensity of the movements must be in accordance with the different moments of the tracks. To do this, I think it is essential to work with dancers and/or choreographers.
As far as movements are concerned, we need different systems:
Movement patterns: the brain likes patterns (if you are not convinced, read this: To learn a song, you must be able to do progress by learning the patterns of the track. The advantage of movement is that synchronized movements, symmetrical movements or alternating movements can occur over time. This allows you to have a palette for creating quite rich sequences. If the flow of required movements are graceful, you can create real choreographies.
Fluid movements: in order to develop a kinetic grace, one should not limit oneself to movements up/down/left/right/left/right. It is necessary to be able to make more fluid movements (S-shaped, infinite shaped…) to make ample and graceful movements. It is important that the system should not be too binary, but should allow you to feel and absorb the music.
Free moments for improvisation: the game must let people move their bodies in addition to music. The game should not require you to rigidly follow positions. Also, some chaotic moments can create a little frenzy: imagine a moment with a boss where you have to smash your arms like a death metal percussionist at 150BPM.

In conclusion

These are for the few guidelines. I tried to make a quick prototype to test some of these ideas that you can see in a video here. It seems to me that VR clearly makes it possible to pass a milestone in terms of musical experimentation. It can be noted that Rez has developed an experience around this incredible experience potential, the REZ Synesthesia Suite.

I must admit that one day I would love to push the idea of a VR musical game concept. You never know, if a publisher hears me, I can prepare a beautiful prototype! 🙂

For you, which musical games are most important to you? Thumber, DJ Hero, Guitar Hero, Audioshield?

Leave a comment

Realityz © 2017. All Rights Reserved. Payoff Technologies © 2017

26 rue Louis Guérin, Villeurbanne, France, 69100