- Introduction
- Navigating in VR-mony
- Music Theory
- Implementation
- Demo Video
- Useful Links
- Group Members
- License
This project has been developed for the course "Advanced Coding Tools and Methodologies" of the MSc in Music and Acoustic Engineering of the Politecnico di Milano.
VR-mony is a minimal abstract space in which the user can interact with spherical synthetic sound sources.
It is a VR web application with positional audio and multiuser option accessible both from VR headsets and from PCs.
The app can be run either by downloading the whole repository and running it with a live server or by clicking on the following link.
Open VR-mony
Remember to switch on the audio of your device and to wear your headphones.
The best way to appreciate the audio-video spaciality of VR-mony is to access the link from the browser of a 6DoF VR headset.
The homepage is just a joke. If the user knows where to touch, the app will ask him/her/them to play.
There are just two rules:
- No means no.
- Two is love, three is a chord.
The user is suddenly teleported into a minimalistic tridimensional room with just a bunch of organized spheres, three lines and a user interface.
PC users can move into the space, activate sounds and modify them with a mouse:
PAN/TILT : Hold the left mouse button to rotate in 3D.
DOLLY : Hold the mouse wheel to move back and forth.
MOVE : Hold the right mouse button to move horizontally and vertically.
PLAY : Left click on a sphere to activate or deactivate its sound.
VR users will be able to see a green button on the bottom right of the screen if their device is VR-ready and clicking it they can access the VR experience and get out of it.
| Enter VR | Exit VR |
|---|---|
-
A 3 DoF device (e.g. smartphones inside a Google Cardboard) can only track rotational movements. This is common in devices which rely exclusively on accelerometer and gyroscope readings to provide tracking. Therefore, the user can rotate the camera around the 3 axes and click by means of a bluetooth controller.
-
A 6 DoF headset (e.g. Oculus Quest, HTC Vive) can track both rotation and translation, enabling precise 1:1 tracking. Therefore, the user can move in the virtual space and use controllers to interact with objects.
☰ Use the hamburger button on the left controller to exit.
A cubic lattice of 3x3x3 spheres is the core of VR-mony.
The glowing sphere at the center of the cube plays the fundamental note while the spheres around it have different notes depending on the direction they are located with respect to the fundamental one.
Default settings generate a cube with a
- x axis: Fifth
- y axis: Major Third
- z axis: minor Seventh
Three lines with different colours constitute a reference system with the origin in the center of the fundamental note sphere.
Each line is thicker on the side where the interval goes toward higher frequencies and it is thinner going towards lower frequencies.
Line colours match the colours in the GUI.
A simple user interface allows the user to control the sound generator, the intervals on each axis and the arpeggiator mode settings.
-
WAVE FORM: allows to set the wave form of the generated sounds
sine square sawtooth triangle
- FUNDAMENTAL FREQUENCY: allows to set the note of the central sphere
| C | C# | D | D# | E | E# | F | F# | G | G# | A | A# | B |
|---|
-
OCTAVE: allows to set the octave of the fundamental frequency from 1 to 6 where
$C_3$ is the middle$C$ with$f \approx 262.626 Hz$ . -
X-AXIS / Y-AXIS / Z-AXIS: allow to set the development interval on each axis.
| min II | Maj II | min III | Maj III | IV | min V | V | min VI | Maj VI | min VII | Maj VII | VIII |
|---|
The "Arp mode" switch activates the arpeggiator and opens the menu folder containing the related settings.
When the arpeggiator is ON, the mouse click on each sphere starts a looping arpeggio using the note of the clicked sphere as the tonic and it is possible to customize on the go not only the sound generator and the axes intervals, but also the sequence of notes played and its speed with the following options:
- PATTERN: selects the way the arpeggiator picks the notes into an array
| Ascending | Descending | Ascending+Descending |
|---|
- BPM: allows to set the BPM of the arpeggiator from 60 to 1000 bpm.
- STEPS: allows to set the number of steps of the arpeggio from 2 to 6.
Thanks to an external database on Firebase it is possible for 2 or more users to access VR-mony at the same time and see and hear the spheres activated by other players in real-time.
An interval is a difference in pitch between two sounds. It may be described as horizontal (or melodic) if it refers to successively sounding tones, such as two adjacent pitches in a melody, and vertical (or harmonic) if it pertains to simultaneously sounding tones, such as in a chord.
In Western music, intervals are most commonly differences between notes of a diatonic scale and the most common naming scheme for intervals describes two properties of the interval: the quality (perfect, major, minor, augmented, diminished) and number (unison, second, third, etc.).
The size of an interval between two notes may be measured by the ratio of their frequencies. When a musical instrument is tuned using a just intonation tuning system, the size of the main intervals can be expressed by small-integer ratios, such as
Most commonly, however, musical instruments are nowadays tuned using a different tuning system, called 12-tone equal temperament. As a consequence, the size of most equal-tempered intervals cannot be expressed by small-integer ratios, although it is very close to the size of the corresponding just intervals. For instance, an equal-tempered fifth has a frequency ratio of
In musical tuning and harmony, the Tonnetz (German for 'tone network') is a conceptual lattice diagram representing tonal space first described by Leonhard Euler in 1739. This bidimensional representation is useful to illustrate and analyze chords and their relationships. Various visual representations of the Tonnetz can be used to show traditional harmonic relationships in European classical music.
Vogel's Tonnetz is a graphical and mathematical representation of the scale range of just intonation, introduced by German music theorist Martin Vogel 1976 in his book Die Lehre von den Tonbeziehungen (English: On the Relations of Tone, 1993). The graphical representation is based on Euler's Tonnetz, adding a third dimension for just sevenths to the two dimensions for just fifths and just thirds. The four-dimensional mathematical representation including octaves allows the evaluation of the congruency of harmonics of chords depending on the tonal material. It can thus also serve to determine the optimal tonal material for a certain chord.
VR-mony is developed with HTML, CSS and Javascript languages taking advantage of the following libraries and Application Programming Interfaces (API).
-
Web Audio API is a high-level Web API for processing and synthesizing audio in web applications.
-
THREE.js is a cross-browser JavaScript library and API used to create and display animated 3D computer graphics in a web browser using WebGL.
-
WebXR API provides access to input (pose information from headset and controllers) and output (hardware display) capabilities commonly associated with Virtual Reality (VR) and Augmented Reality (AR) devices. It allows you develop and host VR and AR experiences on the web.
- Francesca Del Gaudio (francesca.delgaudio@mail.polimi.it)
- Valerio Maiolo (valerio.maiolo@mail.polimi.it)
Shield:
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.