MARS:

Musical Analysis and Representation System

Research team Ralf Klamma (RWTH) Marc Spaniol (RWTH) Matthias Jarke (RWTH,  FIT) Christoph von Blumenröder (Uni Köln) Marcus Erbe (Uni Köln) Jan Simon Grintsch (Uni Köln)	Figure 1:  Musical Analysis and Representation System (MARS)
Students Krishnamoorthi Vijaysaravanan

1. Introduction
    
   "Electroacoustic music is a generic term describing music that uses electronically generated sound or sound modified by electronic means, which may or may not be accompanied by live voices or musical instruments, and which may be delivered live or through speakers." [Smal93] Electroacoustic music is not a single, straightforward musical genre; it is a collection of genres and styles and has a wide variety of musical praxes [Helm96]. It is created with all kinds of sound syntheses as well as sound transformations, so this type of music has a variation in its reliance upon technological mediation. A certain corpus of electroacoustic music is also called "acousmatic". In a broader sense, acousmatic music may be said to be the music, which is produced especially for electroacoustic music and presented through loudspeakers thereby without visible sources of the constituent sounds.
   
   Electroacoustic music is composed by means of a different technology, now primarily computer-based, is used to access, generate, explore, edit and configure the music files. Electroacoustic music is generated using all kinds of sound sources like human and animal voices, traditional instruments, audio oscillators, audio synthesizers and electro acoustic instruments etc. Electroacoustic music is evolved from compositional techniques and aesthetic approaches [Smal93]. Electroacoustic music mainly takes advantage of recorded sounds that are electronically altered to create sonic collages. The electroacoustic music is recorded by using various types of analogue or digital media such as tape, hard disk etc. and it helps the composers by new means for modifying recorded sounds, including splicing (cutting the tape to create new juxtapositions of sound), speed variation, and mixing (which allows two or more different recordings to be played back at the same time).

   As electroacoustic compositions - in comparison to traditional western art music - share a set of unique features and are therefore barely ascertainable by using historical forms of musical notation [Aust96]. Some problems exist for the electroacoustic music in transcription, visualization and description. Most electroacoustic pieces contain spectral changes and other timbre aspects. It is very difficult to find the source musical component for certain sound events. Each musical component has varying frequency ranges. Thus it is impossible to visualize certain aspects of this music by using Oscillogram (amplitude versus time). Instead of using oscillogram a more accurate representation (especially of the spectral dimension) can be achieved by using sonograms (frequency versus time with amplitude as colour  scale. In order to overcome this obstacle, it is required to annotate electroacoustic pieces by using some graphical symbols instead of using notational symbols of traditional music.

    

2. Problems in electroacoustic music visualization and notation

   Most electroacoustic music contains spectral changes and other timbral aspects that are difficult to notate using traditional musical notation. The sonogram has been explored as a tool for understanding the sonic design of musical works. It displays time on the x-axis and frequency on the y-axis. Pitches occurring for some duration are shown as horizontal lines; bands of frequencies correspond to bands across the sonogram while, gradual timbral shifts appear as moving shapes across the display. Components, which have more energy, appear as darker. The sonogram was considered to have the pitch notation, phase analysis, and amplitude display (color scale level) for an initial visualization of electroacoustic music.

   Since electroacoustic music is a collection of genres and styles that exceed conventional western art music, it is - at least in the majority of cases - impossible to notate its sounds traditionally [GaDe84]. Considering these difficulties, a special kind of notation has to be developed. Thus there is a need to highlight the sources for the purpose of documenting and analyzing electroacoustic music. Electroacoustic music can be annotated by using some graphical symbols, text, and bitmaps, and the annotated information’s are stored in a community repository for sharing of information and also for future reference. Electroacoustic music is annotated in order to provide sharing, documenting of musical information between musicologists or analysts.

    

3. The composer's role in electroacoustic music

   Electroacoustic music radically altered the relation between the composer and the performing musician, whereas the former absorbed the role of the latter. To create sounds for electroacoustic music, the composer becomes the instrument builder. In the electronic studio  environment different aspects of musical performance remain in the hands of the composer, like the articulation of single sound events and rhythms. electroacoustic music balancing and coordinating complex parts of the musical work may compare the composer’s role compared to that of a conductor.

    

4. Visualizing the Electroacoustic Music

   Electroacoustic music radically altered the relation between the composer and the performing musician, whereas the former absorbed the  role of the latter. To create sounds for electroacoustic music, the composer becomes the instrument builder. In the electronic studio environment different aspects of musical performance remain in the hands of the composer, like the articulation of single sound events and  rhythms. electroacoustic music balancing and coordinating complex parts of the musical work may compare the composer’s role compared to that of a conductor.

    

5. Community Support

   To support a community of musicologists spread over several locations they need a tool with a common repository to store the musical  information. In the community repository the musical information can be created, stored or edited by a single user or a group of users. With this design issues there would be an opportunity for the musicologists or analysts to work in an environment where knowledge can be exchanged by means of musical data. This pattern has the opportunity to share information and comments on electroacoustic compositions.

    

6. Musical Analysis and Representation System (MARS)

   The Musical Analysis and Representation System (MARS) is designed for the purpose of documenting, analyzing, searching electroacoustic compositions for the collaboration of researchers in a distributed setting being connected via the Internet. In figure 1 the user interface can be viewed. Here, the user can visualize the electroacoustic music in the sonogram window for each different audio track. The user can freely annotate the electroacoustic music files by using the predefined annotation symbols, like straight line, bend line,  circle, etc. In addition, there are options to zoom, mute, solo etc for each different audio track. By using the setting option, the user can freely change the MARS setting like, changing the color for sonogram, etc. Even more, the user can freely operate the multi track audio  player by using the buttons like, play, pause, stop, loop, etc.

   Most of the electroacoustic music compositions have multi-track audio files, so there is a necessity to have an audio player with multi  track feature. This multi track audio player has to play eight ‘monophonic’ audio files through a special sound card. Due to a java bug  (RFE 4558938) the multi channel sound card is triggered by C++ using the ASIO Driver Interface package. The ASIO Driver package is used,  since this type of driver is internationally recognized. The native library for the ASIO Driver Interface was written in C++ and integrated with the java code by means of JNI (Java Native Interface) technology (ref. figure 2). The ASIO Driver Interface package has a native interface for playing the audio files through the multi channel sound card.

   

   Figure 2: Eight channel sound card control with with java

   MARS users may share their compositions. Hence, they can upload audio files into the MARS system in order to initialize the system and also for effective audio player usage. Therefore, the sonogram has to be computed or downloaded from the FTP server. While the audio files are playing the user can annotate the electroacoustic music, by using some graphical symbols, text, bitmaps etc. The user has the option to loop the playback for a specified annotation window, since using this option the user can effectively annotate the musical information by hearing specifically to the looped window. User can use the mute and solo button to hear music for a specified audio track. These annotated information are stored into the XML database by using the MPEG-7 standard. Annotated bitmaps files are stored into the FTP server, and also these bitmaps files are retrieved from the FTP server during the playback option (cf. figure 3).

   

   Figure 3: Meta data management in MARS

   In figure 4 we can view the user who is sitting in centre of a studio room, where eight audio speakers are present. In the studio room the level of acoustic and other issues are checked in order to maintain a good audio effect for the electroacoustic music playback. The user can view the sonogram for the different track and annotate the electroacoustic music files. The annotated information can be simultaneously stored and retrieved from the XML database. The user has the option to create new annotation, delete or modifying the existing annotation. This allows users to discuss compositions in a distributed setting and keeps a discourse going on.

   

   Figure 4: User interaction with MARS

    

7. References

   [Aust96] Austin, K.: Letters: On Identity and Fragmentation of the EA/CM Community. Computer Music Journal 20, No. 1 (1996) pp. 6-8.

   [GaDe84] Gariépy, L. and Décarie, J.: A System of Notation for electroacoustic music. Interface, Vol. 13 (1984) pp. 1-30.

   [Helm96] Helmuth, M.: Multidimensional Representation of Electro-acoustic Music. Journal of New Music Research, Vol. 25 (1996) pp. 77-103.

   [Smal93] Smalley, D.: Can Electro-Acoustic Music be analysed?. In R.Delmonte, M Baroni eds.: Atti del Secondo Convegno Europeo di Analisi Musicale, Universitá di Trento, Trento, Italy (1993) pp. 423-434.