Back

A Solution to Latency Problems in Remote Music Collaboration

Jussi Tuohino, Planning Officer & Producer, Oulu University of Applied Sciences
EdTech Tools Innovation in practice Online learning

In Italy, the LoLa system has been developed for more than ten years to minimize the delay in audiovisual remote collaboration. For example, musicians located more than a thousand kilometers apart can play together in precise time with LoLa. The Oulu UAS’ School of Media and Performing Arts is actively testing the use of LoLa.

Remote collaboration is very challenging in matters that need to take place in the most accurate time synchronization between participants. In audio and video systems, network traffic and acoustic movement of sound there is always delay, which is particularly detrimental to collaborated music performance. It is especially noticeable with the common remote communication tools. During the Covid-19 pandemic, latency (i.e., the delay experienced by remote parties in communicating with each other) has become familiar to everyone involved in musical collaboration and teaching.

Although technically the delay is never completely eliminated due to physical constants, the Italian Conservatorio Giuseppe Tartini and the GARR research network consortium have been developing the LoLa system for more than 10 years, enabling connections with delays of just a few milliseconds at hundreds and even thousands of kilometers between the remote participants. However, LoLa (low latency) requires a fast symmetrical (equal speed in both directions) network connection to work properly, which in practice means a connection mainly to the networks used by research and education organizations. In addition, the system includes high-quality computer, audio and video components and software for managing the connections. [1]

The LoLa system was set up in Oulu UAS in 2019, and from the last year it has been tested more and more actively with partner universities both nationally and internationally. Oulu UAS’ School of Media and Performing Arts sees fruitful opportunities for using LoLa to increase remote collaboration in music and dance studies, and for the excellent sound and picture quality and low latency offered by the system in general. The MUSE – Accessible and Low Latency Solutions for Music Distance Education, Performance, and Production project, implemented in co-operation between the universities of applied sciences of Jyväskylä, Oulu, Tampere, and Turku, will also continue to coordinate and train LoLa co-operation in Finland until 2023.

UNIQUE EXPERIENCES REMOTELY

The user experience of the LoLa system differs from the use of web conferencing applications such as Zoom in that the application itself is very simplified and the user can customize only a few technical settings in addition to choosing contact partners’ IP addresses. Initially, only the image of the user’s own camera system is displayed in a large window, until a connection is established to a remote destination from which the incoming image opens in its own window. The audio connection starts at the same time as the picture.

The picture and sound quality of the connection is simply excellent, and the people in remote locations seem to be present with a surprisingly authentic feel. At reasonable distances (from hundreds of kilometers up to a couple of thousand kilometers) the connection is essentially latency-free.

The operation of the system is based on minimizing the delay caused by the system components at all stages. This means that audio and video streams are in principle uncompressed and are transmitted directly to the recipient as quickly as possible. In terms of the video picture stream, however, LoLa can enable JPEG compression, which significantly reduces the required network connection bandwidth compared to uncompressed video (Full HD resolution at 30 frames would already produce 1700 Mbps data traffic uncompressed, and thus would exceed the limits of conventional network adapters and connections). For comparison, at the default compression level, JPEG-compressed video can be transferred at 60 Mbps.

LoLa, which operated previously only as a two-way connection, has acquired more important features in its new version 2.0: multipoint connections with three sites are possible, and up to four cameras can now be connected to the system, which enhances the image sent to the remote sites. However, LoLa is precise about its system requirements, and only USB3 cameras from the partner manufacturer XIMEA can be used, for example.

DELAY IN THE BIG PICTURE

Minimizing the delay is problematic in video playback. Commonly available video projectors and large televisions have a surprisingly slow response time and are therefore not very compatible with LoLa. As a result, high-speed game screens are often used with LoLa. With bigger screens the remote music concerts suffer from the slowness of video that is sent to the audience, even though the audio experience is similar to a situation where remote players are in the same space as local performers. Fortunately, video projectors with sufficient brightness and low input lag are being developed in addition to large TVs designed for gaming, so the audience will soon be able to enjoy remote collaborated music concerts with much better visual projection of the remote performers.

LoLa does not process or compress the audio signal transmitted and received by the system in any way, and this also means that the system does not have an echo cancellation feature. Therefore, the correct placement of speakers and microphones is important. The remote participants must ensure that their microphones do not pick up too much sound from remote sites’ playback sound to avoid feedback problems.

The purchase price of around € 10,000 for one system cannot be considered unreasonable as an investment by an education organization, but the total cost of building and using the system (including network connection requirements) might make the acquiring and using the system somewhat more complicated for smaller operators. In Finland, LoLa is already in use at the eight universities of applied sciences offering cultural education and at the Sibelius Academy of the University of the Arts Helsinki. [2]

FUTURE REMOTE COOPERATION

With the help of LoLa, it is possible to seamlessly produce performances in which the performers are located at long distances from each other. Teaching musical group performance remotely is finally possible with the help of LoLa, and in the future the system will probably enable even more possibilities for multi-point connections and improved visual projection. Multi-channel audio streams are already supported, leading to interesting possibilities for remote music production and multi-speaker sound reinforcement.

Oulu UAS will continue to actively test the LoLa system with tests already agreed for the upcoming weeks. From the system user’s point of view, it can already be said that a workable solution has been found to the latency problem of remote collaboration.

SOURCES