The Institute of Computer Science of the Poznan University of Technology is building an IT infrastructure for music research in the project “Digital Research Infrastructure for the Humanities and Arts Sciences DARIAH-PL”

Article was published in Głos Politechniki (September-October 2023)

Poznan University of Technology is one of the 16 consortium members of the “Digital Research Infrastructure for the Humanities and Arts Science DARIAH-PL” project, which is being implemented from 2021 to 2023.

The project is co-financed by the European Regional Development Fund, Operational Program Intelligent Development 2014-2020, Measure 4.2 - “Development of modern research infrastructure of the science sector.”

The aim of the DARIAH-PL project is to create a national intelligent digital research infrastructure for the humanities and arts sciences. The infrastructure is to be used for acquiring, storing and integrating diverse research data, as well as processing, visualizing and sharing digital resources. To this end, an affiliated network of distributed laboratories equipped with modern tools and resources for interdisciplinary research requiring the use of diverse materials is being built.

Planned effects of the project:

• expanding the scope of research in the humanities and art sciences conducted in Poland,

• digitization and protection of tangible and intangible cultural heritage and making this heritage available to a wide audience,

• increasing the competitiveness of Polish research centers and those related to culture and art at home and internationally ,

• accelerating the pace of development of creative industries and regional tourism.

Total project value: 129 496 338,76 PLN

Contribution of European Funds: 99 800 000,00 PLN

Project value for Poznań University of Technology: 1 324 232,68 PLN

The infrastructure under construction operates under the name Dariah.lab (https://lab.dariah.pl/).The leader of the project is the Poznan Supercomputing and Networking Center. At Poznań University of Technology, the project is headed by Ewa Łukasik, Ph.D., professor at PP from the Institute of Computer Science at the Faculty of Computer Science and Telecommunications.

Poznan University of Technology is building a system of web-based tools to support musicological research, with the working name MusicPUT, available anywhere and anytime. The subject matter of the project is closely related to the long-standing research interests of the project manager, Ewa Łukasik, PhD, professor at PP, her implementation of earlier projects within the Music Information Retrieval (MIR) Working Group at DARIAH-PL (https://mir-wg.dariah.pl/), and her cooperation with the Art Institute of the Polish Academy of Sciences, also a consortium member of the project in ethnomusicological research. The tools being built cover four subject areas, which required developers to have specialized knowledge beyond their professional competence. Two modules of the MusicPUT system use Artificial Intelligence methods.

ScoreScribe - a tool for automatic transcription of single-voice song recordings

The Art Institute's archives contain many historical recordings of the folk music of artists whose art was unique, and which today can only be made available to a limited extent due to RODO regulations. The laborious work of transcribers makes it possible to publish this music in score form. The ScoreScribe tool allows this process to be greatly expedited by automatically extracting the melody and converting it into computer-playable MIDI sounds and into several formats designed for “machine” representation of music, e.g. MEI, MusicXML, ABC, and score. Of course, a machine will not replace a human, but it can make the transcription process much easier, especially for novice transcribers. Long hours spent discussing the subject with Dr. Arleta Wysocka-Nawrocka of the Art Institute of the Polish Academy of Sciences influenced the functionality of the tool, as well as the development of a methodology for using it.

TimbrA - a tool for parameterization, visualization and timbre comparison

Vincent van Gogh's famous confession is: “I immerse myself in color” . Music lovers immerse themselves in the timbre of the sounds of music. Color in this context is a multidimensional phenomenon that is often expressed descriptively as, for example, light color, dark color, sharp color, harsh color, luscious color, rich color, nasal color, etc. How do we translate these terms into computational parameters if we treat the computer recording of music as data? In the field of Music Information Retrieval, a whole series of parameters defining timbre have been developed, which TimbrA (Timbre Analyzer) calculates, visualizes, compares and contrasts in various configurations to answer questions as specific as: “How does the timbre of different master violins differ?”, “How much does the timbre of a cello change when different musicians play it?”, “Does and how does the natural sound of the oboe differ from the synthesized sound?”. Using the TimbrA tool, these questions can be answered without having to program! To make this happen, the PP team consulted at length on the solutions with Dr. Magdalena Chudy of the Art Institute of the Polish Academy of Sciences.

GOST - music track generation tool for computer games

Using a database of retro computer game music, we taught a system called GOST (Gaming Online Sound Track) to generate game music in MIDI format. All you have to do is specify what emotional overtone you want the music to have - positive or negative, or what the tempo should be, and the tool will generate the music track. You can also provide the initial part of the music in the form of a MIDI file - the system will suggest its extension. There is also a post-processing module, with which you can change the key, tempo or instrument - a lot of room is left for the user to experiment with different parameters. This is a tool mainly for independent game developers, for whom providing a music track for a game is not easy.

OMRAT - automatic tool to support the annotation of sheet music documents for OMR

The solution that music librarians and digital library users, among others, are waiting for is the automatic “reading” of music notation scans. It is made possible by OMR - Optical Music Recognition. This is a complex method of recognizing the many small objects that are notes and other musical signs (more than 100 classes of symbols) packed into a small area, so that one can, for example, play music directly from notes, or search for a musical piece based on a query in the form of a series of sounds. For the system to work at its best, annotated training data for the neural network is needed. Manual annotation is extremely time-consuming. The idea was to build a tool called OMRAT (OMR Annotation Tool) that uses deep neural networks to detect music notation symbols based on the available training data, and leave to specialists the task of only manually correcting the marked documents. The mechanism is the same as for OMR, but the manual correction system allows you to prepare correctly annotated documents that will enrich the learning data and thus improve the performance of the target OMR system.

It is worth mentioning that, in addition, we offer a converter module that allows the conversion of music symbolic formats for multiple files simultaneously (e.g. MusicXML to MEI, MEI to MIDI, etc.). Such conversions are much needed in various archiving and research applications.

WIth fast mobile payment into the land of DARIAH MusicPUT

It is a requirement of the implementing authority that the infrastructure created be maintained by the partners for five years after the end of the project and made available in the form of paid services. In order to make use of the services possible for a wide audience, MusicPUT is equipped with a micropayment system. Small monthly or annual license fees are anticipated. Special agreements will be made with Institutional recipients.

Project contractors

It's time to introduce the project's contractors. They joined at different phases of its development, had different tasks, worked alone and remotely at the beginning, and as the tool integration phase came, they also integrated as a team. The work is headed by Ewa Lukasik, PhD, PP professor, who is also a subject matter expert and music system construction specialist. Dr. Tomasz Lukasik, Eng. is a machine learning specialist. Magdalena Sroczan, MA, is a specialist in administration, finance and UX. The programming team consists of Marcin Palasz, M.Sc., Stanislaw Graczyk, M.Sc., Wojciech Kasperski, M.Sc., Konrad Kubzdela, M.Sc., Filip Szymanski, M.Sc., Zuzanna Piniarska, M.Sc., and Mateusz Kałamoniak, M.Sc., M.Sc. All of them are computer science graduates of our university. Piotr Poznaniak, M.Sc. is a network infrastructure specialist, and Martyna Sarnowska, M.Sc. is a graphic interface designer. For a short period, students Patryk Kaszuba, Mateusz Ogrodowczyk, Kajetan Wencierski , Dorota Solarska, Daniel Parkhanovich and Anna Foltyniewicz, Eng. worked with us.

The project relies on youth and collaboration with the belief that the multifaceted experience that team members have gained from working on the MusicPUT system will bear fruit in future professional experiences and open them up to other fields of human activity in the coming era of transdisciplinarity, understood as a new space so far not completely occupied by any one discipline.

Project promotion

The tools produced have been and will be presented at national and international conferences. Let's list them out of chronicling duty:

• XV National Conference of Music Librarians, Wrocław, November 28-30, 2022,

• 154 Audio Engineering Society Convention, Helsinki, May 13-15, 2023,

• 4th Warsaw Seminar for Computer Science Researchers as part of the “Perspectives Women in Tech Summit”, June 14, 2023 (special invitation),

• DARIAH Annual Event, Budapest, June 6-9, 2023,

• PP-RAI National Conference on Artificial Intelligence, Lodz, June 24-26, 2023,

• European Seminar in Ethnomusicology ESEM, Palermo September 19-23, 2023, International Symposium on Sound Engineering and Tonnenmeisterung, ISSET Warsaw, October 12-14, 2023.

• Conference summarizing the DARIAH-PL project, Poznan, October 17-18, 2023.

• Digital Libraries for Musicology Conference , Milan, November 10, 2023.

• DARIAH-PL Open Seminar for the Greater Poland Region, Poznan, November 21-23, 2023.

There are still a lot of events planned to promote the project. The authors hope to see research papers and doctoral dissertations implemented using MusicPUT's infrastructure. If you are interested in research, please contact us by email at the dedicated address dariah@cs.put.poznan.pl

Invitation to presentation of DARIAH-PL project of Poznan partners

The results of the project obtained by Poznan research institutions will be presented to the community of the Wielkopolska region during the Open Seminar “Research Infrastructure for the Humanities and Arts Dariah.lab. Summary of the work of Poznan project partners UAM, PP, IBL PAN and PSNC", which will be held on November 21-22, 2023.

The invitation is extended to the entire academic community associated with the humanities in the broadest sense and the development of technologies for the humanities, as well as music librarians, archivists, lyricists, translators, musical instrument builders, students, students, game developers, enthusiasts of new technologies and digital humanities, to the community of the city of Poznan and the surrounding area. There will be lectures and demonstrations of the infrastructure produced in the laboratories of Poznan's partners, including, of course, Politecnica Poznan in the Lecture Center.

The event is open, but for organizational reasons there is a request for registration. Details can be found at https://lab.dariah.pl/seminarium.

Part of the MusicPUT project team.