M2 Master Internship on Automatic Alsatian speech recognition


Supervisors

Emmanuel Vincent
Multispeech team, Inria research center at Université de Lorraine, Nancy
emmanuel.vincent@inria.fr

Pascale Erhart
Language/s and Society team, LiLPa, Strasbourg
pascale.erhart@unistra.fr

Motivation and context

This internship is part of the Inria COLaF project (Corpora and tools
for the languages of France), whose objective is to develop and
disseminate inclusive language corpora and technologies for regional
languages (Alsatian, Breton, Corsican, Occitan, Picard, etc.),
overseas languages and non-territorial immigration languages of
France. With few exceptions, these languages are largely ignored by
language technology providers [1]. However, such technologies are keys
to the protection, promotion and teaching of these languages.

Alsatian is the second regional language spoken in France in terms of
number of speakers, with 46% of Alsace residents saying they speak
Alsatian fairly well or very well [2]. However, it remains an
under-resourced language in terms of data and language
technologies. Attempts at machine translation have been made as well
as data collection [3].

Objectives

The objective of the internship is to design an automatic speech
recognition system for Alsatian based on sound archives (radio,
television, web, etc.). This raises two challenges: i) Alsatian is not
a homogeneous language but a continuum of dialectal varieties which
are not always written in a standardized way, ii) the textual
transcription is often unavailable or differs from the pronounced
words (transcription errors , subtitles, etc.). Solutions will be
based on i) finding a suitable methodology for choosing and preparing
data, ii) designing an automatic speech recognition system using
end-to-end neural networks which can rely on the adaptation of an
existing multilingual system like Whisper [4] in a self-supervised
manner from a number of untranscribed recordings [5] and in a
supervised manner from a smaller number of transcribed recordings, or
even from text-only data [6].

The work will be based on datasets collected by LiLPa and the COLaF
project's engineers, which include the television shows Sunndi's Kater
[7] and Kùmme Mit [8] whose dialogues are scripted, some radio
broadcasts from the 1950s-1970s with their typescripts [9], as well as
untranscribed radio broadcasts of France Bleu Elsass. Dictionaries of
Alsatian such as the Wörterbuch der elsässischen Mundarten which can
be consulted via the Woerterbuchnetz portal [10] or phonetization
initiatives [11] could be exploited, for example using Orthal spelling
[12].

The internship opens the possibility of pursuing a PhD thesis funded
by the COLaF project.

Bibliography

[1] DGLFLF, Rapport au Parlement sur la langue française 2023,
https://www.culture.gouv.fr/Media/Presse/Rapport-au-Parlement-surla-langue-francaise-2023

[2] https://www.alsace.eu/media/5491/cea-rapport-esl-francais.pdf

[3] D. Bernhard, A-L Ligozat, M. Bras, F. Martin, M. Vergez-Couret,
P. Erhart, J. Sibille, A. Todirascu, P. Boula de Mareüil, D. Huck,
"Collecting and annotating corpora for three under-resourced languages
of France: Methodological issues", Language Documentation &
Conservation, 2021, 15, pp.316-357.

[4] A. Radford, J.W. Kim, T. Xu, G. Brockman, C. McLeavey,
I. Sutskever, "Robust speech recognition via large-scale weak
supervision", in 40th International Conference on Machine Learning,
2023, pp. 28492-28518.

[5] A. Bhatia, S. Sinha, S. Dingliwal, K. Gopalakrishnan, S. Bodapati,
K. Kirchhoff, "Don't stop self-supervision: Accent adaptation of
speech representations via residual Adapters", in Interspeech, 2023,
pp. 3362-3366.

[6] N. San, M. Bartelds, B. Billings, E. de Falco, H. Feriza,
J. Safri, W. Sahrozi, B. Foley, B. McDonnell, D. Jurafsky, "Leveraging
supplementary text data to kick-start automatic speech recognition
system development with limited transcriptions", in 6th Workshop on
Computational Methods for Endangered Languages, 2023, pp. 1-6.

[7] https://www.france.tv/france-3/grand-est/sunndi-s-kater/

[8] https://www.france.tv/france-3/grand-est/kumme-mit/toutes-les-videos/

[9] https://www.ouvroir.fr/cpe/index.php?id=1511

[10] https://woerterbuchnetz.de/?sigle=ElsWB#0

[11] 10.5281/zenodo.1174213

[12] https://orthal.fr/

Profile

MSc in speech processing, natural language processing, computational
linguistics, or computer science.

Strong programming skills in Python/Pytorch.  Knowledge of Alsatian
and/or German is a plus, but in no way a prerequisite.