The LIG (Laboratoire d'Informatique de Grenoble) proposes the following
Master 2 level internship:

*Title*: Document-Level Neural Machine Translation Evaluation

*Description*: Document-Level Neural Machine Translation (DL-NMT) is
currently one of the main research axes in NLP, with strong impact on
both academic and company research.

DL-NMT systems are evaluated with BLEU and dedicated test suites [Voita
et al., 2019, Muller&Rios 2018, Lopes et al., 2020]. The latter have
been designed to measure specifically to which degree DL-NMT systems
are able to take discourse phenomena into account while performing
translation, the average translation quality measured by BLEU being
inadequate in this respect.

With the work planned in this internship we would like to make a step
ahead in the design of test suites dedicated to DL-NMT evaluation. In
practice we would like to exploit semi-automatic annotation procedures
like those already used in [Muller&Rios 2018, Lopes et al., 2020] to
annotate explicitly discourse phenomena, such like coreferences and
anaphora, on data currently used for DL-NMT evaluation. Such annotation
will allow possibly to design more accurate and adequate evaluation
measures for discourse phenomena aware DL-NMT systems.

In this internship the student will use Machine Learning and Deep
Learning tools to automatically annotate parallel data used for NMT
evaluation with discourse phenomena. The annotation will be manually
verified and corrected. Based on the annotation we will design an
adequate evaluation metric for DL-NMT systems, taking into account the
capability of the system to exploit discourse phenomena. Finally, the
evaluation metric will be tested by evaluating DL-NMT systems already
available or trained from scratch at LIG.

Profile:

   -    Master 2 student level in computer science or NLP
   -    Interested in Natural Language Processing
   -    Skills in machine learning for probabilistic models
   -    Computer science skills:


   1.   Python programming. Some knowledge of deep learning libraries
        Pytorch and Fairseq would be a plus.
   2.   Data manipulation and annotation

The internship may last from 5 up to 6 months, it will take place at
LIG laboratory, GETALP team (http://lig-getalp.imag.fr/), starting from
January/February 2022. The student will be tutored by Marco Dinarelli
(http://www.marcodinarelli.it), and Lorenzo Lupo
(https://lorelupo.github.io).

Interested candidates must send a CV and a motivation letter to
marco.dinarelli@univ-grenoble-alpes.fr,
lorenzo.lupo@univ-grenoble-alpes.fr

[Voita et al., 2019] "When a good translation is wrong in context:
    Context-aware machine translation improves on deixis, ellipsis, and
    lexical cohesion". ACL 2019.

[Muller&Rios 2018] "A large-scale test set for the evaluation of
    context-aware pronoun translation in neural machine translation."
    CMT 2018

[Lopes et al., 2020] "Document-level neural MT: A systematic
    comparison". EAMT 2020