Durée: 4 à 6 mois (à partir de février 2026)
Lieu: ONERA (site de Palaiseau)
Encadrement: Valentina Dragos et Delphine Battistelli
Niveau requis: M2 TAL ou Informatique, Ecole d'ingénieurs
Connaissances nécessaires: Programmation Python, LLM, RAG ;
    connaissances en Ingénierie des connaissances (OWL, RDF) et en
    statistiques  (logiciel R) seraient un plus

Dossier de candidature (dès que possible) : cv, lettre de motivation,
notes M1 (ou équivalent) à envoyer aux deux adresses:
Valentina.dragos@onera.fr, Delphine.Battistelli@parisnanterre.fr

================
Title: Leveraging linguistic annotations to improve automatic detection
    of online hate content

Detection of specific online content remains a persisting issue in
social media analysis.This includes extremism, online hate or offensive
language, which are topics of interest for regulation of social
platforms. Being complex phenomena, automatic methods have to consider a
variety of features to capture their nature [1, 2]. Typically, automatic
approaches for content detection are based on natural language
processing and rely upon both fine-tuning using a specialized dataset
and development of learning models [3]. The objective of this work is to
broaden the scope of those methods by taking into account knowledge from
linguistic annotations, including emotions, appraisals and user
engagement [4,7]. Datasets to be used were collected on French social
platforms and highlight three types of online content: extremist,
hateful and sexist. There are two layers of manual annotation available:
a manual annotation of emotions and user's engagement in what is said.
In addition, there also layers of automatic annotations including
emotion annotation carried out with two different tools and the
annotation of appraisal categories. The interplay of different
linguistic annotations with hate, extremist or sexist content requires
further clarification and several experiments will be designed to
explore it. To consider both annotations and content type, both features
can be learned in a joint model [5]. This work is intended to
investigate whether this learning approach benefits from using enriched
datasets that contain annotations for content types and linguistic
dimensions. To this end, an experimental protocol will be adopted that
first focuses on analyzing the correlation of linguistic annotations and
content types, i.e. extremist, hateful and sexist. Then, preliminary
experiments will be designed to explore methods for learning the
phenomena jointly by leveraging linguistic analysis in online content
type detection. The experiments will use the transformer-based
pre-trained language model BERT [6] and similar architectures to perform
classification for various tasks (extremism, hate or sexism detection,
etc.), by gradually integrating the layers of linguistic descriptors.
Large language models (LLM's) and Retrieval Augmented Generation (RAG)
can also be used to support the excremental tasks [8]. The work has the
following milestones:
-   State of art and problem analysis (1 month)
-   Analysis of correlations between linguistic annotations and types of
    content (0.5 month)
-   Implementation and fine-tuning of models (1.5 months)
-   Set up of the experimental protocol, experimentation and analysis of
    results (1.5 months)
-   Internship report (0.5 month)

References:
1.  Schäfer, J., & Kistner, E. (2023). HS-EMO: Analyzing Emotions in Hate
    Speech. In Proceedings of the 19th Conference on Natural Language
    Processing (KONVENS 2023) (pp. 165-173).
2.  Yin, W., Agarwal, V., Jiang, A., Zubiaga, A., & Sastry, N. (2023).
    Annobert: Effectively representing multiple annotators' label choices
    to improve hate speech detection. In Proceedings of the International
    AAAI Conference on Web and Social Media (Vol. 17, pp. 902-913).
3.  Ljubesic, N., Mozetic, I., & Novak, P. K. (2023). Quantifying the
    impact of context on the quality of manual hate speech annotation.
    Natural Language Engineering, 29(6), 1481-1494.
4.  Dragos, V., Battistelli, D., Sow, F., & Étienne, A. (2024).
    Exploring the Emotional Dimension of French Online Toxic Content. In
    Proceedings of the 2024 Joint International Conference on
    Computational Linguistics, Language Resources and Evaluation
    (LREC-COLING 2024) (pp. 6945-6954).
5.  Rabiul Awal, M., Cao, R., Ka-Wei Lee, R., & Mitrovic, S. (2021).
    AngryBERT: Joint Learning Target and Emotion for Hate Speech
    Detection. arXiv e-prints, arXiv-2103.
6.  Wang, Z., Mayhew, S., & Roth, D. (2020). Extending multilingual BERT
    to low-resource languages. arXiv preprint arXiv: 2004.13640.
7.  Dragos, V., Battistelli, D., Etienne, A., & Constable, Y. (2022).
    Angry or Sad? Emotion Annotation for Extremist Content
    Characterization. In 13th Language Resources and Evaluation
    Conference.
8.  Arslan, M., Munawar, S., & Cruz, C. (2024). Political Events using
    RAG with LLMs. Procedia Computer Science, 246, 5027-5035.