GPT-3, BERT & Co. - When to Use Which Language Model?

Objective

In the past, a variety of language models has been proposed, such as GPT-3 and BERT, for natural language tasks (e.g., question answering, named entity recognition, text summarization). However, data scientists and AI researchers increasingly loose an overview when to use which language model.

In this thesis, the task is to obtain an overview of state-of-the-art language models and to create a conceptual framework (e.g., criteria), which can be used by any researcher or practitioner to quickly know when to use which model.

No language models necessarily need to be executed or trained. However, the student could implement a small online demonstration system as implementation of the above mentioned framework (e.g., input: problem description; output: recommended language model).

Prerequisites

1. Substantial data processing skills (e.g., in Python).
2. Ability to work independently on the topic (based on inputs from the supervisor).
3. Interest in publishing an own research paper based on the written thesis.

Contact Person Prof. Dr.-Ing. Michael Färber, michael.faerber@tu-dresden.de