What is generative artificial intelligence (AI)?

Generative artificial intelligence is a type of artificial intelligence that can generate different types of content. While the current debate centres on AI systems that can generate text (e.g. OpenAI's ChatGPT or Google's Bard), there are also systems that can generate images (e.g. Midjourney), videos or sound.
 

Large Language Models (LLMs)

AI systems such as ChatGPT are based on generative models, often known as large language models (LLMs), which can generate new content based on the data with which they have been trained. An LLM such as GPT-4 is populated with statistical information about the underlying data – in this case natural language – after it has been trained.
 

Training LLMs

Based on this information, the model is able to complete texts (so-called text-to-text models). An LLM "knows" that the sentence "The sky is..." is more likely to end with "blue" than with "yellow". The vast amount of training data means that, despite their very limited ability, language models can solve complex tasks with astonishing competence. It is important to note that language models are not (explicitly) knowledge models, but rather supplement texts probabilistically. Although a language model has "learned" a great deal of non-linguistic information, it is optimized to complete a text in a linguistically correct manner, rather than an accurate one.

In practice, such models are subjected to a further training step, known as alignment. This step is about training an LLM for specific tasks like following instructions, for example, with regard to certain values and norms. For example, when you use ChatGPT, you interact with generative models that are trained (aligned) to conduct dialogues, and to not deal with certain topics that are perceived as problematic.
 

Biases

The quality of the models and their output is fundamentally related to the quality of the training data, both in the first step and during the alignment. Biases in the underlying data are particularly important here. A generative model generates new content based on the probabilistic patterns of the training data. For example, if the training data have a gender bias, this will also be reflected in the model’s products.
 

Prompts

Current generative AI systems operate with natural language in terms of both input and output (for text-to-text models). The input for the model comes in the form of a prompt, a kind of written work order for the model. This prompt is then completed by the LLM in the case of text-to-text models. The quality of the prompt (see e.g. the lecture Generative AI for beginners delivered at the Day of Learning and Teaching 2023) therefore also has a major influence on the quality of the output. 

The prompt can be used to provide language models with important contextual knowledge or examples. This can be clearly illustrated using the example of an e-mail reply. If you prompt the language model to "Write an e-mail reply", you will receive a text that corresponds to an "average" e-mail and most likely does not fit into your context at all. The model cannot predict what it is supposed to answer. However, if you "show" the model the outgoing e-mail in the prompt, it can generate a suitable response. Based on this principle, language models can also be used to structure unstructured data or automatically reformat texts, for example.
 

Plugins and agents

Finally, reference should also be made to plugins and agents. A language model only has the knowledge found in the training data. OpenAI's GPT-4, for example, was not trained on data newer than September 2021. Therefore, the model does not have any information from the last two years. Plugins and agents make it possible to bring language models and other applications together. With a browser plug-in, for example, a language model can search for information on the Internet, helping it overcome its own limitations. For ChatGPT, for example, there is currently a wide range of plugins that allow you to perform calculations with WolframAlpha, or book trips.

• ChatGPT - briefly explained
  A video series developed by AI Campus (Dr Aljoscha Burchardt), which is dedicated to answering basic questions about ChatGPT and generative AI.
   
• HFD link library
  A regularly updated, annotated collection of sources that examine the topic from various perspectives.
   
• Learn Prompting
  A free online curriculum on the topic of prompting