Skip to content

Concepts

GERD is primarly a tool for prototyping workflows for working with Large Language Models. It is meant to act as 'glue' between different tools and services and should ease the access to these tools.

In general, there should be only be two components involved in a GERD workflow: A configuration and a service. The configuration can be assembled from different sources and should be able to be used in different services. The foundation of such a configration is a YAML file. GERD provides a set of those which can be found in the config directory. Configurations in this directly can be accessed by name as shown in the following example. A simple configutation file may look like this:

model:
  name: "Qwen/Qwen2.5-0.5B-Instruct"
  temperature: 0.1
  top_p: 0.90
  max_new_tokens: 256
  repetition_penalty: 1.1
  prompt_setup:
    - [system, text: "You are a helpful assistant. Start your answers with 'Sure thing, buddy!'."]
  prompt_config:
    text: "Please say the word '{word}' three times."

And can be used with a ChatService via load_gen_config shown in the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
"""A 'hello world' example for GERD."""

import logging

from gerd.config import load_gen_config
from gerd.gen.chat_service import ChatService
from gerd.models.model import PromptConfig

logging.basicConfig(level=logging.WARNING)
logging.getLogger("gerd").setLevel(logging.DEBUG)

logging.info(
    "Loading chat service..."
    " When this is the first time you run this script, it will download the model."
    " This may take a few minutes."
)

chat = ChatService(load_gen_config("hello"))
res = chat.submit_user_message({"word": "teleportation"})
logging.info(res)

chat.set_prompt_config(PromptConfig.model_validate({"text": "{message}"}))
res = chat.submit_user_message({"message": "Hello! What is one plus one?"})
logging.info(res)