The model configuration is included in the process of uploading models to Openlayer. It is usually provided as a dictionary/object or as a YAML file. Refer to the API reference for details on the upload process.

Note that the notion of uploading a model only applies to development projects. If you are working with monitoring, you only need to upload/stream production data.

Attributes

For LLMs, the config can contain the following attributes. Note that not all attributes are required, as some contain default values.

AttributeTypeDefaultDescriptionComments
inputVariableNamesList[str][]List of input variable names.
metadataDict[str, any]Dictionary containing metadata about the model.This is the metadata that will be displayed on the Openlayer platform.
modelstr-LLM name, such as gpt-3.5-turbo, or gpt-4.
modelProviderstr-LLM provider, such as OpenAI, Cohere, Anthropic, Google, etc.
modelParametersDict[str, any]-Model parametersE.g., {"temperature": 0, "top_k": 1}
namestr“Model”Name of the model.
promptList[Dict[str, str]]-Prompt for the LLM.The prompt should follow the format [{”role”: “…”, “content”: “…”}]

Examples

Let’s look at an example dataset from one of the sample notebooks from Openlayer’s examples gallery GitHub repository.

A valid model config would be:

model_config = {
    "prompt": prompt,
    "inputVariableNames": ["description", "seed_words"],
    "modelProvider": "OpenAI",
    "model": "gpt-3.5-turbo",
    "modelParameters": {
        "temperature": 0
    },
}

where prompt is:

prompt_template = """
You will be provided with a product description and seed words, and your task is to generate a list
of product names and provide a short description of the target customer for such product. The output
must be a valid JSON with attributes `names` and `target_custommer`.

For example, given:
```
description: A home milkshake maker
seed words: fast, healthy, compact
```
the output should be something like:
```
{
    "names": ["QuickBlend", "FitShake", "MiniMix"]
    "target_custommer": "College students that are into fitness and healthy living"
}

```
description: {{ description }}
seed words: {{ seed_words }}
"""
prompt = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt_template}
]