Custom LLM Parameters

Tambo uses an LLM behind the scenes to process user messages. You can change what model Tambo uses, and while Tambo uses certain parameters by default when calling the LLM, you can override these parameters to customize behavior depending on your chosen provider.

How Does It Work?

Custom LLM parameters allow you to override default model settings with provider-specific configurations. Parameters are stored per model, letting you optimize different models independently.

Example configuration:

• temperature: 0.7
• maxOutputTokens: 1000
• topP: 0.9
• presencePenalty: 0.1

Why Use Custom LLM Parameters?

• Fine-tune output quality - Control randomness, length, and creativity
• Optimize for use cases - Different parameters for creative writing vs. code generation
• Provider compatibility - Support both standard and custom parameters
• Model-specific tuning - Configure each model independently

Configuring Parameters in the Dashboard

All LLM parameters are configured through your project settings in the dashboard.

Step 1: Access Provider Settings

1. Navigate to your project in the dashboard
2. Go to Settings → LLM Providers
3. Select your provider and model

Step 2: Add Parameters

The dashboard shows suggested parameters based on your selected model:

For Common Parameters:

1. Under Custom LLM Parameters, you'll see suggested parameters like temperature, maxOutputTokens, topP, etc.
2. Click + temperature (or any other parameter) to add it
3. Enter the value (e.g., 0.7 for temperature)
4. Click Save to apply the configuration

For Custom Parameters (OpenAI-Compatible Only):

1. If you don't see the parameter you need in the suggestions, you can add custom parameters
2. Click to add a parameter manually
3. Enter the parameter name (e.g., max_tokens, logit_bias)
4. Enter the value in the appropriate format (string, number, boolean, array, or object)
5. Click Save to apply

Example Configuration

Setting up a creative writing model:

1. Select your provider and model
2. Click + temperature → Enter 0.9
3. Click + topP → Enter 0.95
4. Click + presencePenalty → Enter 0.6
5. Click + maxOutputTokens → Enter 2000
6. Click Save

Basic Parameters

Common Parameters (All Providers)

These are the parameters supported by tambo and suggested for use across all LLM providers. For providers other than OpenAI-compatible, users can only use these common parameters - custom parameters are not available. These parameters are supported across all tambo providers:

Parameter Data Types

OpenAI-Compatible Providers

OpenAI-compatible providers support both suggested parameters (the common parameters above) and custom parameters for advanced use cases.

Suggested Parameters for OpenAI-Compatible

All common parameters listed above are available as suggestions for OpenAI-compatible providers.

Custom Parameters for OpenAI-Compatible

For full flexibility with OpenAI-compatible APIs, you can add any custom parameter supported by the provider. These parameters are configured through the tambo UI and passed directly to the OpenAI-compatible API.

Available Custom Parameters

These are examples of custom parameters that may be supported by OpenAI-compatible providers following OpenAI's API:

Advanced Usage Patterns

Creative Writing Model

• temperature: 0.9 (high creativity)
• topP: 0.95 (diverse word choices)
• presencePenalty: 0.6 (encourage new topics)
• maxOutputTokens: 2000 (longer responses)

Code Generation Model

• temperature: 0.2 (low randomness)
• topP: 0.1 (focused word choices)
• frequencyPenalty: 0.3 (reduce repetition)
• stopSequences: ["\n\n", "###"] (stop at logical breaks)

Deterministic Analysis Model

• temperature: 0.0 (completely deterministic)
• seed: 42 (reproducible results)
• maxOutputTokens: 500 (controlled length)

Custom OpenAI-Compatible Setup

• temperature: 0.7
• max_tokens: 1000
• logit_bias: {"50256": -100} (modify token probabilities)
• user: "analytics-user" (for monitoring)
• presence_penalty: 0.1

Integration with Projects

Parameters are configured per project and stored with the following hierarchy:

• Provider (e.g., "openai", "openai-compatible")
• Model (e.g., "gpt-4", "claude-3-sonnet", "custom-model-v1")
• Parameters (key-value configuration)

This allows different projects to have different parameter sets for the same model, enabling fine-tuned optimization across use cases.

Best Practices

• Start with defaults: Begin with suggested parameters before adding custom ones
• Test incrementally: Change one parameter at a time to understand effects
• Document configurations: Note which parameter sets work best for specific use cases
• Monitor usage: Higher token limits and retries can increase API costs
• Use custom parameters sparingly: Only for OpenAI-compatible providers when needed

Troubleshooting

Parameters not applying?

• Verify you're using an OpenAI-compatible provider for custom parameters
• Check parameter syntax matches the expected type (string/number/boolean/array/object)

Model not responding as expected?

• Lower temperature values (0.0-0.3) for more deterministic responses
• Adjust topP and topK for fine-grained control over randomness
• Use stopSequences to prevent rambling responses

API errors with custom parameters?

• Ensure custom parameter names match the provider's API documentation
• Verify parameter values are within acceptable ranges
• Check that the provider supports the custom parameter you're trying to use