Magnitude requires configuring two language models:

  1. “Planner” model (any good multi-modal LLM)
  2. “Executor” model (currently only Moondream is supported)

For the planner model, we currently support Google Vertex AI, Anthropic, AWS Bedrock, OpenAI, and OpenAI-compatible providers.

While many providers are supported, we would strongly suggest using Gemini 2.5 pro (via Vertex AI or OpenAI-compatible) or Claude Sonnet 3.7 (via Anthropic or Bedrock) for the planner.
Magnitude uses BAML’s providers under the hood, so their docs may be a useful secondary reference for credential configuration.

To configure your planner model, pass one of the client interfaces described below to your magnitude.config.ts, like:

import { type MagnitudeConfig } from 'magnitude-test';

export default {
    url: "http://localhost:5173",
    planner: {
        provider: 'anthropic', // your provider of choice
        options: {
            // any required + optional configuration for that provider
            model: 'claude-3-7-sonnet-latest',
            apiKey: process.env.ANTHROPIC_API_KEY
        }
    }
} satisfies MagnitudeConfig;

If no planner is configured, Magnitude will pick a provider and model based on available environment variables in this order:

  • GOOGLE_API_KEY (Google AI Studio gemini-2.5-pro-preview-03-25)
  • OPENROUTER_API_KEY (OpenRouter google/gemini-2.5-pro-preview-03-25)
  • ANTHROPIC_API_KEY (Anthropic claude-3-7-sonnet-latest)
  • OPENAI_API_KEY (OpenAI gpt-4.1-2025-04-14)

​
Providers

​
Google AI Studio

interface GoogleAIClient {
    provider: 'google-ai',
    options: {
        model: string,
        apiKey?: string // defaults to GOOGLE_API_KEY
        temperature?: number,
        baseUrl?: string // defaults to https://generativelanguage.googleapis.com/v1beta
    }
}

​
Google Vertex AI

interface GoogleVertexClient {
    provider: 'vertex-ai',
    options: {
        model: string,
        location: string,
        baseUrl?: string,
        projectId?: string,
        credentials?: string | object,
        temperature?: number,
    }
}

The easiest way to authenticate with Vertex AI is to authenticate using the gcloud CLI.

  1. Create a project in Google Cloud.
  2. Enable Vertex AI in that project by going to Vertex AI Dashboard an selecting “Enable all Recommended APIs”
  3. Install the gcloud CLI (instructions)
  4. Run gcloud auth application-default login --project <your-project-id>

Once you’ve done these steps, you can set up a project to use Vertex with the available credentials like this:

import { type MagnitudeConfig } from "magnitude-test";

export default {
    url: "http://localhost:5173",
    planner: {
        provider: 'vertex-ai',
        options: {
            model: 'google/gemini-2.5-pro-preview-05-06',
            location: 'us-central1'
        }
    }
} satisfies MagnitudeConfig;

If running in GCP, it will query the metadata server to use the attached service account.

More info: BAML Google Vertex Provider Docs

​
Anthropic

interface AnthropicClient {
    provider: 'anthropic',
    options: {
        model: string,
        apiKey?: string,
        temperature?: number
    }   
}

​
OpenAI

export interface OpenAIClient {
    provider: 'openai',
    options: {
        model: string,
        apiKey?: string,
        temperature?: number
    }
}

​
OpenAI-compatible (OpenRouter, Ollama, etc.)

export interface OpenAIGenericClient {
    provider: 'openai-generic'
    options: {
        model: string,
        baseUrl: string,
        apiKey?: string,
        temperature?: number,
        headers?: Record<string, string>
    }
}

​
AWS Bedrock

interface BedrockClient {
    provider: 'aws-bedrock',
    options: {
        model: string,
        // passed to inference_configuration
        temperature?: number
    }   
}

Authenticate with bedrock using environment variables:

export AWS_ACCESS_KEY_ID="your_key"
export AWS_SECRET_ACCESS_KEY="your_secret"
export AWS_REGION="us-east-1"

​
Azure OpenAI

interface AzureOpenAIClient {
    provider: 'azure-openai',
    options: {
        resourceName: string,
        deploymentId: string,
        apiVersion: string,
        apiKey: string
    }
}
More info on authenticating with Azure: https://docs.boundaryml.com/ref/llm-client-providers/open-ai-from-azure

​
Configuring Moondream

Moondream cloud is the easiest way to get set up, and offers 5,000 free requests per day. Get an API key here.

Moondream is open source and can also be self-hosted instead of using their cloud option. See here for instructions.

If self-hosting, configure the baseUrl to point to your server:

import { type MagnitudeConfig } from 'magnitude-test';

export default {
    url: "http://localhost:5173",
    executor: {
        provider: 'moondream', // only moondream currently supported
        options: {
            baseUrl: 'your-self-hosted-moondream-endpoint',
            apiKey: process.env.MOONDREAM_API_KEY // not necessary if self-hosted
        }
    }
} satisfies MagnitudeConfig;