Quickstart

​

Setup

Install our test runner in the node project you want to test (or see our demo repo if you don’t have a project to try it on)

npm install --save-dev magnitude-test

Initialize Magnitude in your project by running:

npx magnitude init

This will create a basic tests directory tests/magnitude with:

• magnitude.config.ts: Magnitude test configuration file
• example.mag.ts: An example test file

​

Configure LLMs

Magnitude requires setting up two LLM clients:

1. 🧠 A strong general multi-modal LLM (the “planner”)
2. 👁️ A fast vision LLM with pixel-precision (the “executor”)

​

Planner Configuration

For the planner, you can use models like Gemini 2.5 pro, Claude Sonnet 3.7, GPT 4.1, or any other model that accepts image input.

Magnitude will automatically read and use any of the following API keys if configured:

• GOOGLE_APPLICATION_CREDENTIALS (gemini-2.5-pro-preview-03-25)
• OPENROUTER_API_KEY (google/gemini-2.5-pro-preview-03-25)
• ANTHROPIC_API_KEY (claude-3-7-sonnet-latest)
• OPENAI_API_KEY (gpt-4.1-2025-04-14)

If you have any of these in your environment, no additional setup is needed for the planner. To explicitly select a specific provider and model, see configuration. Currently we support Google Vertex AI, Anthropic, AWS Bedrock, OpenAI, and OpenAI-compatible providers.

​

Executor Configuration (Moondream)

Currently for the executor model, we only support Moondream, which is a fast vision model that Magnitude uses for precise UI interactions.

To configure Moondream, sign up and create an API with Moondream here, then add to your environment as MOONDREAM_API_KEY. This will use the cloud version, which includes 5,000 free requests per day (roughly a few hundred test cases in Magnitude). Moondream is fully open source and self-hostable as well.

🚀 Once you’ve got your LLMs set up, you’re ready to run tests!

​

Running tests

Run your Magnitude tests with:

npx magnitude

This will run all Magnitude test files discovered with the *.mag.ts pattern. If the agent finds a problem with your app, it will tell you what happened and describe the bug!

To run many tests in parallel, add -w <workers>

To learn more about different options for running tests see here.

​

Building test cases

Now that you’ve got Magnitude set up, you can create real test cases for your app. Here’s an example for a general idea:

import { test } from 'magnitude-test';

test('can log in and create company')
    .step('Log in to the app')
        .data({ username: 'test-user@magnitude.run', password: 'test' }) // any key/values
        .check('Can see dashboard') // natural language assertion
    .step('Create a new company')
        .data('Make up the first 2 values and use defaults for the rest')
        .check('Company added successfully');

Steps, checks, and data are all natural language. Think of it like you’re describing how to test a particular flow to a co-worker - what steps they need to take, what they should check for, and what test data to use.

For more information on how to build test cases see our docs.