Advanced
LLM Roles
Designate different LLMs for different responsibilities
You can customize the Magnitude agent to use different LLMs for each of the three primary operations: act
, extract
, query
.
By default when a single LLM is provided, all responsibilites will be handled by that LLM. However, by specifying different LLMs for certain roles you may be able to save on cost and speed.
Example:
One great use case for this is to reduce the cost of extracting data. While act
requires an intelligent and visually grounded model, extract
and query
do not require grounded models, and can often work fine with less intelligent models.
General recommendations:
act
: MUST use an intelligent, visually grounded modelextract
: Can use a fast and cheap model, likegemini-2.5-flash
or evengemini-2.5-flash-lite
query
: Can use any model that’s reasonably intelligent but fast, depending on the complexity of the queries you plan to ask.gemini-2.5-flash
might be a good option.