Open-Weight Models for Production AI Agents: June 2026 Snapshot

Your agent framework doesn't matter if the model underneath it can't call tools reliably. An agent that hallucinates function names, misparses JSON arguments, or forgets its context mid-chain will fail regardless of how clever your orchestration logic is. The model is the engine. Everything else is plumbing.

As of June 2026, open-weight models are materially more capable for agent workloads than earlier generations. Multiple families now support native function calling, structured output, and multi-turn tool use out of the box. Some score competitively with proprietary alternatives on the Berkeley Function Calling Leaderboard (BFCL), while hosting economics vary sharply by provider and deployment mode. The trade-off used to be capability versus control; for well-scoped agent workloads, that gap is narrower than it used to be.

This review compares eight open-weight models for agent use cases using available public signals as of June 2026: tool calling accuracy, multi-step reasoning, context retention, hosting economics, and licensing terms. This isn't a general benchmark roundup. It's a buying guide for teams building agents that need to work in production, and every leaderboard-dependent claim should be rechecked before procurement. Treat the ordering below as a dated snapshot, not a permanent leaderboard.

How We Evaluated

Five criteria, weighted by what actually breaks agents in the field:

1. Tool-calling accuracy. Performance on BFCL V4 categories where available, plus any real-world function calling reliability evidence a team can gather. This is the single highest-weighted factor.
2. Reasoning depth. Scores on SWE-bench Verified, AIME, and LiveCodeBench as current snapshots. Agents that can't reason through multi-step problems will stall on anything beyond simple API wrappers.
3. Context window and retention. Raw context length matters less than how well the model uses it. Prefer measured recall and retrieval behavior over headline context-window size.
4. Hosting cost. Self-hosted inference costs (GPU requirements, quantization support) and API pricing through providers like Together, Fireworks, and Groq. MoE models with low active parameters can have a structural advantage here.
5. License. Apache 2.0 and MIT allow commercial use without restrictions. Meta's Llama Community License adds a usage threshold. Some enterprise teams treat anything short of Apache 2.0 as a procurement blocker.

Benchmark signals come from official model cards, the BFCL V4 leaderboard, SWE-bench Verified, and Artificial Analysis. Where models have multiple versions (DeepSeek V3 vs. V3.1 vs. V3.2), we used the latest stable release available as of June 2026.

At a Glance

BFCL V4 scores are overall composite across agentic, multi-turn, live, non-live, and hallucination components. Scores are approximate and should be rechecked against the linked leaderboard before procurement.

1. Qwen3-235B-A22B

Developer: Alibaba (Qwen Team) · Parameters: large MoE model · Architecture: MoE with MLA

Qwen3 is a strong choice for general agent workloads in June 2026. It supports both thinking and non-thinking modes, switching between slow deliberative reasoning and fast tool execution depending on the task. The Qwen team specifically optimized Qwen3 for Model Context Protocol (MCP) compatibility, making it an early open-weight model with first-class MCP support.

On benchmarks that matter for agents, Qwen3 performs well on the public evaluations cited here for instruction following and multi-step tool use. The MoE architecture keeps inference costs manageable by activating only a subset of parameters per token, so you get strong reasoning without the footprint of a fully dense model.

The practical advantage is the Qwen-Agent library, which bundles tool-calling templates and argument parsers that handle the messy parts of function calling. If you're building with LangGraph or CrewAI, Qwen3 slots in cleanly. Apache 2.0 licensing avoids the usual commercial-use friction. For teams building multi-tool agents that need to plan, reason, and execute reliably, Qwen3 is a solid starting point. (For a deeper comparison with other open-weight families, see our open-weight models comparison.)

Hosting: Multi-GPU for full precision; quantized deployments can be much lighter.

2. DeepSeek-V3.1

Developer: DeepSeek · Parameters: large MoE model · Architecture: MoE with routed experts + MLA

DeepSeek's V3 line has evolved rapidly. V3.1 merged the base V3 model with R1's reasoning capabilities into a single hybrid that switches between thinking and non-thinking modes. The result is a strong option for coding-oriented agents that value hybrid reasoning and tool use.

For agent builders, the key improvement in V3.1 is post-training optimization specifically targeting tool usage and multi-step task execution. Function calls that V3 would occasionally malform, V3.1 handles cleanly. The model also excels at error recovery, identifying when a tool call returns unexpected output and adjusting its approach rather than repeating the same failing request.

Cost is DeepSeek's structural advantage. It is generally among the lower-cost frontier-class options as of this snapshot, and the MIT license removes all commercial barriers. The main drawback is hosting complexity: the model is large enough that self-hosted deployment still needs serious infrastructure, even with quantization. Most teams will use DeepSeek through API providers. (For background on DeepSeek's architecture, see our DeepSeek explainer.)

Hosting: Multi-GPU for full precision; API usage is often the practical route.

3. Llama 4 Maverick

Developer: Meta · Parameters: large MoE model · Architecture: MoE with routed experts + 1 shared expert

Llama 4 Maverick has shown competitive public leaderboard results in the cited snapshots against major proprietary and open-weight competitors. Its defining feature for agent use is the very large context window combined with a sparse active-parameter footprint. That's an unusual combination: massive context at relatively modest compute cost.

For agent workloads, Maverick supports native function calling through an OpenAI-compatible tool interface. Fireworks AI and other providers expose structured function_call objects, so integration with existing agent frameworks requires minimal adaptation. The routed-expert architecture keeps latency predictable even on long contexts.

The limitation is the Llama 4 Community License. It's permissive for most teams, but very large consumer platforms need a separate agreement with Meta. For the vast majority of production deployments, this isn't a practical concern. In the public signals cited here, Maverick tends to lag Qwen3 on raw tool-calling accuracy in longer multi-step chains. It handles single-turn and parallel function calls well but occasionally loses track during longer sequential tool-use sessions.

Hosting: Quantized deployments can be relatively light; full model deployments still need multi-GPU.

4. Mistral Large 2

Developer: Mistral AI · Parameters: dense model · Architecture: Dense Transformer with GQA

Mistral Large 2 takes a different architectural approach. Instead of MoE, it's a fully dense model, meaning every parameter activates on every token. That makes it more expensive to run than MoE alternatives, but it also means more predictable behavior. Dense models don't have the expert routing variance that occasionally causes MoE models to produce inconsistent outputs on similar inputs.

One area where Mistral Large 2 compares well is function calling. In release comparisons, it showed strong tool-use behavior against major proprietary models. It natively supports parallel function calling, letting an agent dispatch multiple tool requests simultaneously and aggregate results. The context window handles substantial conversation histories and retrieval contexts without truncation.

The multilingual support is genuinely strong across many coding and natural languages including Chinese, Japanese, Arabic, and Hindi. For teams building agents that operate across language boundaries, Mistral Large 2 is a solid open-weight choice. The research-use Apache 2.0 license is more restrictive than Qwen3's full Apache 2.0, so verify your use case qualifies.

Hosting: Dense deployment needs substantially more memory than smaller models. API pricing varies by provider.

5. Command R+

Developer: Cohere · Parameters: dense model · Architecture: Dense Transformer

Command R+ was purpose-built for retrieval-augmented generation, and it shows. The model generates inline citations by default, pointing to specific passages in its context that support each claim. For RAG-based agents that need to cite sources, this eliminates the post-processing step of mapping outputs back to retrieved documents.

On function calling, Command R+ performed strongly on tool-use evaluations when it launched. Its multi-step tool use capability handles error correction automatically: if a tool call fails, the model identifies the problem and retries with corrected arguments. The context window accommodates large retrieval sets without chunking compromises.

The main constraint is the CC-BY-NC-4.0 license for the open-weight version, which prohibits direct commercial use without a Cohere enterprise agreement. Teams that need RAG-optimized agents without licensing overhead should evaluate whether Qwen3 or DeepSeek can match Command R+'s citation quality for their specific retrieval patterns. For enterprise deployments where Cohere's managed service is acceptable, Command R+ remains a strong option for knowledge-intensive agent tasks.

Hosting: Enterprise API pricing through Cohere varies by contract; self-hosting remains substantial.

6. Gemma 3 27B

Developer: Google DeepMind · Parameters: small dense model · Architecture: Dense Transformer

Gemma 3 occupies a sweet spot that the larger models can't reach: strong enough for real agent work, small enough to run on modest hardware with quantization. That makes it one of the more accessible models on this list for teams without cloud GPU budgets.

Google released FunctionGemma, a much smaller variant specifically fine-tuned for function calling on mobile and edge devices. This makes Gemma 3 one of the few families on our list with a purpose-built model for on-device agent deployment. The full model supports function calling, planning, and structured output through standard tool-use interfaces, with the context window providing enough room for complex agent memory.

Where Gemma 3 falls behind is raw reasoning depth. On current coding and tool-use evaluations, it trails the larger MoE models. It is not a drop-in replacement for Qwen3 for complex coding agents. But for chatbots with tool access, smart home controllers, customer service agents, or any workload where latency and hosting cost matter more than peak reasoning, Gemma 3 is a practical option. (For more on when smaller models beat larger ones, see our SLMs vs. LLMs comparison.)

Hosting: Works on modest consumer hardware with quantization. Free via Google AI Studio API.

7. Phi-4-mini

Developer: Microsoft · Parameters: very small dense model · Architecture: Dense Transformer

Phi-4-mini shows that a very small model can handle function calling. Microsoft added native tool-use support to the Phi-4 family, enabling both single and parallel function calls. For edge devices, IoT controllers, and mobile applications where running even a mid-size model is impractical, Phi-4-mini is a practical option.

The model performs surprisingly well on instruction following and basic reasoning for its size. Microsoft's post-training pipeline specifically targeted function calling accuracy, and the results show in structured scenarios with well-defined tool schemas. Where it struggles is ambiguity. Give Phi-4-mini a vaguely specified tool and it will sometimes hallucinate function names or fabricate URL parameters. Production deployments need strict input validation and well-constrained tool definitions.

The MIT license is maximally permissive. The model runs on low-power hardware and mobile chipsets. For teams building agent functionality into devices that can't phone home to a cloud API, Phi-4-mini is one of the few realistic options at this scale.

Hosting: Runs on low-power hardware. No GPU required for basic inference.

8. Yi-Large

Developer: 01.AI · Parameters: dense model · Architecture: Dense Transformer

Yi-Large trails the other models on this list in agent-specific capabilities, but it earns its place through strong multilingual performance and solid general reasoning. On the public leaderboards cited here, it sits in a similar overall-quality band to several larger proprietary models. Its particular strength is East Asian language performance, where it's competitive with larger models.

For agent workloads, Yi-Large handles knowledge retrieval, data classification, and conversational tasks competently. Tool-calling support exists but lags behind the dedicated function-calling optimization in Qwen3 or Mistral Large 2. The context window is smaller than the others on this list and a real constraint for agents that need to maintain long conversation histories or process large retrieval sets.

The Apache 2.0 license and reasonable size make it an accessible choice. Teams building multilingual agents focused on Chinese, Japanese, or Korean markets should benchmark Yi-Large against Qwen3 for their specific language distribution. For English-primary agent workloads, the other models on this list offer better tool-calling reliability.

Hosting: Requires meaningful GPU capacity or managed API access.

Decision Matrix: Which Model for Which Agent?

What About GLM-4.5?

GLM-4.5 from Zhipu AI has shown competitive BFCL V4 snapshots and in some function-calling categories compares well with proprietary models. We didn't include it in the main rankings because its English-language documentation and international hosting infrastructure are still maturing. For teams comfortable operating primarily in Chinese or with bilingual engineering staff, GLM-4.5 deserves serious evaluation. It's built on MoE architecture and specifically optimized for tool use, web browsing, and software development. Worth watching.

Frequently Asked Questions

Can open-weight models match GPT-4o or Claude for agent tasks?

On tool calling specifically, sometimes. Multiple open-weight models now score competitively with proprietary systems on the Berkeley Function Calling Leaderboard. Qwen3 and DeepSeek-V3.1 are competitive on some multi-step reasoning benchmarks too. Where proprietary models may still hold an edge is in the long tail of unusual tool-calling patterns and graceful error recovery on ambiguous inputs. For well-defined agent workflows with clear tool schemas, the gap may be small enough to justify testing open-weight options.

How much does it cost to self-host these models?

The range is enormous. Small models can run on consumer hardware, while the larger models need serious GPU infrastructure. For most teams, the practical choice is between self-hosting a smaller model or using API providers for the larger ones. MoE models offer a middle ground: their active parameter count is much lower than total parameters, so quantized versions are more deployable than the headline numbers suggest.

Which license is least restrictive for commercial use?

MIT (DeepSeek) and Apache 2.0 (Qwen3, Yi-Large) impose no meaningful commercial restrictions. Mistral Large 2's research-use Apache 2.0 is more restrictive, so confirm the fit for your use case. Llama 4's Community License adds a usage threshold that affects only the largest consumer platforms. Command R+'s CC-BY-NC requires a Cohere commercial agreement. Gemma's terms are permissive but custom. If licensing simplicity is your top priority, DeepSeek's MIT license is a straightforward option.

Should I use thinking mode or non-thinking mode for agents?

Both Qwen3 and DeepSeek-V3.1 support hybrid thinking/non-thinking modes. Use thinking mode for complex planning steps: deciding which tools to call, interpreting ambiguous results, recovering from errors. Switch to non-thinking mode for straightforward tool execution where the action is obvious from context. Most agent frameworks let you toggle this per step. Thinking mode is slower, so using it on every turn will noticeably slow your agent. (See our guide on Qwen's open-source approach for more on how thinking modes evolved.)

Related: AI Agents in Legal: What Works, What Fails,

Sources:

• Qwen3 Official Blog
• Llama 4 Model Card
• DeepSeek V3.1 Release Notes
• Mistral Large 2 Announcement
• Cohere Command R+ Documentation
• Gemma 3 Overview
• Phi-4-mini Function Calling Guide
• Berkeley Function Calling Leaderboard V4
• SWE-bench Verified Leaderboard
• Artificial Analysis Model Benchmarks
• FunctionGemma for Edge Devices