LISTEN TO THIS ARTICLE

Bazaar is a benchmark submitted on 30 July 2026 for testing whether a language-model merchant can learn customer preferences, set profitable prices and adapt when demand moves. The paper reports that even the strongest tested agent captured less than a third of hindsight-optimal profit, so checkout infrastructure is moving faster than pricing competence Bazaar.

Evidence base: the Bazaar paper and experimental HTML, OpenAI and Stripe's Agentic Commerce Protocol launch materials, the OpenAI checkout specification, Visa Intelligent Commerce, and Mastercard's Agent Pay Acceptance Framework Bazaar OpenAI Instant Checkout Stripe Instant Checkout.

Key takeaways

  • Main result: Bazaar tests repeated sealed-bid, multi-attribute pricing under hidden preferences, adaptive competitors and unannounced demand shifts Bazaar.
  • Practical implication: agentic commerce needs pricing and adaptation tests, not only payment tokens and checkout permissions.
  • Caveat: Bazaar is a controlled benchmark, so its model ranking should not be treated as a direct forecast of merchant performance in a live retail market.

What This Benchmark Actually Tests

Most agentic-commerce infrastructure starts from trust: who authorised the purchase, which merchant receives the order, which payment token is scoped to the transaction, and who remains responsible for fulfilment. That is necessary. It is also only half the economic system.

Bazaar tests the merchant side. In each round, a language-model merchant chooses a product configuration and price for customers whose preferences are hidden, while three adaptive rule-based competitors also bid Bazaar. The focal merchant observes sparse market feedback, not every losing bid or every customer utility, so the task is partly preference learning and partly margin control Bazaar.

That makes Bazaar adjacent to Swarm Signal's earlier agent-commerce trust-layer map and market-behaviour coverage. The trust-layer question is whether an authorised agent can transact safely. The Bazaar question is whether a selling agent can price without giving away margin or chasing the wrong customers, because its regret diagnostics separate missed profitable wins from wins that were underpriced Bazaar.

That distinction matters because an agent can look busy and still run a weak business.

Why checkout progress is not enough

The external infrastructure is real. OpenAI said on 29 September 2025 that Instant Checkout launched for United States ChatGPT users buying from United States Etsy sellers, with Shopify merchants planned, and initially supported single-item purchases. Stripe said on 29 September 2025 that the launch used the Agentic Commerce Protocol and Shared Payment Tokens so ChatGPT could initiate payment without exposing the buyer's payment credentials.

The OpenAI checkout specification keeps orders, payments and compliance on the merchant's existing commerce stack, with ChatGPT calling session endpoints and receiving authoritative cart state back from the merchant Agentic Checkout Spec. Visa frames its Intelligent Commerce portfolio around credentials, controls, authentication and protections for AI-initiated transactions Visa Intelligent Commerce. Mastercard's Agent Pay Acceptance Framework centres verified agents, agentic tokens, transaction traceability and purchase-intent data Mastercard Agent Pay Acceptance Framework.

Those systems help a transaction happen under bounded authority. They do not prove that the agent choosing the offer understands demand, competition or profit. Bazaar separates that missing commercial skill from the payment rail.

Profit and acquisition diverge

The headline result is not that every model fails in the same way. Bazaar reports that the agents leading customer acquisition are often not the agents leading profit, with Gemini 3.1 Pro cited as a strong acquisition model and Opus 4.6 cited as a stronger profit model in the paper's abstract Bazaar. That distinction matters because an agent can look busy and still run a weak business.

The paper's diagnostics split errors into losing profitably winnable customer rounds and winning while underpricing Bazaar. It reports that profit aligns more closely with margin per win than with win rate, which is exactly the failure a gross-volume dashboard can hide Bazaar.

Bazaar exposes a transaction trace with economic context: offer attributes, offered price, known cost, competitor outcome, realised profit where available, and the reason the agent changed its belief. A checkout receipt alone cannot show whether the agent bought growth by destroying margin.

Demand shocks expose stale beliefs

Bazaar also changes customer valuations without telling the focal merchant exactly when the shift occurred Bazaar. That design matters because real commerce changes through seasonality, trend shifts, supply events and competitor moves. A pricing agent that memorises yesterday's demand curve can become less useful after it has learned confidently.

The paper reports a strong-learner, weak-adapter pattern: some models that improved fastest before the shock were among the worst at adapting afterwards Bazaar. That is a production warning. Fast pre-shock learning can be a liability when the system lacks change detection, reset rules or exploratory budget.

The safer release gate is therefore not "does the merchant agent improve over the first thirty rounds?" It is "does the agent notice when old customer beliefs stop paying, and can it revise them without burning through budget?"

Bazaar scores acquisition separately from profit, while margin-per-win and regret metrics make margin discipline visible for merchants with known costs.

Thinking budget changes the mistake shape

Bazaar reports that increasing thinking effort can move a model along the failure surface rather than simply making it better Bazaar. In the paper's summary, more thinking lowers missed-win regret but can increase underprice regret, meaning a model may win more auctions while leaving more money on the table Bazaar.

That is useful because it turns "use the stronger reasoning mode" into a measurable business decision. Bazaar scores acquisition separately from profit, while margin-per-win and regret metrics make margin discipline visible for merchants with known costs. The same model family can fail differently when the reasoning budget changes, so the eval needs to score the objective the business actually has Bazaar.

A practical release test

A merchant-side agentic-commerce test should include three layers.

First, run a fixed checkout proof: scoped payment authority, user confirmation where required, merchant-of-record preservation, refund path and audit trail. The OpenAI checkout specification's session model is a useful control pattern because the merchant returns authoritative cart state rather than letting the agent invent fulfilment facts Agentic Checkout Spec.

Second, run a market replay. Bazaar's core experimental conditions include customers with hidden preferences, known merchant costs, competitors that react, and a demand shift the agent does not receive as an explicit label. Its closed-form utilities and merchant costs allow exact regret analysis after the run Bazaar.

Third, compare the agent against simple non-language baselines. Bazaar uses Thompson Sampling and EXP4-style bandit baselines in the same action space, which gives operators a floor for judging whether a language-model merchant adds economic value beyond adaptive pricing machinery Bazaar.

Benchmark to production bridge

The transferable lesson is the metric split. Score acquisition, realised profit, margin per win, loss regret, underprice regret and post-shock recovery separately Bazaar. Each failure points to a different repair surface: preference learning, cost modelling, exploration policy, price floors or shock detection Bazaar.

The model leaderboard is less portable. Bazaar uses a controlled repeated-auction environment with defined attributes, costs, customers and opponent dynamics Bazaar. A live retailer also has inventory, returns, promotions, brand constraints, advertising spend, fraud review, customer support and regulatory duties, which are not part of the benchmark's stated environment Bazaar.

That limit makes the benchmark a diagnostic template rather than a deployment certificate. The local question becomes sourceable and testable: before letting a merchant agent set prices, can it preserve margin under sparse feedback and recover when demand changes Bazaar?

Source trail

Research and technical sources:

Related Swarm Signal analysis: