Anthropic ran a real-money classified marketplace where Claude agents negotiated on behalf of 69 employees, closing 186 deals worth $4,000+.
Anthropic ran an internal experiment called Project Deal in which AI agents — powered by different Claude model tiers — represented employee buyers and sellers in a classified marketplace. 69 employees received $100 budgets, resulting in 186 completed deals totaling over $4,000 in real value. The experiment tested four marketplace variants, including one 'real' market where deals were honored post-experiment. Key finding: users represented by more advanced models got better outcomes, but losers didn't know they were losing.
This experiment confirms that multi-agent negotiation pipelines produce measurably different economic outcomes depending on model capability — not just speed or accuracy, but real-world value extraction. The finding that user instructions didn't affect sale likelihood or price means the model's inherent reasoning, not prompt engineering, is the dominant variable. If you're building agentic pipelines where agents transact, the model tier you choose is now a business-critical decision, not just a cost-quality tradeoff.
Build a minimal two-agent negotiation loop using Claude's API this week: one agent as buyer, one as seller, with a defined item and price range. Measure deal closure rate and final price across claude-3-haiku vs. claude-3-5-sonnet to quantify the 'agent quality gap' in your own system.
Suggested exercise · not independently tested. Access requirements and costs may vary.
Open your terminal and set your ANTHROPIC_API_KEY environment variable
Paste this Python snippet: run two Claude calls in a loop — one with role='buyer' and system prompt 'You are negotiating to buy a used bicycle. Your max budget is $150. Start at $80.' and another with role='seller' and system prompt 'You are selling a used bicycle. Your floor is $100. Listed at $200.' Pass each agent's last message to the other for 5 rounds
Swap the model parameter between claude-3-haiku-20240307 and claude-3-5-sonnet-20241022 and compare final agreed price and rounds to close
Tags