Perverse Incentives
A perverse incentive is a reward mechanism that encourages counterproductive behavior, generating outcomes that undermine the objectives of the system, organization, or policymakers. These arise when individual rationality exploits structural flaws, metric misalignments, or unintended consequences of rule design.
Core Dynamics
- Misalignment: Agent goals diverge from principal objectives.
- Metric Gaming: Optimization of measured proxies over actual value (Goodhart’s Law).
- Unintended Consequences: Solutions exacerbate the problem (e.g., Cobra Effect).
- Rent-Seeking: Resources directed toward capturing value rather than creating it.
Mechanisms and Examples
- Fee Structures: Volume-based payments incentivize overconsumption or unnecessary intervention.
- AI Proxy Optimization: In artificial intelligence, focusing on token generation/consumption metrics rather than genuine intelligence or productive output creates a “billion-dollar mistake” where the proxy (tokens) is confused with the goal (intelligence) Goodhart’s Law in AI: The Cost of Confusing Tokens with Intelligence.