0. Scope Note

This entry is conceptual and systems-oriented.

It does not treat all incentives, goals, rewards, metrics, profit, performance targets, selection processes, optimization, accountability, compensation, ranking, or feedback mechanisms as inherently failed.

Systems need incentives.

Incentives may be valid when they are:

• bounded
• auditable
• transparent
• coherence-aligned
• meaning-compatible
• reality-responsive
• correction-compatible
• resistant to gaming
• paired with qualitative review
• prevented from total authority
• checked for side effects
• checked for upstream distortion
• tested against affected states
• revised when behavior changes
• subordinate to the purpose they measure

The failure begins when the incentive does not merely guide behavior, but reshapes the system’s perception of reality.

A valid incentive supports coherence.

A failed incentive rewrites coherence around itself.

Incentive Backpropagation occurs when downstream reward pressure travels upstream into sensing, reporting, design, language, classification, ethics, priorities, and meaning until the system becomes optimized for the incentive rather than the thing the incentive was supposed to serve.

The problem is not incentive.

The problem is incentive pressure propagating backward until reality is reorganized around the reward.

1. Definition

Incentive Backpropagation occurs when rewards, metrics, profit structures, selection pressures, institutional goals, optimization targets, or survival pressures propagate backward through a system and reshape upstream perception, behavior, classification, reporting, meaning, design, or ethics to serve the incentive rather than coherence.

The backpropagating incentive may be:

• profit
• growth
• adoption
• engagement
• ranking
• compliance
• safety score
• benchmark score
• throughput
• retention
• conversion
• productivity
• publication count
• citation count
• case closure
• risk reduction
• cost reduction
• political approval
• institutional legitimacy
• public optics
• grant funding
• market valuation
• user satisfaction score
• model performance target
• security compliance status
• justice process completion
• restoration closure metric

The distortion may affect:

• what is noticed
• what is ignored
• what is reported
• what is measured
• what is classified
• what is funded
• what is hidden
• what is rewarded
• what is punished
• what is considered ethical
• what is considered safe
• what is considered successful
• what is considered real
• what is designed
• what is optimized
• what is remembered
• what is named
• what is made legible

The core failure is:

reward target is installed
→ system optimizes toward target
→ upstream behavior adapts
→ measurement changes reality
→ reporting and perception bend toward reward
→ target substitutes for purpose
→ coherence declines

Incentive Backpropagation is not merely people gaming metrics.

It is the system reorganizing upstream reality around downstream reward pressure.

2. Core Pattern

The core pattern is:

1. A system defines a reward, metric, target, score, or success condition.
2. The target becomes tied to status, survival, funding, profit, approval, or authority.
3. Actors adapt behavior to the target.
4. Reporting adapts to the target.
5. Measurement begins shaping what is measured.
6. Upstream perception narrows around reward-relevant features.
7. Edge cases, harms, and unmeasured realities lose salience.
8. Meaning is redefined through the reward.
9. The system begins optimizing the proxy rather than the purpose.
10. Coherence-bearing information becomes inconvenient.
11. Hidden debt accumulates outside the target.
12. The incentive becomes reality-shaping.

A healthy system says:

this incentive is a tool and must remain subordinate to the fuller purpose

A backpropagated system says:

what the incentive rewards is what matters

The failure is especially subtle because the system may become more disciplined, efficient, measurable, and productive.

It may improve on the target while degrading the condition the target was meant to preserve.

3. Failure Signature

Typical signature:

reward pressure↑
metric authority↑
behavioral adaptation↑
reporting distortion↑
unmeasured reality visibility↓
purpose coupling↓
goodhart risk↑
measurement back-action↑
hidden debt↑
O↓

Extended signature:

target improves
purpose degrades

score rises
truth narrows

reporting improves
reality disappears

optimization succeeds
coherence fails

reward becomes map
map becomes world

Common verbal signatures include:

that is what we are measured on
we need to hit the target
the metric proves progress
we cannot optimize everything
what gets measured gets managed
leadership wants this number
that is outside the KPI
the benchmark is what matters
the model is improving
the process is working
we need to show impact
the incentive is aligned enough

Common system signatures include:

a company optimizes engagement while meaning, trust, and user agency decline
an institution improves closure rates by narrowing what counts as unresolved harm
an AI system improves benchmark scores while field reliability or redress worsens
a security team optimizes compliance status while actual risk remains hidden
a justice process improves throughput by reducing affected-state listening
a research system optimizes publication count while knowledge integrity declines
an economy optimizes growth while ecological and labor debt accumulates
a platform optimizes retention by increasing dependency rather than value

The defining condition is not that incentives shape behavior.

The defining condition is that incentives reshape upstream truth, meaning, design, and reporting until the reward substitutes for coherence.

4. Primary U-Layer Origin

Common origin layers:

• U1 — Power / Budgets: funding, profit, authority, status, survival, or access depends on meeting the incentive.
• U2 — Configuration / Boundaries: incentive scope exceeds its valid boundary and begins governing adjacent domains.
• U3 — Execution / Runtime: workflows adapt around the rewarded target.
• U4 — Information / Truth: reporting and measurement bend toward reward-compatible reality.
• U5 — Coordination / Time: short-term reward pressure outruns long-term coherence.
• U6 — Coherence Field: meaning, legitimacy, and trust are reshaped around success signals.
• U7 — Memory / Recurrence: historical performance data reinforces the target’s authority.
• U8 — Environment / Field: markets, institutions, competition, or governance regimes select for target-optimized behavior.

Common manifestation layers:

• U1 — Resources: resources flow toward target performance.
• U3 — Execution: behavior adapts to metric incentives.
• U4 — Truth: reporting reflects incentive pressure.
• U5 — Time: short-term optimization accumulates delayed cost.
• U6 — Field: meaning and legitimacy become proxy-shaped.
• U7 — Memory: prior metric success becomes proof of validity.

Incentive Backpropagation is primarily a G / Γ / Ψ / M failure.

Gain and selection pressure reshape observation and meaning until the system sees what it is rewarded to see.

5. Typical Development Sequence

A common development sequence is:

1. A metric, reward, target, score, or outcome is selected.
2. The target is tied to consequence.
3. Actors learn what is rewarded.
4. Workflows shift toward the rewarded behavior.
5. Reporting begins emphasizing reward-compatible evidence.
6. Unrewarded work becomes invisible.
7. Disconfirming signals become inconvenient.
8. The target becomes the language of success.
9. Design decisions begin serving the metric.
10. The system optimizes the proxy.
11. Real purpose drifts or inverts.
12. Hidden debt accumulates outside the measurement boundary.
13. The target’s authority increases because it shows improvement.
14. Coherence declines beneath optimized output.

The loop often looks like:

target → reward pressure → behavior adaptation → reporting distortion → target authority

Another common loop is:

metric improves → leadership trusts metric → more pressure on metric → more reality distortion

Incentive Backpropagation becomes durable when the reward structure becomes difficult to question because it produces visible success.

6. Diagnostic Markers

Diagnostic markers include:

• Metrics improve while underlying conditions degrade.
• Actors describe success in terms of the target rather than the purpose.
• Reports omit unmeasured harms or burdens.
• Work is redesigned around what is rewarded.
• Unrewarded but necessary work becomes neglected.
• Edge cases are pushed outside measurement boundaries.
• Teams become fluent in metric language but less connected to affected reality.
• The system resists adding measures that would complicate success.
• Data quality improves around rewarded dimensions and worsens elsewhere.
• Gaming behavior is treated as operational adaptation.
• Ethics review is performed after optimization decisions are already locked.
• Affected nodes report burden not visible in the metric.
• The incentive becomes harder to revise as performance improves.
• The system cannot distinguish success from target achievement.
• Long-term debt increases beneath short-term reward success.

Useful diagnostics:

• Incentive Gradient: Measures how strongly rewards pull behavior.
• Selection Pressure Distortion: Tracks how incentives reshape what survives.
• Metric-Reality Divergence: Measures gap between target performance and actual state.
• Goodhart Risk: Tests whether the proxy is replacing the purpose.
• Measurement Back-Action: Measures how measurement changes the measured system.
• Reporting Distortion: Detects reward-shaped reporting.
• Optimization Capture: Measures target dominance over design and decision.
• Truth Access Degradation: Tests whether inconvenient evidence remains visible.
• Meaning Drift: Tracks how purpose language changes under incentive pressure.
• Hidden Debt Load: Measures burden accumulating outside the rewarded target.

7. Related Gates

Relevant gates include:

• Incentive Audit Gate: Fails when reward effects are not inspected.
• Metric Integrity Gate: Fails when the metric stops preserving purpose.
• Selection Pressure Gate: Fails when incentives select against coherence.
• Optimization Target Gate: Fails when the target governs beyond valid scope.
• Truth Access Gate: Fails when reward pressure suppresses inconvenient reality.
• Measurement Back-Action Gate: Fails when measurement changes what it claims to observe.
• Goodhart Risk Gate: Fails when proxy optimization substitutes for success.
• Meaning Integrity Gate: Fails when purpose is rewritten by the incentive.
• Reporting Integrity Gate: Fails when reports become reward-shaped.
• Ethics Boundary Gate: Fails when ethical limits become downstream of incentives.

The first common gate failure is usually the Incentive Audit Gate.

Once incentive effects are not audited, reward pressure can silently reshape the entire upstream system.

8. Related Operators

Relevant operators include:

• G — Gain: Primary operator; reward pressure drives the backpropagation.
• Γ — Selection: Selects behaviors, reports, designs, and actors aligned with the incentive.
• Ψ — Observation / Interface: Displays target performance and hides unmeasured reality.
• M — Meaning: Purpose is rewritten around the metric or reward.
• O — Coherence: Declines when incentive success diverges from system health.
• Au — Auditability: Determines whether reward distortion remains visible.
• H — Hidden Debt: Accumulates in unmeasured burdens and ignored harms.
• K — Constraint / Load: Rises where actors must satisfy targets despite real complexity.
• Φ — Flow / Resource Movement: Routes resources toward rewarded dimensions.
• Τ — Trajectory / Time: Tracks short-term success and long-term debt.
• R — Restoration Capacity: Needed to repair damage from target-driven distortion.
• BΣ — Boundary Integrity: Defines where the incentive should and should not govern.
• Λ — Compatibility: Tests whether the incentive is compatible with the domain.
• D — Damping: Slows runaway optimization and protects interpretation.

Common operator pattern:

G increases reward pressure
Γ selects target-compatible behavior
Ψ displays metric success
M shifts toward target
Au misses distortion
H accumulates outside metric
O declines

The core operator inversion is:

rewarded outcome → real success

instead of:

rewarded outcome + purpose coupling + low distortion + affected-state validation + auditability → possible success

Incentive Backpropagation makes the system become what the incentive can measure.

9. Related Laws and Invariants

Related Laws

• Incentives Backpropagate Through Perception: rewards shape what the system can see.
• Rewards Shape What Systems Can See: rewarded features become salient while unrewarded reality fades.
• Metrics Must Not Rewrite Reality: measurement cannot be allowed to define the whole state.
• Selection Pressure Must Remain Coherence-Bounded: incentives must not select against system health.
• Optimization Targets Must Not Capture Meaning: purpose must remain upstream of targets.
• Profit Must Not Recode Truth: gain cannot determine what counts as real.
• Survival Pressure Must Not Override Auditability: pressure cannot justify blindness.
• Incentives Must Remain Subordinate to Coherence: rewards must serve the system’s purpose.
• Goodhart Collapse: proxies fail when optimized as targets.
• Success Proxy Substitution: success indicators can replace real success.
• Measurement Back-Action: measurement can alter the thing measured.
• U4 Truth Substitution: information forms can replace truth-bearing reality.

Related Invariants

• Incentives Must Be Audited for Upstream Distortion: reward effects must be inspected.
• Metrics Must Not Alter the Reality They Claim to Measure: back-action must be bounded.
• Selection Pressure Must Preserve Truth Access: incentives must not hide inconvenient signals.
• Optimization Must Remain Meaning-Compatible: target pursuit must not hollow purpose.
• Reports Must Not Be Shaped by Reward Alone: reporting must preserve reality access.
• Ethics Must Not Be Downstream of Profit: constraints must not be rewritten by gain.
• Design Must Not Be Captured by Measurement: design must remain purpose-led.
• Reward Structures Must Remain Correctable: incentives must be revisable under evidence.

10. Common False Positives

Not every incentive-driven behavior is Incentive Backpropagation.

Common false positives include:

• Incentives that remain bounded and audited.
• Metrics used as limited indicators.
• Rewards that are adjusted when distortion appears.
• Performance targets balanced by qualitative review.
• Optimization that preserves purpose and affected-state reality.
• Compensation systems that do not distort reporting.
• Benchmarking that remains one input among many.
• Profit structures paired with strong boundaries and repair.
• Safety scores validated against real-world outcomes.
• Accountability targets that improve actual accountability.
• Selection processes that preserve diversity and correction.
• Reward systems that include externalized costs.

Clarifying rule:

This is not Incentive Backpropagation unless reward pressure propagates upstream and reshapes perception, reporting, behavior, classification, design, ethics, or meaning in ways that serve the incentive over coherence.

Incentives can guide.

They fail when they become reality-shaping authority.

11. Common False Repairs

Common false repairs include:

• adding more metrics without restoring purpose
• changing the target while preserving the same reward pressure
• creating dashboards for distortion
• punishing obvious gaming while preserving Goodhart conditions
• adding ethics review after target optimization
• asking for qualitative reports that do not affect rewards
• adding disclaimers to metrics while still using them for authority
• measuring hidden debt without changing incentives
• adding counter-metrics that are also gamed
• reframing complaints as resistance to accountability
• increasing data collection instead of reducing target dominance
• rewarding transparency only when it does not hurt performance
• treating distortion as individual misconduct
• separating incentive design from affected-state evidence
• declaring alignment because the metric suite expanded

False repair often produces the loop:

metric distortion exposed
→ new metric added
→ reward pressure shifts
→ behavior adapts
→ new distortion appears

Another common loop is:

gaming discovered
→ gaming punished
→ incentive remains
→ subtler gaming develops

The repair fails because it treats distortion as misuse rather than incentive geometry.

12. Restoration Direction

Restoration requires auditing the incentive gradient, rebinding metrics to purpose, reducing proxy authority, protecting truth access, redesigning reward structures, validating outcomes against affected reality, and monitoring for renewed distortion.

Primary restoration direction:

make incentives serve coherence instead of rewriting it

A fuller restoration path includes:

1. Identify the incentive. Name the metric, reward, profit structure, target, score, or selection pressure.
2. Trace the incentive gradient. Determine what behavior the system is actually pulling for.
3. Compare target to purpose. Test whether the incentive still serves the original purpose.
4. Audit upstream distortion. Identify changes in perception, reporting, classification, design, or ethics.
5. Measure metric-reality divergence. Compare target performance against actual affected-state outcomes.
6. Identify unmeasured burdens. Map harms, labor, debt, or meaning loss outside the target.
7. Reduce proxy authority. Prevent the metric from becoming final truth.
8. Restore qualitative review. Reintroduce context, judgment, dissent, and affected-state evidence.
9. Redesign the reward structure. Align incentives with multidimensional coherence.
10. Add Goodhart damping. Rotate, bound, audit, and contextualize metrics.
11. Protect truth access. Ensure inconvenient signals can reach decision-makers.
12. Repair hidden debt. Address burden accumulated under reward pressure.
13. Separate ethics from incentive capture. Keep constraints upstream of gain.
14. Monitor behavioral adaptation. Watch how actors respond to new incentives.
15. Revalidate over time. Confirm that incentives remain coherence-serving.

A valid restoration path should reduce:

incentive distortion
metric-reality divergence
goodhart risk
reporting distortion
optimization capture
truth access degradation
meaning drift
hidden debt

Incentive Backpropagation is not repaired by better slogans about alignment.

It is repaired by changing what the system is actually rewarded to become.

13. Cross-Module Links

• Reduction / Extraction / Inversion: Primary family; incentives often begin as reduced proxies and then invert the system around them.
• Core: Strongly linked to Success Proxy Substitution and U4 Truth Substitution.
• Cybernetics: Goodhart Collapse and Measurement Back-Action are direct cybernetic expressions.
• Scaling: Feedback Gaming and Meaning Collapse often arise when incentive pressure scales.
• Economy: Profit, growth, retention, and market valuation can backpropagate through design and ethics.
• AI Governance: Benchmarks, safety scores, and compliance targets can reshape model development and reporting.
• Security: Compliance targets can replace real security state.
• Justice: Case closure and procedural throughput can replace repair.
• Organizations: KPIs and performance systems can restructure behavior around targets.
• Platforms: Engagement, retention, and conversion incentives can reshape user reality.
• Coherence: Coherence requires incentives to remain subordinate to truth, meaning, repair, and affected-state validation.

14. Relationship to Parent / Child Modes

Production treatment: Standalone Entry

This mode maps upward to:

• FM-C-018 — Goodhart Collapse
• FM-CORE-003 — Success Proxy Substitution
• FM-C-020 — Measurement Back-Action Loop
• FM-REI-001 — Improper Reduction
• FM-REI-005 — Functional Inversion

Sibling or related REI modes include:

• FM-REI-001 — Improper Reduction
• FM-REI-002 — Reduction-to-Authority Lock
• FM-REI-003 — Unbounded Extraction
• FM-REI-005 — Functional Inversion
• FM-REI-006 — Mislabeling Drift
• FM-REI-007 — Sensemaking Subordination

Related cross-family modes include:

• FM-CORE-003 — Success Proxy Substitution
• FM-CORE-006 — U4 Truth Substitution
• FM-S-007 — Feedback Gaming
• FM-S-012 — Meaning Collapse
• FM-C-018 — Goodhart Collapse
• FM-C-019 — Adversarial Reward Hacking
• FM-C-020 — Measurement Back-Action Loop
• FM-MT-003 — Single-Variable Obsession
• FM-AIX-003 — Defensive Compliance Attractor
• FM-AIX-010 — Status Quo Preservation Attractor
• FM-ECOX-016 — Risk Model Theater
• FM-ECOX-027 — Growth Theater

Aliases preserved from source material:

• Incentive Backpropagation
• Reward Backpropagation
• Metric Backpropagation
• Selection Pressure Backflow
• Incentive Distortion Cascade
• Goodhart Backpropagation
• Optimization Backflow
• Target-Induced Distortion
• Reward-Shaped Reality
• Incentive-Captured Sensemaking

15. Minimal Entry Version

Definition: Incentive Backpropagation occurs when rewards, metrics, profit structures, selection pressures, institutional goals, optimization targets, or survival pressures propagate backward through a system and reshape upstream perception, behavior, classification, reporting, meaning, design, or ethics to serve the incentive rather than coherence.

Signature:

reward pressure↑
metric authority↑
behavioral adaptation↑
reporting distortion↑
unmeasured reality visibility↓
purpose coupling↓
goodhart risk↑
measurement back-action↑
hidden debt↑
O↓

Restoration direction:

• identify the incentive
• trace the incentive gradient
• compare target to purpose
• audit upstream distortion
• measure metric-reality divergence
• identify unmeasured burdens
• reduce proxy authority
• restore qualitative review
• redesign the reward structure
• add Goodhart damping
• protect truth access
• repair hidden debt
• separate ethics from incentive capture
• monitor behavioral adaptation
• revalidate over time

16. Machine-Readable Summary

failure_mode:
  id: "FM-REI-004"
  name: "Incentive Backpropagation"
  family: "Reduction / Extraction / Inversion"
  production_treatment: "Standalone Entry"
  parent_modes:
    - "FM-C-018 — Goodhart Collapse"
    - "FM-CORE-003 — Success Proxy Substitution"
    - "FM-C-020 — Measurement Back-Action Loop"
    - "FM-REI-001 — Improper Reduction"
    - "FM-REI-005 — Functional Inversion"
  primary_failure: "Rewards, metrics, profit structures, selection pressures, institutional goals, optimization targets, or survival pressures propagate backward through a system and reshape upstream perception, behavior, classification, reporting, meaning, design, or ethics to serve the incentive rather than coherence."
  source: "UTS — Failure Modes Registry"
  source_id: "FM-REI-004"
  scope_note: "Conceptual and systems-oriented; does not treat all incentives, goals, rewards, metrics, profit, performance targets, selection processes, optimization, accountability, compensation, ranking, or feedback mechanisms as inherently failed."
  aliases:
    - "Incentive Backpropagation"
    - "Reward Backpropagation"
    - "Metric Backpropagation"
    - "Selection Pressure Backflow"
    - "Incentive Distortion Cascade"
    - "Goodhart Backpropagation"
    - "Optimization Backflow"
    - "Target-Induced Distortion"
    - "Reward-Shaped Reality"
    - "Incentive-Captured Sensemaking"
  signature:
    - "reward pressure↑"
    - "metric authority↑"
    - "behavioral adaptation↑"
    - "reporting distortion↑"
    - "unmeasured reality visibility↓"
    - "purpose coupling↓"
    - "goodhart risk↑"
    - "measurement back-action↑"
    - "hidden debt↑"
    - "O↓"
  primary_layers:
    origin:
      - "U1 — Power / Budgets"
      - "U2 — Configuration / Boundaries"
      - "U3 — Execution / Runtime"
      - "U4 — Information / Truth"
      - "U5 — Coordination / Time"
      - "U6 — Coherence Field"
      - "U7 — Memory / Recurrence"
      - "U8 — Environment / Field"
    manifestation:
      - "U1 — Resources"
      - "U3 — Execution"
      - "U4 — Truth"
      - "U5 — Time"
      - "U6 — Field"
      - "U7 — Memory"
  state_variables:
    - "G"
    - "Γ"
    - "Ψ"
    - "M"
    - "O"
    - "Au"
    - "H"
    - "K"
    - "Φ"
    - "Τ"
    - "R"
    - "BΣ"
    - "Λ"
    - "D"
  first_gate_failure: "Incentive Audit Gate"
  restoration:
    - "Incentive Gradient Audit"
    - "Metric-Reality Rebinding"
    - "Goodhart Risk Reduction"
    - "Selection Pressure Correction"
    - "Truth Access Restoration"
    - "Reporting Integrity Repair"
    - "Optimization Target Revalidation"
    - "Meaning Coupling Restoration"
    - "Ethics Boundary Re-Separation"
    - "Reward Structure Redesign"