Methodology
How To Read Reality

How To Read Reality

The FYOS Reality surface is an evidence layer, not a promise layer.

What Reality Is

Reality shows:

  • Predicted vs realized outcome comparisons
  • Historical evidence of model accuracy
  • Error profiles across cohorts
  • Reliability grades for trust assessment

Reality answers the question:

"How has the model performed historically, and what does that suggest about confidence levels?"

What Reality Is Not

Reality is not:

  • A guarantee of future accuracy
  • A promise that predictions will be correct
  • A replacement for careful interpretation
  • A blanket endorsement of all opportunities

Reality is an evidence surface — it shows you what happened, not what will happen.

Prediction Snapshot Persistence

FYOS persists prediction snapshots:

  • Captures model predictions at specific points in time
  • Records the expected outcomes based on model state
  • Enables later comparison against realized outcomes

This creates an audit trail for model evaluation.

Realized Outcome Evaluation

When outcomes become observable, FYOS evaluates:

  • Realized return vs predicted return
  • Prediction error magnitude and direction
  • Error patterns across cohorts, exchanges, and time

This evidence feeds back into trust and reliability systems.

Prediction Error Profiling

Error profiling reveals:

  • Systematic biases: Does the model consistently over- or under-predict?
  • Cohort patterns: Which segments have better accuracy?
  • Temporal patterns: Does accuracy vary over time?

Error profiles inform reliability grades and runtime haircuts.

Reliability Grading

Based on accumulated evidence, FYOS assigns reliability grades:

GradeMeaning
AStrong historical accuracy
BGood accuracy with some variance
CModerate accuracy, use caution
DWeak accuracy, significant caution
ungradedInsufficient history

Reliability grades inform:

  • Trust-layer confidence
  • Runtime haircut levels
  • User-facing warnings

How Reality Supports Decisions

Reality helps you understand:

  1. Confidence calibration: How much to trust model outputs
  2. Segment quality: Which exchanges/assets have better track records
  3. Uncertainty bounds: What range of outcomes is realistic
  4. Red flags: Where the model has struggled historically

Limitations

Reality has important limitations:

  • Past is not future: Historical evidence doesn't guarantee future accuracy
  • Regime changes: Market conditions can shift
  • Coverage gaps: Not all segments have equal evidence
  • Sample size: Small-sample cohorts have noisy estimates

Use Reality as context, not as a crystal ball.

Reality vs Trust Layer

ConceptPurpose
RealityShow evidence of model accuracy
Trust LayerGate and adjust based on evidence

Reality provides the evidence. Trust Layer uses it to inform product decisions.

Correct Interpretation

Right way:

  • "Reality shows this cohort has historically been accurate within X%"
  • "This is evidence, not a guarantee"
  • "I should adjust my confidence based on this evidence"

Wrong way:

  • "Reality proves the model is always correct"
  • "This historical accuracy guarantees future accuracy"
  • "I can ignore all other risk factors because Reality looks good"

Evidence-Oriented Philosophy

FYOS is intentionally designed as a pessimistic, evidence-oriented system.

Reality embodies this philosophy:

  • Show what actually happened
  • Don't hide model limitations
  • Don't oversell accuracy
  • Let users make informed decisions

Summary

  • Reality is an evidence surface, not a promise surface
  • It shows predicted vs realized outcomes
  • It informs reliability grades and trust
  • It has limitations — past is not future
  • Use Reality for confidence calibration, not certainty
  • Combine with other methodology components for complete interpretation
Cookie preferences
We use essential cookies for docs functionality and optional analytics cookies to improve the beta documentation experience. You can accept or reject non-essential cookies. Learn more in our Privacy Policy.