# ORION

## The Citability Intelligence System

### Public Methodology v5.0

**Published:** April 2026
**Operator:** Orion (WhatsMyGeoScore)
**Status:** Research release. Empirical calibration in progress. See §8.

---

## 1. What ORION Measures

ORION is a Citability Intelligence system. It predicts, measures, and diagnoses the probability that a webpage, or a specific unit of content within that page, will be retrieved, extracted, and cited by an LLM-based answer engine when that engine responds to a topically relevant query.

The output is a single composite score between 0 and 100, reported alongside module-level scores for the four underlying measurement dimensions and a set of diagnostics that explain *why* the score is what it is. The composite is calibrated so that a score of 72 means "the model estimates a 0.72 probability that this page is citable on the queries for which it is the most relevant candidate." No arbitrary weight scheme; no theatre.

ORION is not a ranking tool and is not a substitute for traditional SEO analytics. It operates on a different object entirely: the page viewed as a citation candidate for LLM answer engines, not as a search result competing for a SERP position.

---

## 2. The Category, Citability Intelligence

The public web is being split into two retrieval surfaces. One surface is the classical SERP: ten blue links returned by a search engine in response to a query. The other surface is the LLM answer: a synthesized response that silently cites, paraphrases, or omits sources drawn from a retrieval corpus the end user never sees.

The measurement disciplines for those two surfaces are not the same. SEO was built to optimize link-based ranking under keyword relevance and authority signals. The LLM answer surface operates on chunk-level retrieval, semantic coverage, entity resolution, structural extractability, and trust heuristics the model applies post-retrieval. A page can rank highly on Google and still be invisible to ChatGPT. The inverse is also common.

Citability Intelligence is the measurement discipline for the LLM answer surface. Its object of measurement is the **Citation Candidate Unit**, the contiguous chunk of a page that an LLM retrieval pipeline would treat as a single retrievable unit. Its primary signal is the **citation event**, a single (page, query, platform, timestamp, cited-or-not) observation. Its core diagnostic is the **Citability Gap**, the difference between what the model predicts a page should achieve and what the page actually achieves under live LLM probing.

ORION v5 is the first system built natively for this category. Where SEO tooling retrofits keyword-era metrics onto the LLM era, ORION starts from what LLM retrieval pipelines actually do and measures those operations directly.

---

## 3. The Four Modules

ORION v5 decomposes citability into four measurement modules. Each module produces a 0-100 score; the four combine into the composite through a calibrated model (§6).

**ORION-C1, Mechanical Citability.** Can the content be extracted and cited at all? This measures the structural and parsing characteristics that determine whether an LLM retrieval pipeline can surface a specific span of content at all. Signals: chunk length and density, heading hierarchy, structured data (schema.org, JSON-LD), semantic HTML quality, anchor and list structure, crawler accessibility for known AI user agents, and the presence of self-contained declarative statements that work as standalone citations. C1 is an engineering measurement; its fixes are editorial and technical.

**ORION-C2, Informational Value.** Does the content carry enough substance to be worth citing? C2 quantifies per-500-words density of numerical claims, specific named entities (with entity-relevance weighting against the page topic, so a name-drop of an off-topic celebrity contributes near-zero), claim-evidence pairing, primary data presence (with an outbound methodology-artifact requirement so that "proprietary study" claims without a linkable artifact do not count), and the ratio of concrete to abstract language. C1 asks whether the page is *extractable*; C2 asks whether it is *worth extracting*.

**ORION-Q, Query-Cluster Fit.** Does the page match the questions where LLMs actually seek sources? Q generates 60-120 natural-language queries scoped to the page's topical domain, clusters them into 8-20 intent clusters, measures per-cluster answer coverage through a calibrated binary judge, and then tests Fan-out Robustness, the page's coverage under paraphrased reformulations of each query, which approximates the internal query rewriting that LLMs perform before retrieval. A page that covers the user's literal query but collapses on paraphrase is a fragile citation candidate; Q measures exactly that brittleness.

**ORION-T, Trust & Evidence.** Is the content likely to survive the trust heuristics LLMs apply after retrieval? T measures citation quality (do outbound links resolve; do the cited snippets actually support the citing paragraphs), author verifiability (author schema with domain-bound signatures, not just name-dropping), content freshness on topics that decay, domain-level authority signals, and evidence integrity, the ratio of claims that are supported by cited or primary-source evidence to claims that float unsupported. In v5, E-E-A-T is no longer a separate multiplier. The E-E-A-T considerations flow into T as additive features, which removes the double-counting problem that appeared in earlier versions and makes T's contribution to the composite honest and inspectable.

---

## 4. Three Resolutions, Page, Chunk, Query

ORION v5 measures citability at three resolutions, and each resolution is useful for a different operational question.

At the **page** resolution, ORION produces the composite score most users will interact with. This is what shows up on the dashboard and in the API.

At the **chunk** resolution, ORION scores each Citation Candidate Unit independently. This is critical because LLM retrieval pipelines do not embed and cite whole pages; they retrieve specific chunks. A 3,000-word long-form guide will score well at the page level only if it contains several high-scoring chunks; conversely, a concise 800-word post with one exceptional chunk can outperform a bloated guide. Chunk scores let operators see which sections of a page are doing the work and which sections are dead weight.

At the **query** resolution, ORION scores page-to-query fit per intent cluster and per fan-out paraphrase. Operators see which intent clusters a page is strong on, which clusters it is adjacent to but losing to competitor pages, and which clusters it fails on entirely. This is the resolution at which content-strategy decisions actually get made.

Chunk and query resolutions are also the resolutions at which anti-gaming detection runs; a page that looks healthy on aggregate but contains one poisoned chunk gets caught, where a page-only system would miss it.

---

## 5. Predictive ORION and Observed ORION, the Citability Gap

ORION maintains two separate scores per page and treats their difference as its most important diagnostic signal.

**Predictive ORION** is the score produced without live LLM queries. It is a function of the four modules above applied to the page's structural and semantic signals, calibrated against historical observation data. Predictive ORION is what you get back from the API immediately on page submission; it is fast, cheap, and forward-looking.

**Observed ORION** is the score computed from actual live probes against ChatGPT, Claude, Perplexity, and Gemini. It measures what the page actually achieves in LLM answers right now. Observed ORION requires the live probe cycle and is therefore produced on a delayed cadence.

The **Citability Gap** is `Predictive − Observed`. A positive gap indicates a page that the model believes should be citable but is underperforming in practice, usually a signal of a trust issue, a platform-specific indexing problem, or a competitive displacement event. A negative gap indicates a page that is outperforming its structural profile, useful as a positive learning signal, and occasionally a flag that a manual-authority tailwind is compensating for structural weakness.

The gap is not a failure mode of the system. It is the system's most direct diagnostic output. A world in which Predictive and Observed always agree is a world where no learning is possible.

---

## 6. How the Score Is Produced

The composite ORION score is produced by a calibrated probabilistic model, not a fixed weight scheme. The v5 pipeline is:

Feature vectors from C1, C2, Q, and T are computed. At cold-start, those features pass through a regularized logistic regression with priors anchored to published defaults; as labeled observations accumulate, each feature's coefficient transitions automatically from its prior value to a data-driven value through per-feature Bayesian decay. Once the labeled training set reaches the density required for non-linearity to be detectable, the logistic model is superseded by a LightGBM gradient-boosted model with isotonic calibration; the logistic model stays in service as a fallback and as the system's inspectable baseline. Both model families produce calibrated probabilities, and the composite score is exactly that probability scaled to a 0-100 range.

Because the score is a calibrated probability, it has a clean interpretation: over a large cohort of pages scored at 70, roughly 70 percent will be cited by the target platforms on the queries for which they are the most relevant candidates. Calibration error is tracked continuously and published with each release.

Every score ships with a confidence interval computed from three statistically grounded sources: the model's epistemic uncertainty on the feature vector, the aleatoric variance inherent in LLM probe outcomes, and the data-support density around the feature vector (the model is less confident on pages whose feature profile falls outside the dense region of its training data). Low-confidence scores are flagged, not hidden.

---

## 7. The Learning Loop

ORION is a continuous learning system. Priors at bootstrap are scaffolding, not defaults, they carry the model through the cold-start period and are then automatically replaced, per feature, by empirical coefficients as labeled data accumulates.

The loop runs continuously. Predicted scores drive an adaptive probe queue that prioritizes pages whose probing would maximally reduce the model's uncertainty, high-uncertainty pages, stale pages, pages in drifting verticals, and customer-monitored URLs. Probes produce citation observations across the four target platforms; observations produce labeled training rows; labeled rows update the model through time-weighted training (with a half-life on observations so that recent reality counts more than 2024 data); the updated model redistributes uncertainty across the population, and the probe queue shifts accordingly. Three drift detectors, on feature distributions, on observed citation rates, and on prediction-observation error, trigger out-of-cycle retraining when platform behavior shifts materially. Retrains are only promoted to production if they pass a per-vertical non-regression gate; failing retrains are quarantined, and the prior version continues to serve.

The loop is the architectural center of the system, not a maintenance function bolted onto it. "Learning," in ORION, means Bayesian-style reduction of posterior uncertainty through targeted observation, not merely retraining on more data.

---

## 8. Validation and the Honesty Gate

We say what we know and we say what we don't.

ORION v5 is shipped as a **research release**. That label is not marketing softness; it is an engineering state, enforced in code by a boolean flag (`is_research_release`) wired directly to the system's calibration version. The flag is `true` today. It does not flip to `false` until a set of auditable thresholds is met on a reproducible dataset.

Those thresholds are not secret. Before the research-release label can be removed, ORION must reach a minimum of 2,000 labeled training rows from a page frame of at least 3,000 pages drawn under citation-propensity stratification, with at least 300 positive-label rows. The training set must span at least six verticals with no single vertical exceeding thirty percent of rows. All four target platforms must have at least 300 rows each. A held-out temporal test set of at least 500 rows, with a cutoff at least thirty days after training, must achieve PR-AUC ≥ 0.50, ROC-AUC ≥ 0.70, expected calibration error ≤ 0.08 over ten bins, and a top-quintile lift of at least 2.0× significant at p < 0.01.

When those thresholds are met, we publish a reproducible validation report, dataset metadata, model version, coefficient or feature-importance summary, calibration plot, lift table by quintile, per-vertical and per-platform breakdown, and the research-release flag flips in a single auditable event.

Until then, every API response carries `is_research_release: true`. The public methodology (this document) carries it. Marketing materials make no claim of empirical grounding that requires the flag to have flipped. We do not say "validated" until it is validated.

This is the Honesty Gate, and it is the most important single feature of the v5 release. Citability Intelligence as a category cannot be built on claims that outrun the data. Building the system around a gate we cannot bypass is how we intend to earn the category rather than assert it.

---

## 9. Anti-Gaming

Any scoring system that rewards measurable features will attract adversarial content engineered to maximize those features without delivering the substance the features are meant to proxy. ORION's anti-gaming layer is not a principle; it is a set of enforceable detection rules with measurable thresholds and scoring penalties, versioned independently from the model so that rule changes are traceable.

Representative rules: uncorroborated credential claims lose trust-layer credit; numerical claims without adjacent citation, named primary source, or a linkable data page receive a per-claim C2 penalty that accumulates without a ceiling (pages that stuff twenty unsupported statistics are scored accordingly); prompt-injection patterns in page content trigger disqualification; keyword stuffing beyond a statistically anomalous density relative to the reference corpus triggers a sliding C1 penalty; outbound citations are resolved (404s penalized, and for high-scoring pages, every outbound citation is sampled and semantically checked for whether the cited content actually supports the citing paragraph); named entities are weighted by relevance to the page topic and by knowledge-graph resolvability, so that off-topic name-drops and fabricated "institutes" no longer game entity density; first-party-data claims require a linkable methodology artifact to be counted; and novelty without evidence support is penalized, not rewarded, closing the "boilerplate stuffing as original analysis" vector.

Every detection event is logged with a pattern-version stamp, so scoring regressions can be traced to rule changes. This is how the category stays honest over time.

---

## 10. What ORION Does Not Predict

ORION predicts citability. It does not predict content quality in any broader sense. A page can be extractable, substantive, topically fit, and trustworthy on ORION's measurement and still be poorly written, ethically compromised, or factually wrong in ways ORION's probes do not surface. Citability is a necessary condition for LLM visibility but not a sufficient condition for editorial excellence.

ORION also does not predict behavior on closed consumer-grade LLM surfaces that expose no retrieval or citation layer. Models that answer purely from training data without retrieval are outside ORION's measurement frame; ORION measures the retrieval era, not the memorization era.

ORION's predictions are probabilistic and degrade on content types that fall outside the training distribution. Pages in brand-new verticals, pages with unusual structural patterns, and pages on topics with thin historical probe coverage receive lower-confidence scores, and those scores are explicitly labeled as such. We do not hide uncertainty in aggregate.

---

## 11. Public Positioning, What We Claim, What We Don't

**What we claim.** ORION is a Citability Intelligence system. It operates at three resolutions (page, chunk, query) across four measurement modules (C1, C2, Q, T), produces a calibrated probability score with confidence intervals, separates prediction from observation and surfaces the gap between them as a diagnostic, runs on a closed learning loop with documented drift handling, enforces anti-gaming rules as versioned detections rather than slogans, and gates its empirical claims behind an auditable honesty checklist.

**What we don't claim.** We do not claim to be validated until the honesty gate flips. We do not claim that our scores predict broader content quality. We do not claim coverage of LLM surfaces outside of ChatGPT, Claude, Perplexity, and Gemini. We do not claim that citation by an LLM implies factual correctness of the cited content, retrieval and truth are different objects.

---

## 11b. Release Status, What Is Live, What Is Rolling Out

ORION v5 is an architectural release. Implementation is rolling out in phases, and we are explicit about where each component sits today.

**Live today, in production.** The composite ORION score with confidence bounds. The v4 underlying engine continues to serve: structural and semantic features are computed, a 5-pillar breakdown (Originality, Retrievability, Information Extractability, Observed Visibility, Notability) is produced, E-E-A-T is measured as a trust layer, and live probes against the four target platforms generate Observed Visibility. The composite and the module-level breakdowns surfaced in the API and dashboard reflect this engine. The `is_research_release: true` flag is present on every API response.

**Rolling out through Q2-Q3 2026.** The C1/C2 split (Mechanical Citability separated from Informational Value) replaces the single Citability module with two independently inspectable measurements. Chunk-level scoring at the Citation Candidate Unit resolution exposes which sections of a page are doing the citation work. The full ORION-Q pipeline (query generation, intent clustering, Fan-out Robustness) replaces the current Observed Visibility signal with a structured query-fit model. Predictive ORION and Observed ORION split into two fields, and the Citability Gap becomes a first-class diagnostic in the API response. Entity-relevance weighting, graduated citation sampling, first-party-data requirements, and the other v5.0.1 anti-gaming patches come online with the C2 split.

**Gated behind the Honesty Gate flip.** The `is_research_release: false` state, the published validation report, and any public claim of empirical grounding. Until the thresholds in §8 are met on reproducible data, the flag stays `true` and every API response says so.

This is the release trajectory. It is transparent, inspectable, and the flag ships with the product from day one so the reader and the dashboard always agree on what state the system is in.

---

## 12. Using ORION

ORION is accessed through the WhatsMyGeoScore product surface (dashboard and API). Every API response includes the composite score, the four module scores, chunk-level scores for each identified Citation Candidate Unit, query-cluster fit scores, the current Citability Gap, the research-release flag, and the confidence interval. SHAP feature attributions are available per request so that operators can see which signals are driving each score.

The recommended workflow: score a page, read the module breakdown and the chunk-level map to identify which sections are doing the citation work, use the query-cluster fit to identify where the page is fit and where it is adjacent, and use the Citability Gap to decide whether the operational problem is a content issue (Predictive is low) or a trust / platform issue (Predictive is high but Observed is not following).

---

## 13. Glossary

**Citability**, the probability that a content unit will be retrieved, selected, and cited by an LLM answer engine under a topically relevant query.

**Citation Candidate Unit (CCU)**, the contiguous span of content that an LLM retrieval pipeline would treat as a single retrievable chunk. The unit LLMs actually cite.

**Citability Gap**, the difference between Predictive ORION and Observed ORION; ORION's primary diagnostic signal.

**Citability Intelligence**, the measurement discipline for the LLM answer surface; the category ORION operates in.

**Fan-out Robustness**, a page's coverage under paraphrased reformulations of a query, measuring resilience to the internal query rewriting LLMs perform before retrieval.

**Honesty Gate**, the auditable boolean flag that flips ORION from research release to validated, enforced in code by a specific checklist.

**Observed ORION**, the measured citation score computed from live LLM probes.

**Predictive ORION**, the score produced from structural and semantic signals without live LLM probing.

**Research Release**, an engineering state, not a marketing softness. The label is present in the API, in this document, and in internal dashboards until the Honesty Gate thresholds are met on reproducible data.

---

Orion · WhatsMyGeoScore · [whatsmygeoscore.com](https://whatsmygeoscore.com)

*Methodology v5.0, April 2026. Research release. This document supersedes v4.0.*