Reliability

What we measure, and what happens when a source fails

The audit engine pulls from 10 third-party APIs, 5 in-house parsers, and 8 in-process probes (23 total sources). Each external provider has its own quota, throttling policy, and failure mode. We never invent a number to cover a gap — when a source is unavailable, the audit says so. This page is the public receipt for that contract.

Operational targets

Audit completion

99.5% / 30 days

Percentage of audit jobs that finish within their 12-minute outer watchdog. Tracked via the engine's own `processingTimeMs` field; published weekly on the /transparency page.

AI explanation grounding

100%

Every AI-written explanation we ship is post-validated against the input JSON via the constrained schema parser (whatChanged contains a real number/tech/category, citation key exists in the data). Anything that fails validation is silently discarded — the audit shows the verified-fix block alone, with no AI prose.

Verified fix coverage

> 90% of common findings

Our fix library currently covers 25 signal IDs that account for >90% of real audit findings. Long-tail signals fall through to AI-prose-only mode, clearly labelled. We add new fixes as we see them in production.

Provider-failure recovery

Fail-silent on 100% of provider errors

Every provider returns null on any failure (rate-limit, timeout, malformed response, network error). The orchestrator never blocks the audit on a single provider, never invents data to fill a gap, and surfaces "unavailable" status in the data-sources panel.

Per-provider reliability

For each provider: the quota or rate limit, how we throttle our own calls, and exactly what the audit looks like when the source is unavailable.

Google PageSpeed Insights (PSI)

Performance + Core Web Vitals

Public docs

Quota: 25,000 requests/day per API key. We rotate across 4 keys (100,000/day combined).
Our throttling: Google enforces a soft per-second limit (~5/s/key). Our circuit breaker opens after 3 consecutive 429/5xx responses on a key, then routes to the next key in the pool. Competitor benchmarking (POST /api/competitor-benchmark) is rate-limited to 1 request per IP per 10 minutes because each call burns 10 PSI requests (2 URLs × 5 runs).
Typical duration: 15-90s (5 runs × 2 devices × ~3-9s each, parallelised)
What the audit looks like when this fails: When all 4 keys are exhausted or returning errors, the audit completes WITHOUT a Lighthouse score. The Performance category reverts to structural-only signals (probe-derived). The audit shows a banner: "PSI temporarily unavailable — Lighthouse measurements skipped this run."

Chrome UX Report (CrUX)

Real-user field data

Public docs

Quota: Pulled from PSI as a sub-feed; same 25,000/day quota applies. CrUX has no separate limit.
Our throttling: Inherits PSI throttling.
Typical duration: Free with PSI call
What the audit looks like when this fails: CrUX is only available for sites with sufficient real-user traffic. When CrUX has no data for a URL (small sites, new sites, intranets), we say so explicitly: "CrUX field data unavailable for this URL — only Lighthouse lab data shown." We never substitute lab numbers for field numbers.

Mozilla Observatory

Security headers

Public docs

Quota: Free, no API key. Mozilla rate-limits to ~5 fresh scans/minute/IP.
Our throttling: On rate-limit (429), Observatory returns a cached result if available, otherwise null. We respect the rate limit and queue requests when needed.
Typical duration: 3-8s
What the audit looks like when this fails: When Observatory is unavailable, the Security category reverts to header-presence checks performed by our own probe. The grade letter (A/B/C) is omitted; users see "Mozilla Observatory: temporarily unavailable" in the data-sources panel.

W3C HTML Validator

HTML standards compliance

Public docs

Quota: Free, no API key. W3C asks for one request per second per IP.
Our throttling: We hold to that limit by serialising W3C calls within a single audit. On 503/timeout, we silently skip and note it.
Typical duration: 5-10s
What the audit looks like when this fails: When W3C is unavailable, the HTML-validation panel is omitted. We never claim a page passes HTML validation without seeing the W3C response.

SSL Labs

TLS configuration grade

Public docs

Quota: Free, no API key. SSL Labs caches results for ~24 hours and rate-limits fresh scans aggressively (one scan per IP per host every ~10 minutes).
Our throttling: On a rate-limit response, we return null. SSL Labs scans are slow (1-3 minutes for a fresh run); we ONLY use cached results to keep audits fast.
Typical duration: ~1s on cache hit, null on miss
What the audit looks like when this fails: When no cached SSL Labs result exists, the TLS grade panel is omitted. Users see "SSL Labs: cache miss — re-run in a few minutes for a fresh grade." No invented grade.

Google Safe Browsing

Malware / phishing

Public docs

Quota: 10,000 requests/day per API key. Free tier.
Our throttling: Negligible — we use one lookup per audit.
Typical duration: <1s
What the audit looks like when this fails: When the API is unavailable, the Safe Browsing badge is omitted from the report. We never claim a site is "safe" without a positive Google response.

Website Carbon

Sustainability

Public docs

Quota: Free, no API key. Website Carbon does not publish a public rate limit.
Our throttling: We treat 429/5xx as null and move on. Empirically we see <0.1% failure rate.
Typical duration: 5-10s
What the audit looks like when this fails: When unavailable, the Sustainability panel is omitted. The overall score is unaffected — sustainability is informational and does not feed the score formula.

axe-core (Deque)

Accessibility (WCAG 2.1 AA)

Public docs

Quota: In-process — runs locally inside our server via jsdom. No external quota.
Our throttling: Latency is bounded by the size of the rendered HTML. We cap analysis at 15 violation entries per page to keep payloads small.
Typical duration: 300-800ms
What the audit looks like when this fails: axe runs on the static DOM only — dynamic interactions (focus traps, ARIA-state changes triggered by JS) are not tested. We say so explicitly in the violation list. If jsdom fails to parse the HTML, the Accessibility category reverts to probe-derived structural signals.

On-page SEO checks

Content / SEO (NEW in v8.0)

Public docs

Quota: In-process — pure regex on the rendered HTML. No external quota.
Our throttling: Always finishes in <50ms.
Typical duration: <50ms
What the audit looks like when this fails: When the probe failed to capture rawHtml (network timeout during probe), the content-SEO provider returns null and the on-page checks panel is omitted. No invented findings.

Schema.org / JSON-LD validation

Structured data / Rich Results (NEW in v8.0)

Public docs

Quota: In-process — extracts and parses <script type="application/ld+json"> blocks. No external quota.
Our throttling: Always finishes in <50ms.
Typical duration: <50ms
What the audit looks like when this fails: When rawHtml is missing the provider returns null. We validate against a hard-coded table of ~17 types that drive 95%+ of real Google Rich Results eligibility (Article, Product, Recipe, FAQPage, Event, Organization, etc.). For @types outside this table we emit "type not validated" rather than inventing pass/fail.

The four honesty rules

Fail silent, not loud. A provider that fails returns null. The audit completes; the affected panel is omitted; the data-sources list shows "unavailable". We never block the whole audit on one source.
Never invent. If we don't have a real measurement, we don't ship a number. Cells are empty, panels are omitted, scores are recomputed without the missing source. The badge "UNVERIFIED" appears next to anything that didn't trace to a populated upstream field.
AI prose is post-validated. Every AI-generated explanation is parsed against a strict JSON schema; the citation must reference a real input field; whatChanged must contain a real number, a detected-tech name, or a category keyword. Anything that fails the check is silently discarded — the verified-fix block stands alone.
Verified fixes own the code. The library at fix-library.ts is human-curated. The model writes the prose; the library writes the code. We never ship an LLM-generated code snippet as the canonical fix.