Skip to content

KPIs

The measurement contract for 961tech. Every metric here is paired with what it is, where it's collected, target value or range, cadence, and the tickets it informs. The ordering matters: north-star is the one number that, if moving in the right direction, means the project is working; secondary metrics decompose it; health metrics tell us whether the data we're acting on is trustworthy.

Produced for Foundation: KPIs + observability + performance budget (#43). The observability stack that collects these metrics is decided in RFC-0007. Per-surface latency / weight budgets that some metrics police live in performance-budget.md.

1. North-star

One metric, one row. Everything else exists to explain its movement.

KPI Definition Source Target Cadence Informs
Weekly Qualified Outbound Clicks (WQOC) Distinct visitors per ISO week who clicked at least one outbound retailer link via /api/go/r/[retailerId]/p/[listingId]. "Qualified" excludes self-referrer reloads, bot UAs, and duplicate clicks within a 5-min window per (visitor, listing) tuple — these are filtered server-side before insert into the Click row. Server-side: existing Click table + Workers Analytics Engine writeDataPoint per click M1 ramp: any reading > 0; M2 floor: 50 WQOC; M3 stretch: 500 WQOC Real-time append, weekly rollup #15, #28, #41, #43

Why this metric. The aggregator's reason-for-existing is matching a Lebanese buyer to a Lebanese retailer. The clickout is the moment value is delivered: everything before is intent, everything after happens at the retailer. Distinct visitors per week captures that we're serving a population, not the same enthusiast hitting refresh. Qualified filters bots and impulse double-clicks so the number reflects intent, not noise. Weekly smooths daily traffic spikes (Black Friday banners, FB-group share waves) without hiding multi-week trends.

Why not "raw outbound clicks." A single Karim doing a 9-part build generates 9 clicks. Counting raw clicks weights one shopper as nine; counting distinct visitors weights one shopper as one. WQOC asks "how many people did the site help this week?" — that's the question the project's existence answers.

Why not "build sessions completed." Builds are the Builder track only. Casual customers (per personas §3.3) never touch the Builder and would be invisible. WQOC counts both tracks in one number.

Why not "GMV" or "estimated affiliate revenue." Inferred from outbound clicks via attributed conversions, none of which exist at M1 (#17 affiliate postback is M2+). WQOC is a directly-measurable proxy until postback wires up.

Open question for MASTER. WQOC presupposes that visitor reach is the priority over conversion quality. If MASTER prefers a quality-first north-star — e.g. median match-rate per ingested listing, where catalog accuracy outranks growth — say so before RFC-0007 reaches Accepted. See § Open questions.

2. Secondary metrics — track-specific decomposition

Each track has 3-5 metrics that explain WQOC movement for that track. Tracks mirror personas.md §6 ticket implications.

2.1 Builder track (Karim, First-time builder)

KPI Definition Source Target Cadence Informs
UC-9 build sessions started Count of distinct sessions that loaded /build and selected ≥1 component slot Server-side: route handler instrumentation → Workers Analytics Engine M1: any reading > 0; M2: 200/wk Daily rollup, weekly headline #7, #28
UC-C builds saved Count of Build rows persisted (signed cookie or authenticated user, per ADR-0003) Server-side: DB count ≥30% of started sessions reach saved Daily #13, #28
UC-D builds shared Count of public build URLs accessed by a different visitor than the creator Server-side: /builds/[id] hit logged with creator-vs-visitor cookie diff ≥10% of saved builds get ≥1 external view Weekly #13, #9
Compat-warning effectiveness (UC-B) Ratio: (times user changed selection after warning) / (times warning was shown). Inverse: ignore-rate. Server-side: warning shown event + next-action event in same session Change-rate ≥ 60%, ignore-rate ≤ 40% Weekly #22, #28 §6.1.1
Build → outbound click conversion % of build sessions that produce ≥1 qualified outbound click Server-side: session-id correlation between build and /api/go/... M1: track baseline; M2 target: ≥40% (per personas §3.1 Karim validation) Weekly #28, #43

2.2 Casual track (Casual parts customer; gated by ADR-0005)

KPI Definition Source Target Cadence Informs
UC-13 product-detail views Count of /products/[slug] pageviews from non-builder sessions (no /build visit in same session) Cloudflare Web Analytics + server-side session classifier M1: track baseline; M2: ≥30% of total pageviews Daily #28 §6.1.3
UC-8 retailer comparisons surfaced Count of product-detail views where ≥2 retailer rows rendered (Listing for the same Product across retailers) Server-side: count of listings rendered per pageview event ≥70% of product-detail views show ≥2 retailers (proves matcher is doing its job) Daily #28, #29
Casual → outbound click conversion % of casual sessions that produce ≥1 qualified outbound click Server-side: session classifier + Click join M2: ≥25% (lower than Builder because casual is browse-mode) Weekly #28, #41
WhatsApp deep-link click rate % of outbound clicks that go to a WhatsApp deep link rather than a direct retailer URL Server-side: outbound URL parse ≥20% (per personas §7 cross-cutting validation target) Weekly #28 §6.1.2

2.3 Aggregator-core (catalog truth)

These police the data quality the rest of the product depends on. Movement here is what makes WQOC achievable; degradation here invalidates everything downstream.

KPI Definition Source Target Cadence Informs
Match rate % of Listing rows with productId IS NOT NULL and matchConfidence ≥ threshold (per architecture/ingest-pipeline.md) DB query, per category M1: CPU/GPU ≥ 90%, RAM/Storage/PSU/Cooler ≥ 70% (LLM-extraction in #21 lifts the laggards) Daily after each scrape #21, #22, #33
Price freshness Median age (in hours) of latest ListingPrice.scrapedAt per Product per category DB query Median ≤ 24h, P95 ≤ 48h Hourly #18, #20
Retailer coverage Count of retailers in active scraper roster vs target roster size Static config + DB count M1: 3/3; M2: 6-8 per retailers.md §6 Weekly #20
Listings per retailer Total Listing rows per retailer; week-over-week delta DB query No retailer drops > 30% week-over-week without explanation Daily, weekly trend #18, drift detection
Three-state stock distribution % of listings classified Listed / Call-For-Price / Out across the catalog DB query (latest ListingPrice per Listing) Monitor — no fixed target; sudden swings (e.g. all-Out for a retailer) are drift signals Daily #3

2.4 Health (system + scraper trust)

The metrics that say "the data above is real."

KPI Definition Source Target Cadence Informs
Scraper success rate per retailer % of scheduled scrape jobs that completed without exception, per retailer per day pg-boss pgboss.job table queries (per RFC-0002) ≥ 95% per retailer per 7-day rolling window Daily #18, #19
Failed-to-match rate trend Week-over-week delta of (% Listings with productId IS NULL) per category DB query Drift signal: any category jumps > 5pp WoW (suggests retailer changed listing format) Weekly #21
Catalogue churn Net Listing count delta per retailer per day (new listings − listings unseen for 7+ days) DB query Monitor — sustained negative churn for any retailer signals catalog shrinkage or scraper drift Daily #18
Worker error rate % of Worker invocations that returned a 5xx or threw an uncaught exception Cloudflare Workers Observability + Sentry events < 1% of invocations Real-time alerting threshold RFC-0007, #19
Worker P95 latency P95 wall-clock time per Worker invocation Cloudflare Workers Observability < 600ms TTFB-equivalent (matches performance-budget.md backend budget) Hourly performance-budget.md, #28
Email delivery health Per #14 once price-drop alerts ship: send-rate, bounce-rate, complaint-rate from the email-provider API Provider-side dashboard + Workers webhook for events Bounce < 5%, complaint < 0.1% Daily #14
Security-relevant alert hygiene Count of CSP violations, rate-limit triggers, suspicious-pattern flags per day (M2 once #11 / #12 / #44 ship) Cloudflare WAF + Workers logs + Sentry Baseline-and-watch; sustained step-changes investigate Daily #44

3. What we measure at launch vs later

M1 (ship-now, the "ten things")

The minimum viable measurement set. All of these run on the Cloudflare-native + Sentry-Free + Axiom-Free stack picked in RFC-0007, at $0 incremental observability spend.

  1. Weekly Qualified Outbound Clicks — north-star
  2. UC-9 build sessions started
  3. Build → outbound click conversion
  4. Match rate (per category, daily)
  5. Price freshness (median hours, hourly)
  6. Scraper success rate per retailer (daily)
  7. Worker error rate (real-time)
  8. Worker P95 latency (hourly)
  9. Pageviews + Core Web Vitals (Cloudflare Web Analytics, free, automatic)
  10. Cron execution history (Cloudflare Workers dashboard, free, automatic)

If a metric isn't on this list, it's not gated for M1 launch.

M2 (after first product feedback loop)

  1. UC-C builds saved + UC-D builds shared
  2. Compat-warning effectiveness (UC-B)
  3. UC-13 product-detail views (after Casual flow ships)
  4. Casual → outbound click conversion
  5. WhatsApp deep-link click rate
  6. Failed-to-match-rate trend per category (drift detection)
  7. Email delivery health (#14)
  8. Persona-anti-pattern drift signals (per personas.md §5.7 — crypto rigs, wholesalers, used-only, Apple-only, phone-only, Arabic-cohort)

4. Cross-references

  • #19 hosting / deploy ← KPIs #4-#10 are collected by infra picked in RFC-0001. The two RFCs are complementary: 0001 picks where things run, 0007 picks how we see them running.
  • #18 scraper queue ← Scraper success rate, price freshness, catalogue churn all read from the pg-boss tables that ticket creates (RFC-0002).
  • #14 price-drop alerts ← Email delivery health is the operational metric for that ticket; tracked via the email provider's webhook back into Workers Analytics Engine.
  • #28 page design ← Builder + Casual track KPIs are the design contract; per-page LCP / CLS / INP land in performance-budget.md.
  • #29 DB / queries ← Worker P95 latency target presupposes the per-query budget envelope in performance-budget.md §3 (product-list < 100ms p95, search < 300ms p95).
  • #44 security review ← Security-relevant alerts (CSP violations, rate-limit triggers, WAF events) are the M2 health row above; that ticket scopes which alerts pass the "wake someone up" bar.
  • #43 this ticket ← driver.

5. Open questions for MASTER

These need a call before RFC-0007 reaches Accepted.

  1. North-star confirmation. WQOC assumes growth-quality balance — visitor reach over per-visitor depth. If MASTER prefers a quality-first metric (e.g. median match-rate per category, or per-visitor "comparable retailers shown"), that changes the secondary tier. Default kept as WQOC because it's the most defensible single number for an aggregator's reason-for-existing.
  2. WQOC qualifying rules. Current draft: dedupe within 5 min per (visitor, listing), exclude bot UAs, exclude self-referrer reloads. Tightening (e.g. require ≥10s on the source page before the click counts) is plausible — let's start loose and tighten if signal is muddy.
  3. Telemetry retention. Workers Analytics Engine is effectively unbounded but slow to query at depth; proposing 90-day rollups stored in Postgres, raw events kept 30 days in AE. Confirm before RFC-0007.
  4. Alerting destination. Default in RFC-0007 is Cloudflare email Notifications. MASTER's brain has a telegram-notify MCP — do we wire it as the primary alert channel? Personal-vs-project notification mixing is the trade-off; left as opt-in by default.

6. See also