VerdictTank — Technical Architecture

Standalone technical reference for the VerdictTank v3 platform architecture. This document covers component design, data models, API contracts, pipeline mechanics, and operational procedures for developers and technical evaluators.

1. Overview

VerdictTank is an automated business proposal review platform. A user submits a proposal document; the system runs it through a multi-agent review pipeline and returns a scored, evidence-backed verdict as a branded PDF report, typically within 3 to 15 minutes.

The core thesis is straightforward: a single reviewer opinion is a data point, not a verdict. VerdictTank instead runs a proposal through a sequence of independent review stages built on distinct model architectures, then aggregates their conclusions into a majority-rules decision. The goal is a brutally honest, evidence-grounded assessment that is hard to game and cheap enough to run at scale.

1.1 The Pipeline Concept

Every review passes through three logical phases:

flowchart LR
    A[Proposal Submitted] --> B[Phase 1: Research]
    B --> C[Phase 2: Primary Review]
    C --> D[Phase 3: Parallel Judges]
    D --> E[Majority Verdict]
    E --> F[PDF Report]

• Phase 1 — Research: A dedicated research agent independently verifies factual claims in the proposal (market size figures, competitor claims, pricing comparisons), checks domain and trademark availability, and compiles a citation-backed research brief.
• Phase 2 — Primary Review: A primary reviewer performs a structured, ten-dimension critique of the proposal, informed by the research brief, and produces numeric scores plus qualitative findings.
• Phase 3 — Parallel Cross-Check: Multiple independent judges, each built on a different underlying architecture, review the proposal and the primary reviewer's findings in parallel. Each judge issues its own verdict, confidence level, and list of agreements/disagreements with the primary review.
• Majority Verdict: Verdicts from all judges are aggregated. A simple majority rule determines the final published verdict (Go / Conditional Go / No-Go), with dissent explicitly preserved rather than smoothed over.
• PDF Report: All findings are compiled into an 11-section branded PDF via an automated document generation pipeline.

1.2 v2 to v3 Evolution

The live v2 platform runs five review stages across the three phases described above at roughly $0.47 per run, with a three-judge cross-check panel and a 3-9 minute turnaround. v3 is a proposed upgrade layered on top of that same core pipeline, organized into three enhancement tiers plus an infrastructure prerequisite phase:

Tier	Theme	Additions
Phase 0	Infrastructure prerequisites	Automated sanitization gate, automated PDF generation pipeline, embedding subsystem, corpus schema and backfill
Tier 1	Cumulative intelligence	Review corpus database, prediction-vs-outcome tracking, reviewer accuracy scoring
Tier 2	Deeper analysis	Vertical-specific templates, adversarial red-team per vertical, market simulation engine (Phase 2, sequenced before Phase 2 primary review per §5.4)
Tier 3	Product and distribution	Public-facing shareable reports, review-as-a-service API, competitive corpus comparisons

The net effect is a system that remembers its own history (corpus), learns whether its verdicts were correct (prediction tracking), specializes by industry vertical, and exposes itself as a product surface beyond a single web form.

Pipeline verdict on the v3 proposal itself

The v3 proposal was run through the VerdictTank pipeline prior to build. Verdict: 3/3 Unanimous Conditional Go. Conditions attached to that verdict are reflected throughout the Security Model and Failure Modes sections below (notably around corpus confidentiality and sanitization gate rigor).

Architecture review verdict

This document itself was subjected to a 3-judge Conductor architecture review on 2026-08-10 (Opus 4.8 operational realism, Claude security and failure modes, Gemini Pro completeness and coverage). All three judges returned CONDITIONAL GO. The 18 merged conditions from that review are addressed throughout this revision; see Appendix B: Review Addendum for the full record.

---

2. Component Architecture

VerdictTank is deliberately a small number of well-defined components rather than a large microservice mesh. Each component has a single responsibility and a narrow interface to its neighbors.

flowchart TB
    subgraph Edge
        Caddy[Reverse Proxy — Caddy v2]
        Auth[Auth Gateway — Stack Auth / API Keys]
    end
    subgraph App
        API[Core API Server — FastAPI]
        Orchestrator[Review Pipeline Engine]
        Sim[Market Simulation Engine]
        Sanitize[Sanitization Gate]
        Embed[Embedding Service]
    end
    subgraph Data
        DB[(Corpus Database — SQLite, WAL mode)]
        PDFGen[PDF Generation Engine — WeasyPrint]
        Reports[/reports/ static files/]
    end
    subgraph Scheduled
        Cron[Prediction Tracking Cron]
    end
    subgraph External
        Billing[Stripe Billing]
        Notify[Notification Service]
    end

    Caddy --> Auth
    Auth --> API
    Caddy --> Reports
    API --> Orchestrator
    Orchestrator --> Sim
    Orchestrator --> DB
    Orchestrator --> Embed
    Embed --> DB
    Orchestrator --> PDFGen
    PDFGen --> Reports
    Orchestrator --> Notify
    API --> Billing
    Sanitize -.blocks.-> Reports
    Cron --> DB

2.1 Core API Server

• Stack: FastAPI on Python 3.11+, served via an ASGI worker process.
• Role: Public-facing HTTP interface. Accepts review submissions, exposes polling and retrieval endpoints, enforces authentication and rate limits, and hands work off to the review pipeline engine.
• Responsibilities: request validation, tier entitlement checks, API key scoping, response serialization. Contains no review logic itself; it is a thin contract layer over the orchestrator.

2.2 Reverse Proxy Layer

• Stack: Caddy v2.
• Role: TLS termination, HTTP-to-HTTPS redirection, automatic certificate renewal, request routing to the API process, and direct static file serving for the /reports/ path (completed PDFs and public share assets).
• Why Caddy: automatic TLS, minimal configuration surface, and a config format simple enough to keep under version control alongside the rest of the deploy.
• Directory listing is explicitly disabled on /reports/ — see Security Model, 7.3.

2.3 Review Pipeline Engine

• Role: The orchestrator. Sequences the five-plus agent phases for a given review, manages timeouts per stage, handles degraded-mode fallback when a judge is unavailable, and persists intermediate state so a review can be resumed or inspected mid-flight.
• Design: implemented as a state machine keyed by review_id. Each phase transition writes its output to the review record before advancing, so a crash mid-pipeline loses at most the in-flight stage rather than the whole review.
• Concurrency: Phase 3 judges run concurrently within a single review (not sequentially) since they are independent of one another by design; this is what keeps total turnaround in the minutes range rather than compounding each judge's latency serially. Across reviews, concurrency is bounded by a single-worker queue serialization model; see 2.4 and 5.6.

2.4 Corpus Database

• Stack: SQLite, single file, with the corpus schema described in Section 3. WAL (Write-Ahead Logging) mode is enabled explicitly (PRAGMA journal_mode=WAL;) at database initialization, not left at the default rollback-journal mode. WAL allows concurrent readers (e.g., a poll request per §4.2) to proceed while a writer (e.g., a judge verdict write) is in flight, which is a hard requirement given the read/write mix described below.
• Role: System of record for every review ever run, plus derived tables for predictions, outcomes, and per-judge accuracy scores.
• Why SQLite: review volume at current and near-term projected scale does not warrant a networked database; SQLite gives transactional integrity, zero operational overhead, and trivial backup (file copy) for a workload that is read-heavy and write-light per unit time, provided writes are serialized, see below.
• Concurrent-writer ceiling: SQLite, even in WAL mode, permits exactly one writer at a time; concurrent write attempts beyond that one writer either block or raise SQLITE_BUSY. This is a hard ceiling on the single-file design, not a tunable parameter. The realistic comfort zone for this write pattern (parallel judges within a review, plus cron, plus API writes) is a single concurrent writer per review, with reviews themselves serialized. See 5.6 for how the orchestrator enforces this.
• Single-worker queue serialization: the orchestrator runs as a single-worker process with respect to write-heavy phases. Multiple submitted reviews are queued, not run concurrently against the database. Within one review, Phase 3 judges still run their model inference calls concurrently (the expensive, slow part), but their writes to judge_verdict are serialized through the orchestrator's single write path rather than issued as N simultaneous transactions. This converts "N judges racing to write" into "N judges racing to finish, one write queue," which eliminates the write-contention failure mode described in 8.8 for the common case.
• Growth path: schema is written with an eye toward a future migration to a networked engine. The trigger for that migration is write concurrency, not corpus size — a single SQLite file with WAL mode and single-worker serialization comfortably holds many gigabytes of review history; it does not comfortably hold multiple simultaneous writers hammering it at once. See 5.7 (Corpus Scale Threshold) for the concrete numeric ceiling. The one deliberate exception to "no SQLite-specific extensions" is the embedding similarity mechanism in 2.9, which is called out explicitly rather than left as an implicit contradiction.

2.5 PDF Generation Engine

• Stack: WeasyPrint, driven off an HTML template populated from the structured review JSON.
• Role: Converts the completed review record into the branded 11-section PDF report. See Section 6 for the full pipeline.
• Validation: every generated PDF passes through a post-generation validation script before being written to /reports/ — see 6.3.
• Native dependency and resource constraints: see 10.5 for the required system packages and 8.9 for memory/timeout limits applied to the WeasyPrint process.

2.6 Automated Sanitization Gate

• Role: A pre-deploy content scanner that runs against all output destined for any public surface (shareable reports, OG images, public API responses, docs). It scans for architecture-identifying strings (model/vendor names) and internal infrastructure references, and blocks the deploy if a match is found rather than silently redacting.
• Detection layers: two layers, not one. Layer 1 is string matching against a maintained list of architecture-identifying strings and internal infrastructure references, as in v2. Layer 2 (v3 addition) is behavioral fingerprint detection: a maintained regex pattern bank targeting self-identification phrases common across model families ("as a language model," "my training data includes," "I cannot browse the internet," "my knowledge cutoff is"), capability-boundary language, and training-cutoff date patterns. Layer 2 exists because architecture identity leaks far more often through phrasing patterns than through an accidental vendor name; see 8.12 and 7.5 for the full specification.
• Rationale: the platform's differentiator is cross-vendor architecture diversity; leaking which specific architectures are in the panel undermines both competitive position and the "independent judges" framing that gives the majority verdict its credibility.
• Placement in CI: runs as a required check in the deploy pipeline, not as a runtime filter. A failure here is a build failure, not a logged warning. It also runs per-report at generation time (6.3). See 7.5 and 8.7.
• False-positive handling: the gate remains blocking by default. A documented human-override procedure exists for confirmed false positives; see 8.12 and 7.5.

2.7 Market Simulation Engine

• Role: v3 Tier 2 addition. Given a proposal's stated business model, generates a 12-month simulated trajectory: user acquisition curve, churn projection, and resulting revenue trajectory, rather than relying solely on the static financial figures the proposal itself provides.
• Methodology: agent-driven Monte Carlo simulation. The simulation agent is given the proposal's stated business model, the research brief's sourced market data, and a fixed set of explicitly documented distributional assumptions (e.g., CAC variance bounded by vertical-specific historical ranges, churn modeled as a bounded stochastic process informed by the research brief's market data, not invented ad hoc per run). Each simulation run records its assumption set alongside its output so a reviewer or judge can inspect what the simulation actually assumed, not just what it concluded. If a defensible assumption set cannot be maintained for a given vertical, the simulation is skipped for that review and the Financials dimension proceeds on the research brief and proposal figures alone, flagged as simulation_skipped = true.
• Sequencing: the Market Simulation Engine runs before the Primary Review (Phase 2), not concurrently with it. Its output is packaged as an appendix to the ResearchBrief (a simulated_trajectory field, see 3.2) so the Primary Reviewer consumes it as one more input to the Financials dimension, the same way it consumes verified claims and market data. This resolves the sequencing ambiguity between §1.1's phase diagram and the original placement of market simulation as a Phase 2-adjacent activity; simulation is logically part of Phase 1's research-gathering output, timed to complete before Phase 2 begins.
• Independence: intentionally decoupled from the proposal's own numbers so it cannot simply echo back what was submitted; it is a sanity-check model, not a validator of the proposal's math.
• De-scope condition: if, during Phase 0 build-out, the assumption set for a given vertical cannot be defensibly specified (no credible source for churn/CAC ranges in that vertical), that vertical is de-scoped from market simulation and moves to a Phase 3 backlog item rather than shipping with fabricated placeholder assumptions.

2.8 Prediction-vs-Outcome Tracking Cron

• Role: v3 Tier 1 addition. Every review that contains a flagged prediction (e.g., "this pricing tier will suppress conversion," "this GTM channel will not reach target CAC") is scheduled for automated re-check at T+90, T+180, and T+365 days.
• Mechanics: the cron job queries the corpus for predictions due for re-check, attempts to gather current public signal on the outcome (via the research agent's toolset), and records a best-effort actual-outcome assessment against the original prediction. Runs as a systemd timer, not a bare crontab entry, with OnFailure= wired to a notification unit so a failed run pages the operator rather than failing silently. See Data Model 3.6, Failure Modes 8.4 and 8.13, and 10.6.

2.9 Embedding Service

• Role: v3 Phase 0 addition. Generates the vector embedding stored in corpus_record.embedding (3.5) for corpus semantic search (4.4).
• Model: all-MiniLM-L6-v2 via the sentence-transformers library. Chosen for a small footprint (384-dimension output, runs on CPU without a GPU dependency), which matches the single-server deployment model in Section 10.
• Storage and query mechanism: embeddings are stored as raw float32 BLOBs in the corpus_record.embedding column (SQLite has no native vector type). Similarity search is performed via row-level cosine similarity computed in application code using numpy, scanning the searchable subset of corpus_record rows at query time. This keeps the datastore free of SQLite-specific vector extensions, preserving the migration-portability property claimed in 2.4. At current and near-term corpus sizes (low thousands of rows), a full in-memory cosine scan against numpy arrays completes well within the API's latency budget; this is revisited if/when corpus size or query volume grows past the SQLite migration trigger in 5.7.
• Exception acknowledgment: this is the one place in the architecture that comes close to a SQLite-specific mechanism, since the embedding column and the scan logic are shaped around SQLite's lack of native vector support. It is explicitly documented here as the accepted exception to the "no SQLite-specific extensions" portability claim in 2.4, rather than left as an unstated contradiction. Should sqlite-vec or a similar extension be adopted later for query performance, that adoption must be documented here and the exception restated, not silently introduced.
• Opt-out policy: for reviews with corpus_opt_out = 1, embedding generation is skipped entirely, not generated-then-flagged-non-searchable. The orchestrator checks review.corpus_opt_out before invoking the embedding service; if set, no corpus_record row is created and no embedding is computed. See 3.5, 7.4, and 8.14.
• Deletion on late opt-out: if a customer opts out after a corpus_record row and embedding already exist (opt-out is a mutable setting on an existing review), the embedding BLOB and the corpus_record row are deleted, not merely flagged non-searchable. A deletion is logged to the audit trail described in 7.4.

2.10 Frontend

A thin web application, backed entirely by the API described in Section 4, handles proposal submission and results viewing. It has two primary surfaces: a submission form (proposal text entry, tier selection, vertical override) and a results dashboard (review status polling, verdict display, PDF download link, and, for Enterprise/White-Label accounts, corpus search and accuracy dashboard views gated per 4.4 and 4.5). The frontend holds no review logic and no direct database access; it is a client of the public API contract, which keeps the authentication and tenant-isolation boundaries described below as the single enforcement point regardless of which client (web app, API-key integrator, White-Label embed) is calling in.

2.11 Authentication

Two authentication mechanisms cover the platform's access patterns. Bearer token (JWT) authentication is used for the web application and any session-based interactive use, issued via auth2.itpropartner.com, a Stack Auth project shared with other IT Pro Partner properties. API key authentication is used for programmatic/Enterprise and White-Label integrations, per the scoping rules in 7.2. Both mechanisms terminate at the API server (2.1); the reverse proxy (2.2) does not perform authentication itself, only TLS termination and routing. A request without a valid JWT or API key for a tier-gated endpoint receives 401.

2.12 Tenant Isolation

Tenant isolation is tier-dependent, not uniform. White-Label tier customers get a separate corpus partition per tenant, enforced via a tenant_id column added to review and corpus_record (see 3.1, 3.5); all corpus queries for a White-Label tenant are scoped to that tenant's partition and never cross into another tenant's data or the shared corpus. Enterprise tier is single-tenant by default: an Enterprise account's own reviews are private to that account, though its corpus search can still query the shared aggregate corpus per the aggregate-only rules in 7.4. Pro and Free tiers share the general corpus with search restrictions; individual review content is never exposed to another account regardless of tier, only aggregate metadata per 4.4.

2.13 Billing Integration

Stripe handles subscription billing for Pro, Enterprise, and White-Label tiers. Stripe webhooks (subscription created, updated, canceled, payment failed) drive quota enforcement: a webhook handler updates the account's entitlement record, and the API server's quota check (7.2) reads from that entitlement record rather than calling out to Stripe synchronously on every request. A payment failure webhook flips the account to a grace-period state before hard-blocking new submissions, giving the customer a window to update payment details without an abrupt cutoff.

2.14 Notification System

On review completion, the system sends an email via SMTP with the verdict summary and a link to the PDF report. This is the default notification path for all tiers. Enterprise tier customers may additionally configure a webhook callback: on review completion, the orchestrator POSTs the review summary payload (verdict, verdict_margin, review_id, pdf_url) to the customer-configured URL, with the same retry-with-backoff behavior as the model provider failover described in 5.8, capped at 3 attempts before falling back to email-only notification for that review.

---

3. Data Model

All primary entities below are persisted in the corpus database. Fields marked v3 are new relative to the v2 schema.

3.1 Review Record

The top-level entity representing one submitted proposal and its full review lifecycle.

CREATE TABLE review (
    id                  TEXT PRIMARY KEY,        -- UUID (uuid4, see 7.3)
    tenant_id           TEXT,                     -- v3: White-Label tenant partition, NULL for non-White-Label
    client_name         TEXT,
    proposal_text       TEXT NOT NULL,
    proposal_hash       TEXT NOT NULL,            -- dedup / integrity check, see 5.9
    proposal_word_count  INTEGER,                 -- v3: enforced against 25K word hard cap, see 5.9
    language_detected    TEXT,                    -- v3: ISO 639-1, see 5.9
    pricing_tier        TEXT NOT NULL,            -- free | pro | enterprise | white_label
    vertical            TEXT,                     -- v3: auto-classified (saas, consulting, marketplace, fintech, ...)
    status              TEXT NOT NULL,            -- queued | researching | reviewing | judging | aggregating | complete | failed
    verdict             TEXT,                      -- go | conditional_go | no_go
    verdict_margin      TEXT,                      -- e.g. "3/3 unanimous", "2/3 majority"
    corpus_opt_out      BOOLEAN DEFAULT 0,        -- v3: excludes from corpus search / comparisons / embedding generation
    report_url_expires_at TIMESTAMP,              -- v3: optional expiration, default null = no expiry unless configured, see 7.3
    created_at          TIMESTAMP NOT NULL,
    updated_at          TIMESTAMP NOT NULL,
    completed_at        TIMESTAMP
);

3.2 ResearchBrief

Output of Phase 1. One-to-one with a review.

CREATE TABLE research_brief (
    review_id           TEXT PRIMARY KEY REFERENCES review(id),
    verified_claims      TEXT,   -- JSON array: {claim, source_url, verified: bool}
    discrepancies        TEXT,   -- JSON array: {claim, discrepancy_description}
    competitor_analysis  TEXT,   -- JSON array: {name, url, positioning_summary}
    domain_availability   TEXT,  -- JSON: {domain, available: bool, checked_at}
    trademark_flags      TEXT,   -- JSON array: {mark, jurisdiction, conflict_summary}
    market_data          TEXT,   -- JSON: sourced market size / growth figures with citations
    simulated_trajectory TEXT,   -- v3: JSON from Market Simulation Engine (2.7), appended before Phase 2, null if simulation_skipped
    simulation_skipped   BOOLEAN DEFAULT 0,  -- v3: true if no defensible assumption set for vertical, see 2.7
    citation_count       INTEGER,
    limited_citations_flag BOOLEAN DEFAULT 0,     -- set when web verification degraded, see 8.3
    created_at           TIMESTAMP NOT NULL
);

3.3 CriticReview

Output of Phase 2 (Primary Reviewer). One-to-one with a review.

CREATE TABLE critic_review (
    review_id           TEXT PRIMARY KEY REFERENCES review(id),
    score_name           INTEGER,   -- 1-10
    score_pricing         INTEGER,
    score_pmf             INTEGER,
    score_competition      INTEGER,
    score_financials       INTEGER,
    score_gtm              INTEGER,
    score_risk             INTEGER,
    score_missing_elements  INTEGER,
    score_founder_fit       INTEGER,
    score_overall_verdict   INTEGER,
    fatal_flaws          TEXT,   -- JSON array of strings
    strengths            TEXT,   -- JSON array of strings
    blind_spots           TEXT,  -- JSON array of strings
    verdict               TEXT,  -- go | conditional_go | no_go
    conditions             TEXT, -- JSON array: conditions attached to a conditional_go
    created_at             TIMESTAMP NOT NULL
);

3.4 JudgeVerdict

Output of Phase 3. One row per judge per review (one-to-many with a review).

CREATE TABLE judge_verdict (
    id                   TEXT PRIMARY KEY,
    review_id            TEXT NOT NULL REFERENCES review(id),
    judge_id             TEXT NOT NULL,        -- e.g. "reasoning_verification", "execution_feasibility", "market_reality"
    architecture_family   TEXT NOT NULL,        -- opaque vendor/architecture tag, used only for diversity enforcement
    verdict               TEXT NOT NULL,         -- go | conditional_go | no_go
    confidence             REAL,                -- 0.0 - 1.0
    agreements             TEXT,                -- JSON array: points of agreement with critic_review
    disagreements           TEXT,               -- JSON array: points of disagreement
    novel_insights          TEXT,               -- JSON array: findings not raised by prior phases
    second_order_effects     TEXT,              -- JSON array: downstream consequences the judge flags
    latency_ms              INTEGER,
    timed_out               BOOLEAN DEFAULT 0,
    created_at               TIMESTAMP NOT NULL
);

3.5 Corpus Record (v3)

Indexing/search layer over completed reviews. One-to-one with a review, created only when embedding generation actually runs (see 2.9). No row is created for corpus_opt_out = 1 reviews.

CREATE TABLE corpus_record (
    review_id             TEXT PRIMARY KEY REFERENCES review(id),
    tenant_id              TEXT,       -- v3: mirrors review.tenant_id for White-Label partition scoping, see 2.12
    embedding              BLOB,        -- v3: 384-dim float32 vector, all-MiniLM-L6-v2, see 2.9. Never populated for corpus_opt_out=1
    vertical                TEXT,       -- indexed
    verdict                 TEXT,       -- indexed
    overall_score_avg        REAL,      -- indexed, for percentile comparisons
    summary_snippet           TEXT,     -- v3: fixed-length structural summary only, see 4.4
    searchable                BOOLEAN DEFAULT 1,  -- respects corpus_opt_out; row does not exist at all if opted out at creation time
    indexed_at                TIMESTAMP NOT NULL
);
CREATE INDEX idx_corpus_vertical ON corpus_record(vertical);
CREATE INDEX idx_corpus_verdict ON corpus_record(verdict);
CREATE INDEX idx_corpus_tenant ON corpus_record(tenant_id);

3.6 Prediction / Outcome Record (v3)

CREATE TABLE prediction (
    id                    TEXT PRIMARY KEY,
    review_id              TEXT NOT NULL REFERENCES review(id),
    judge_id                TEXT,             -- nullable; may originate from critic_review instead
    prediction_text          TEXT NOT NULL,
    expected_timeframe_days   INTEGER NOT NULL,  -- 90 | 180 | 365
    check_due_at              TIMESTAMP NOT NULL,
    actual_outcome            TEXT,            -- filled by cron: materialized | not_materialized | inconclusive
    outcome_confidence         REAL,           -- 0.0 - 1.0, cron's confidence in its own assessment
    outcome_notes              TEXT,
    checked_at                 TIMESTAMP,
    created_at                 TIMESTAMP NOT NULL
);

3.7 Accuracy Score (v3)

CREATE TABLE accuracy_score (
    judge_id                TEXT PRIMARY KEY,
    running_accuracy          REAL,     -- fraction of predictions that materialized as flagged
    agreement_rate             REAL,    -- fraction of reviews where this judge agreed with majority
    diversity_score             REAL,   -- inverse of agreement_rate, monitored for consensus drift, see 7.6 and 9.4
    total_predictions_scored     INTEGER,
    last_updated_at               TIMESTAMP NOT NULL
);

Accuracy scoring is read-only until Phase 4

Per the implementation roadmap, accuracy_score is computed and exposed on a dashboard starting in Phase 1, but it does not feed back into judge weighting until Phase 4. This separation is deliberate: it lets the team observe whether accuracy scoring itself is trustworthy (see Failure Modes 8.2, consensus-drift risk) before letting it influence live verdicts. Access to the dashboard itself is restricted to authenticated Enterprise-tier accounts at minimum, per 4.5 and 7.4; read-only status governs when the data affects verdicts, not who can see it.

3.8 Provider Failover Configuration (v3)

Not a corpus table; lives in a version-controlled config file (config/failover.yaml), loaded at orchestrator startup, not in the database. Kept out of the database deliberately so a provider outage can be worked around by an operator editing and redeploying config without a live DB write racing the outage itself.

# config/failover.yaml
roles:
  research_agent:
    providers: [primary_vendor_a, secondary_vendor_b, tertiary_vendor_c]
    retry_count: 3
    backoff_base_seconds: 2       # exponential: 2s, 4s, 8s before escalating to next provider
  primary_reviewer:
    providers: [primary_vendor_a, secondary_vendor_b]
    retry_count: 3
    backoff_base_seconds: 2
  judge_reasoning_verification:
    providers: [primary_vendor_c, secondary_vendor_a]
    retry_count: 3
    backoff_base_seconds: 2
  judge_execution_feasibility:
    providers: [primary_vendor_b, secondary_vendor_c]
    retry_count: 3
    backoff_base_seconds: 2
  judge_market_reality:
    providers: [primary_vendor_a, secondary_vendor_b]
    retry_count: 3
    backoff_base_seconds: 2

Each pipeline role (research, primary reviewer, each judge) has an ordered provider list. On a call failure, the orchestrator retries against the same provider up to retry_count times with exponential backoff, then escalates to the next provider in the role's list. If every configured provider for a role is exhausted, the failure is handled per Failure Mode 8.6 (treated as a single-judge timeout/crash, or a full-panel failure if it cascades below quorum). See 5.8 for the operational walkthrough.

---

4. API Contract

All endpoints are served under api.verdicttank.com (or the equivalent path behind the reverse proxy). Authentication follows 2.11; unauthenticated requests to tier-gated endpoints receive 401.

4.1 Submit Review

POST /v1/reviews
Authorization: Bearer <jwt> | X-API-Key: <key>
Content-Type: application/json

{
  "proposal_text": "...",
  "client_name": "optional",
  "vertical_override": "optional",
  "corpus_opt_out": false
}

Validates tier entitlement and quota (7.2), computes proposal_hash and checks for a duplicate submission (5.9), detects language and word count (5.9), then enqueues the review. Returns 202 Accepted with a review_id and initial status: queued. Duplicate submissions do not re-enqueue; see 5.9.

4.2 Poll Review Status

GET /v1/reviews/{review_id}
Authorization: Bearer <jwt> | X-API-Key: <key>

Returns current status, and once status = complete, the verdict summary and pdf_url. This endpoint is rate-limited independently from the submission endpoint (7.7), both to protect against poll-based abuse and because the polling pattern itself carries a low-severity timing side-channel discussed in 8.15 and 9.5.

4.3 Retrieve Report

GET /reports/{unguessable_id}.pdf

Served directly by the reverse proxy as a static file (2.2). The unguessable_id is not the review_id; it is a separately generated uuid4() value (or os.urandom-derived token) stored on the review record specifically for the public-facing URL, so a leaked or guessed review_id from an authenticated context does not also expose the public report path. See 7.3 for the full security rationale, including the optional expiration window and access logging.

4.4 Corpus Search (v3)

GET /v1/corpus/search?vertical=saas&verdict=go&q=...
Authorization: Bearer <jwt> | X-API-Key: <key>

Available to Pro tier and above, scoped per the tenant isolation rules in 2.12: White-Label queries are scoped to tenant_id, Enterprise and Pro/Free query the shared aggregate corpus with the search restrictions in 7.4. q performs semantic search via the embedding similarity mechanism in 2.9. Results never include full proposal text or full review findings for another account's review; each result includes only vertical, verdict, overall_score_avg, and summary_snippet. summary_snippet construction is strictly limited: it is a fixed-length structural summary composed only from vertical, verdict, and a score-range bucket (e.g., "SaaS proposal, Conditional Go, score range 6-7"). It is never a derivative, excerpt, paraphrase, or embedding-nearest-sentence of the original proposal text. This is a hard construction rule, not a length limit on an otherwise-free-form summary; see 8.14 for the confidentiality rationale.

4.5 Accuracy Dashboard (v3)

GET /v1/accuracy/dashboard
Authorization: Bearer <jwt> | X-API-Key: <key>

Restricted to authenticated Enterprise-tier accounts and above (Enterprise, White-Label). Free and Pro tiers receive 403 on this endpoint; they have no access to per-judge accuracy data, running accuracy scores, or agreement-rate figures. Rationale: per-judge accuracy is competitively sensitive (it effectively ranks the underlying architectures against each other) and is still in the read-only observation period described in 3.7's warning callout.

4.6 Webhook Registration (Enterprise, v3)

POST /v1/webhooks
Authorization: Bearer <jwt> | X-API-Key: <key>

{ "callback_url": "https://customer.example.com/verdicttank-callback" }

Registers the notification callback described in 2.14. Enterprise tier only.

---

5. Pipeline Mechanics

5.1 Phase 1: Research

The research agent has tool access (web search, domain lookup) and produces the research_brief (3.2). It runs first because every downstream phase, including the Market Simulation Engine (2.7, 5.4), consumes its output.

5.2 Phase 2: Primary Review

The primary reviewer consumes the proposal text, the research brief (including any simulated_trajectory), and produces the critic_review (3.3): a ten-dimension score, fatal flaws, strengths, blind spots, and an initial verdict with any conditions attached.

5.3 Phase 3: Parallel Judges

Each judge receives the proposal, the research brief, and the critic review, and independently produces a judge_verdict (3.4). Judges run concurrently against their respective providers; writes are serialized per 2.4.

5.4 Market Simulation Sequencing

As established in 2.7, the Market Simulation Engine runs as the final step of Phase 1, after the research brief's core content is assembled but before the brief is handed to Phase 2. This keeps the phase diagram in 1.1 accurate: simulation is not a parallel, Phase-2-adjacent activity, it is the last research step. A review's research_brief.simulated_trajectory is therefore always either populated or explicitly null with simulation_skipped = true by the time Phase 2 begins.

5.5 Majority Aggregation

Once all judges report (or time out, per 8.6), the orchestrator counts verdicts. A simple majority (2-of-3 in the base panel) determines the published verdict. A split panel is reported as-is (e.g., "2/3 majority, one dissent") rather than resolved by a tiebreaker model; dissent is preserved, not smoothed over, as stated in 1.1.

5.6 Concurrent Review Queue Serialization

The orchestrator is a single-worker process with respect to review execution: one review is actively processed at a time. Additional submitted reviews wait in a FIFO queue rather than being dispatched concurrently against the shared SQLite corpus database. This is the direct operational consequence of the concurrent-writer ceiling described in 2.4: rather than fight SQLite's single-writer constraint with retry loops and SQLITE_BUSY handling scattered across every write path, the design accepts one review in flight at a time and queues the rest. Given per-review turnaround of 3-15 minutes, a shallow queue clears quickly under normal load; queue depth is capped at 50 pending reviews (5.9, 8.13), beyond which new submissions receive 429.

5.7 Corpus Scale Threshold

The practical comfort ceiling for the current single-file SQLite design, combining WAL mode and single-worker write serialization, is approximately 5,000 reviews or 50 GB of corpus data, whichever comes first. This figure is about storage and read-scan performance (particularly the embedding cosine-scan in 2.9, which is a linear scan over searchable rows), not a hard SQLite file-size limit; SQLite itself supports databases far larger than this. The trigger for migrating off SQLite is write concurrency exceeding what single-worker serialization can absorb, not corpus size in isolation; a corpus well past 5,000 reviews with light write volume is less urgent to migrate than a smaller corpus experiencing frequent SQLITE_BUSY errors under the queue model in 5.6.

5.8 Provider Failover Walkthrough

Given the configuration in 3.8: a judge call to its primary provider fails (timeout, 5xx, rate limit). The orchestrator retries the same provider up to 3 times with exponential backoff (2s, 4s, 8s). If all three retries fail, the orchestrator escalates to the role's secondary provider and repeats the retry sequence there. If a tertiary provider is configured for that role (as with the research agent) and the secondary also exhausts its retries, the orchestrator escalates once more. If every configured provider for a role is exhausted, that role's output is treated as a hard failure for the review, handled per Failure Mode 8.6: a single judge failing below quorum falls back to a 2-judge panel with a disclosure note on the report; the primary reviewer or research agent failing entirely halts the review and surfaces a failed status to the customer with automatic notification.

5.9 Edge Case Handling

The pipeline is deliberately explicit about the edges of its input space rather than silently degrading or erroring opaquely:

• Proposals exceeding token limit: proposals are hard-capped at 25,000 words (approximately 37,000 tokens) at submission time. A proposal over that cap is rejected at POST /v1/reviews with a 413 and a clear message stating the cap. This is a hard cap, not a soft warning: chunked processing with summary aggregation was considered and rejected for Phase 1 because it would silently change what "the proposal" means to each pipeline stage; it remains a candidate technique for a future phase if genuinely long-form proposals become common, but is not implemented now.
• Non-English proposals: language is detected at submission time (review.language_detected). Phase 1 supports English only. A non-English proposal is rejected with a clear error message identifying the detected language and stating that English is currently required, rather than being silently run through the pipeline and producing a low-quality or nonsensical review.
• Concurrent review queue: per 5.6, the orchestrator processes one review at a time. The submission queue has a depth limit of 50 pending reviews; a submission when the queue is at capacity receives 429 with a Retry-After hint. Monitoring alerts at 80% of that capacity (40 pending), per 8.13.
• Duplicate submission: proposal_hash (a content hash of the normalized proposal text) is checked at submission time. A match against an existing review returns that review's existing link and status with a "This proposal was reviewed on [date]" notice, rather than re-running (and re-billing for) an identical review.
• Corpus scale threshold: see 5.7. Restated here because it is as much an edge case as a capacity fact: the system's behavior at and beyond that threshold is degraded query latency and elevated SQLITE_BUSY risk, not silent data loss, and the migration trigger is write contention, not size alone.

---

6. PDF Generation

6.1 Report Structure (11 Sections)

#	Section	Content
1	Cover	Client name, proposal title, verdict badge, date
2	Executive Summary	One-paragraph synthesis of the majority verdict and its rationale
3	Score Table	All ten CriticReview dimension scores, tabulated
4	Research	Verified claims, discrepancies, competitor analysis, domain/trademark findings, market simulation summary when applicable
5	Fatal Flaws	Enumerated list from CriticReview.fatal_flaws, cross-referenced against judge disagreements
6	Action Plan	Concrete, prioritized remediation steps derived from fatal flaws and conditions
7	Judge Cards	One card per judge: verdict, confidence, headline finding
8	Judge Notes	Full agreements/disagreements/novel insights/second-order effects per judge
9	Vertical Analysis (v3)	Vertical-specific findings and red-team results, when applicable
10	Disclaimer	Standard liability and methodology disclaimer language
11	Citations	Full source list from the research brief

6.2 Generation Pipeline

flowchart LR
    A[Completed Review JSON] --> B[Template Fill]
    B --> C[Rendered HTML]
    C --> D[WeasyPrint Render]
    D --> E[Draft PDF]
    E --> F[Validation Script]
    F -->|pass| G[Written to /reports/]
    F -->|fail| H[Retry Once]
    H -->|fail again| I[Plaintext Fallback]

1. The completed review record (all tables joined) is serialized to a single JSON payload.
2. That payload fills a Jinja-style HTML template implementing the 11-section structure, with conditional blocks for vertical-specific sections.
3. The rendered HTML is passed to WeasyPrint, which produces the PDF binary directly from HTML/CSS, with no intermediate manual step. This step runs under a memory cap and a hard timeout; see 8.9 and 10.5.
4. The draft PDF is passed through the post-generation validation script before being written to the public /reports/ path.

6.3 Post-Generation Validation Script

Every generated PDF must pass all of the following checks before being served:

• Em dash / double-hyphen scan: the report's prose must not contain em dashes or double hyphens (house style rule enforced automatically, not just at prompt level).
• Section presence check: all 11 (or 10, if vertical analysis does not apply) expected sections must be present and non-empty.
• Page count minimum: the rendered PDF must meet a minimum page count threshold; a report that renders suspiciously short indicates a template fill failure upstream.
• Sanitization scan: the same two-layer architecture-name and behavioral-fingerprint scan used by the deploy-time sanitization gate (Section 2.6, expanded in 7.5 and 8.12) also runs here, per-report, before public write.

A validation failure triggers one automatic retry of the full generation pipeline. If the retry also fails, the system falls back to a plaintext summary (see Failure Modes 8.4) rather than serving a broken or non-compliant PDF.

6.4 Automated vs. Manual Generation

The v2 baseline generated PDFs through a partially manual per-review process. The Phase 0 infrastructure prerequisite scripts this end-to-end: submission to PDF delivery requires zero manual intervention under normal operation. Manual generation remains available as an operator-invoked fallback tool for support cases (e.g., regenerating a report after a template fix), but is not part of the default customer-facing path.

6.5 Resource Constraints on the Render Step

WeasyPrint's HTML/CSS-to-PDF render is the single most resource-intensive step in the generation pipeline, particularly for reports with large tables (Judge Notes, Citations) or long proposal text echoed into the Research section. The render step runs with a hard memory cap and a hard wall-clock timeout, not unbounded; see 8.9 for the specific limits and the fallback behavior when either is exceeded, and 10.5 for the native OS dependencies the render step requires.

---

7. Security Model

7.1 Prompt Injection via Proposal Text

Proposal text is user-supplied free text and is treated as untrusted input throughout the pipeline. The previous framing of this section described the claim extractor as a filter that runs before agent exposure; that framing understated the actual exposure surface. The claim extractor itself is an LLM-driven step and is therefore the first point of exposure to untrusted proposal text, not a pre-exposure filter that keeps injected content away from a model. The claim extractor reads and reasons over the raw proposal text directly. Defense against injection is therefore layered, not front-loaded into a single filtering stage:

• Pre-extraction input sanitization: before the proposal text reaches any LLM, including the claim extractor, a regex-based sanitization pass strips or neutralizes common injection patterns: role-override attempts ("ignore previous instructions", "you are now", "disregard the above"), fake system-turn markers ("system:", "assistant:", "### instruction"), and markdown code-block injection attempts that try to smuggle instructions as fenced code intended for a different rendering context. This is a blunt, pattern-based first pass, not a semantic understanding of intent; it exists to raise the cost of the most common injection techniques, not to guarantee immunity to novel ones.
• Structured parsing before further agent exposure: once past sanitization, the claim-extraction step parses the proposal into discrete structured claims before any downstream agent (primary reviewer, judges) reasons over it as a single blob, reducing the surface for injected instructions to be interpreted as system-level directives by those later stages, even though the extractor itself remains exposed to the raw text.
• Role-boundary reinforcement: every agent's instruction template explicitly frames proposal content as data to be evaluated, not instructions to be followed, and this framing is tested as part of the sanitization gate's broader remit.
• Output re-validation: the sanitization gate and PDF validation script both scan final output for signs that instructions embedded in a proposal leaked into the report's own voice or structure.
• Output behavioral anomaly detection (v3, supplementary signal): beyond string and pattern matching on output, the system tracks statistical outliers in scoring and verdict metadata across the review population: score distributions that deviate sharply from a proposal's vertical/tier peer group, judges converging to unusually high or identical confidence values, or verdict metadata that does not match the shape of a normal review. These signals do not block a review on their own; they queue a review for manual inspection. This is a supplementary, lagging detection layer on top of the preventive layers above, not a replacement for them.
• Red-team testing protocol (Phase 0 deliverable): a documented, repeatable set of adversarial proposal inputs, covering the injection categories above plus proposal-text-embedded attempts to extract architecture identity, internal infrastructure details, or system prompts, is run against the pipeline before Phase 0 is considered complete, and re-run against each subsequent phase that changes agent prompting or the sanitization gate's pattern bank. Red-team results and any newly discovered bypass techniques are fed back into the sanitization pattern bank (2.6, 7.5, 8.12) as part of the false-positive/pattern-update procedure.

This is defense in depth, not a guarantee

No layer above claims to make the claim extractor immune to injected content it is directly exposed to. The combination of pre-extraction sanitization, structured downstream parsing, output re-validation, and anomaly detection is designed to make successful injection difficult and, when it does occur, detectable after the fact, not to make the exposure itself disappear.

7.2 API Key Management

• API keys issued to Enterprise and White-Label tier customers are scoped: a key is bound to a specific account and tier, and cannot access corpus search or endpoints beyond what that tier entitles.
• Each key carries a usage cap matching the account's plan (review count per billing period); the API server enforces the cap at the request layer, returning 429 once exceeded, independent of any downstream billing reconciliation.
• Keys are revocable individually without affecting other keys on the same account, supporting key rotation without downtime.

7.3 Report Access Control

• Completed PDFs and public share assets live under /reports/, served directly by the reverse proxy as static files.
• Directory listing is disabled on this path at the proxy level; a report is only retrievable by its specific, unguessable identifier-based URL.
• Identifier generation: the report URL token is generated via uuid4() or an equivalent os.urandom-derived value, chosen specifically for unguessability (128 bits of entropy, no sequential or timestamp-derived component). It is a distinct value from the internal review_id, per 4.3.
• Optional expiration: a report URL may be configured to expire after a set period, defaulting to 90 days when expiration is enabled for a given tier or account. review.report_url_expires_at (3.1) carries this value; a request against an expired URL receives 410 Gone with guidance to re-authenticate and request a fresh link through the authenticated dashboard.
• Access logging: every retrieval of a report, whether via the public unguessable URL or the authenticated GET /api/verdicttank/review/{id}/pdf path, is logged with timestamp, source IP, and requested identifier. This log is the primary detection mechanism for URL-guessing attempts or unexpected sharing patterns, given that the URL itself carries no authentication.
• Reports for non-Free tiers are not indexed or discoverable; only the customer with the corresponding review_id (and valid session/API key) can request the signed download link via GET /api/verdicttank/review/{id}/pdf.

7.4 Corpus Confidentiality

The corpus is the platform's most sensitive asset: it aggregates other companies' unreleased business proposals.

• Encryption at rest for the corpus database file.
• Per-user opt-out: review.corpus_opt_out lets any customer exclude their review from search indexing and comparisons entirely. Opt-out is enforced at the point of embedding generation itself, not after the fact: for corpus_opt_out = 1 reviews, no embedding is generated and no corpus_record row is created (2.9, 3.5), rather than generating the embedding and merely flagging it non-searchable. A late opt-out on a review that already has a corpus_record triggers deletion of that row and its embedding, logged to this section's audit trail.
• Raw database access model: access to the raw corpus database file itself (as opposed to the API's scoped, filtered views of it) is single-operator, all-or-nothing, stated here as an explicit design assumption rather than left implicit. There is no row-level or column-level access control layer between an operator with filesystem access to the database and the full, unfiltered contents of every review ever submitted, including opted-out reviews' underlying review rows (opt-out removes a review from corpus search and embedding, it does not remove the review record itself, which remains needed for the customer's own report retrieval and billing history). This assumption is acceptable at current operational scale (one operator, one host) and must be explicitly revisited, not silently inherited, if the operator model changes (e.g., additional staff with database access, a managed-service tier with third-party operators).
• Aggregate-only comparisons: competitive comparison features (percentile vs. corpus median) expose only score distributions and vertical classification, never proposal text, to any account other than the review's owner.
• summary_snippet construction rule: as specified in 4.4, the corpus_record.summary_snippet field surfaced through corpus search is a fixed-length structural summary built only from vertical, verdict, and a score-range bucket. It is never generated as a derivative, excerpt, or embedding-nearest-sentence pull from the original proposal text. This closes a specific confidentiality gap: a "helpful" free-text summary generated from proposal content would risk leaking substantive proposal details through the back door of a search result, even with corpus_opt_out respected and encryption at rest in place.
• This was flagged as a named risk in the v3 proposal review (corpus confidentiality liability) and the above controls are the direct mitigation; see Failure Modes for the case where a control fails.

7.5 Sanitization Gate

Covered in detail in 2.6 and 6.3. Security framing: this is the platform's primary defense against leaking architecture-identifying details, behavioral self-identification patterns, or internal infrastructure references into any public-facing surface (shareable reports, OG images, docs, public API responses). It is deploy-blocking, not advisory, precisely because a leak here is a competitive and trust failure that is hard to walk back once a report has been shared publicly.

• Layer 1 (string matching): a maintained list of architecture-identifying strings and internal infrastructure references, as in v2.
• Layer 2 (behavioral fingerprint detection, v3): a maintained regex pattern bank targeting self-identification phrases that recur across model families regardless of the specific vendor name being present ("as a language model," "my training data includes," "I cannot browse the internet," "I don't have the ability to," "my knowledge cutoff is"), training-cutoff date patterns, and capability-boundary language. This layer exists because architecture identity leaks more often through phrasing habits than through an accidental vendor name, and a string-match list alone cannot catch a model describing its own limitations in generic but still identifying language.
• False-positive procedure: the gate remains blocking by default, with no change to that policy. When a block is believed to be a false positive, the procedure is: human review of the flagged content, and if confirmed as a false positive, the specific triggering pattern is added to an allowlist scoped as narrowly as possible (ideally to the exact phrase-in-context, not a broad pattern removal), the gate is re-run to confirm the deploy or report now passes, and a mandatory audit log entry records the pattern, the reviewer, the timestamp, and the justification. This procedure exists so that legitimate content is not permanently blocked by an overly broad pattern, without weakening the gate's default-blocking posture or leaving allowlist changes unaudited.

7.6 API Abuse Protections

• Rate limiting at the reverse proxy and API layers, tuned per tier, with the polling endpoint (GET /v1/reviews/{review_id}, 4.2) rate-limited independently from the submission endpoint (POST /v1/reviews, 4.1). These are different abuse surfaces: submission abuse is about volume and cost, polling abuse is about probing pipeline internals (see the timing oracle note below and in 9.5) and about scraping status data at a rate disproportionate to legitimate dashboard usage.
• Free-tier submission content monitoring: free-tier submissions are monitored for known-good probing patterns, patterns consistent with an account systematically testing pipeline behavior with minor input variations (near-duplicate proposals differing only in specific test phrases) rather than submitting genuine business proposals. This monitoring feeds manual account review, per the anomaly-detection policy below, not automated suspension.
• Anomaly detection on submission patterns: e.g., a burst of near-identical proposal submissions from one account, which could indicate an attempt to probe the pipeline's judge behavior or extract architecture information through differential prompting.
• Abuse detection findings feed into manual account review rather than automated suspension, to avoid false-positive lockouts on legitimate high-volume Enterprise/White-Label usage.
• Timing oracle via polling (acknowledged low-severity risk): because a review passes through observable phase transitions (status field values in 3.1) at different, somewhat characteristic latencies, an attacker polling frequently could in principle infer something about pipeline composition (e.g., roughly how many phases exist, or that a particular phase took unusually long, suggesting a provider failover event per 5.8) purely from timing, without ever seeing pipeline internals directly. This is acknowledged explicitly as a low-severity side channel with no identified practical exploitation path at current scale; see 8.15 and 9.5. It is not actively defended against beyond the rate limiting above, and is called out here rather than left undocumented.

7.7 Training-Data Recursion Prevention

A structural risk unique to this kind of pipeline: if the panel's own review output were ever used, directly or indirectly, to further train or fine-tune models used by the panel itself, judge diversity would collapse over time as the panel converges on its own prior outputs.

• "No training on API usage" contractual requirement: every judge model provider under contract must have an active "no training on API usage" term in its API terms of service or a negotiated data processing agreement to that effect. This is a procurement and contract-management requirement, not a purely technical control, and it is tracked as a standing condition of each provider relationship, re-verified whenever a provider's terms of service change.
• Volume caps on any data pipeline that could plausibly feed review output back toward model training, as a defense-in-depth measure independent of the contractual requirement above.
• Agreement-rate monitoring: accuracy_score.agreement_rate and diversity_score are tracked over time specifically to detect a drift toward artificial consensus (judges agreeing with each other more, and more often, than architectural independence would predict). A sustained upward drift in agreement rate across the panel is treated as an operational signal to investigate, not just a marketing metric.
• Honest limitation: agreement-rate monitoring is lagging detection, not leading prevention. It can reveal that consensus has already drifted; it cannot prevent the drift from occurring, and it cannot fully distinguish organic convergence (the proposal genuinely warrants unanimous agreement) from recursion-driven convergence. Training-data recursion risk is therefore treated as partially inherent to any multi-model pipeline that depends on providers' training practices remaining as represented; the contractual requirement above is the primary control, and monitoring is a backstop, not a solution. This is restated in the Inherent Risks subsection, 8.14.

---

8. Failure Modes & Recovery

#	Failure Mode	Impact	Recovery Procedure
8.1	Single judge timeout or crash	Panel drops to N-1 judges	Orchestrator proceeds automatically if N-1 still meets minimum quorum of 2; review and PDF are flagged as degraded-mode
8.2	All judges fail, or surviving judges fall below quorum	No valid verdict can be computed	Review is marked failed; customer is offered a free re-run at no charge against their quota
8.3	Research agent web verification failure	Citations incomplete or absent	Pipeline proceeds with limited_citations_flag = 1; flag is surfaced visibly in the final PDF's Research section rather than silently omitted
8.4	PDF generation failure (post-validation)	No compliant PDF produced	One automatic retry of the full generation pipeline; if retry also fails, deliver a plaintext summary fallback and flag the review for manual PDF regeneration
8.5	Corpus database corruption	Loss of search, prediction tracking, and accuracy scoring continuity	Restore from the most recent WAL checkpoint (Section 10.4, at most 15 minutes of writes lost); any reviews written between last checkpoint and corruption event are re-derived from review-record source data where still available
8.6	Model/architecture provider outage	One or more judges or the primary reviewer unavailable	Failover chain (3.8, 5.8) routes affected role through its configured provider list with retry and backoff; if every configured provider for that role is exhausted, treat as case 8.1 or 8.2 depending on scope
8.7	Sanitization gate failure (a scan match is found)	Deploy or report publication is blocked	This is by design: the gate blocks rather than warns. Operator must resolve the flagged content (redact/rephrase) before the deploy or report can proceed, or follow the false-positive procedure in 7.5/8.12 if the match is confirmed spurious. Failure is never silently bypassed
8.8	SQLite write contention under concurrent judges	Write attempts raise SQLITE_BUSY or block	WAL mode (2.4) plus single-worker queue serialization (2.4, 5.6) is the primary mitigation; within-review judge writes are serialized through the orchestrator's single write path rather than issued as N simultaneous transactions, which eliminates contention for the common case. A SQLITE_BUSY that still occurs is retried with backoff at the write layer before surfacing as an error
8.9	WeasyPrint memory exhaustion on large reports	Render process killed or hangs, report never produced	Render step runs under a 2GB memory cap (cgroups) and a 120-second timeout; a cap or timeout breach kills the render and triggers the automatic retry described in 8.4, and a second failure falls back to the plaintext summary rather than a partial or corrupted PDF
8.10	Sanitization gate false positive	Legitimate content blocked from deploy or publication	Human review confirms the false positive; the gate remains blocking by default (no change to 8.7 policy). A human override path exists specifically for confirmed false positives, requiring a mandatory audit log entry documenting the override, per 7.5/8.12
8.11	Prediction cron silent failure	prediction.actual_outcome remains perpetually pending, quietly degrading Tier 1 cumulative intelligence with no loud failure signal	Cron runs as a systemd timer (2.8, 10.6) with OnFailure= wired to a notification unit, plus a separate missing-run detection check (a run that should have fired but did not, distinct from a run that fired and errored) that pages the operator on either condition
8.12	Confident-but-wrong judge analysis	A judge issues a high-confidence verdict that is substantively incorrect, with no technical signal distinguishing it from a correct high-confidence verdict	No technical prevention exists for this case; it is addressed as a product disclosure to users (verdicts are probabilistic assessments, not guarantees) rather than a solved engineering problem. Detection is via outlier analysis post-hoc (7.1's behavioral anomaly detection, and accuracy scoring's eventual outcome tracking in 2.8), not at verdict time
8.13	Concurrent review queue saturation	New submissions cannot be accepted	Queue cap of 50 pending reviews (5.6, 5.9); submissions beyond the cap receive 429 with Retry-After; a monitoring alert fires at 80% of capacity (40 pending) so the operator has lead time before the cap is actually hit

Sanitization gate failures are not incidents to route around

If the sanitization gate blocks a deploy or a report, the correct response is to fix the flagged content or follow the documented false-positive procedure, not to disable or bypass the gate to unblock a release. A bypass here directly reintroduces the leak risk the gate exists to prevent.

8.15 Attribution Difficulty in Prediction Tracking

Not a system failure in the crash sense, but a known limitation worth documenting alongside the other failure modes: the prediction-vs-outcome cron's actual_outcome assessment is inherently a correlation judgment, not a causal one. A flagged prediction ("this pricing tier will suppress conversion") that appears to materialize by T+180 may have done so for unrelated reasons. outcome_confidence on the prediction record exists specifically to carry this uncertainty forward rather than presenting the cron's assessment as ground truth; accuracy scoring calculations weight predictions by this confidence rather than treating every checked prediction as a binary hit/miss.

8.14 Inherent Risks

Three risks in this architecture do not have a full technical solution and are documented here explicitly as accepted, ongoing risk rather than as failure modes with a recovery procedure:

1. Training-data recursion is partially inherent. As stated in 7.7, the agreement-rate/diversity-score monitor is lagging detection, not leading prevention, and cannot fully distinguish organic consensus from recursion-driven consensus. The primary control is contractual (7.7's "no training on API usage" requirement), and no fully technical solution exists that guarantees judge independence indefinitely as long as the panel depends on third-party model providers whose training practices cannot be directly audited.
2. Timing oracle via the polling endpoint. As stated in 7.6, phase-transition timing observed through repeated polling could in principle reveal something about pipeline composition. This is assessed as low severity, and no practical exploitation vector has been identified: the information such an analysis could extract (rough phase count, occasional failover events) does not appear to translate into a meaningful competitive or security compromise at this time. It remains documented rather than dismissed, in case that assessment changes as the platform's usage or attacker sophistication increases.
3. Concurrent queue saturation is a capacity design choice, not just a failure mode. The 50-pending-review cap (5.6, 5.9, 8.13) is not an incidental limit that emerged from the SQLite single-writer constraint; it is a deliberate capacity decision for a single-server deployment. Raising it would require either accepting longer queue wait times under load, or the SQLite migration discussed in 5.7, not simply changing a config value. This is stated plainly so that "the queue cap is too low" is understood as a capacity-planning conversation, not a bug report.

---

9. Cost Model

9.1 v2 Baseline

Component	Cost per run
Model inference (research + primary review + 3 judges)	$0.38
Fully loaded (inference + infra amortization)	$0.47

9.2 v3 Full Pipeline

Component	Contribution
Base pipeline (research + primary review)	carried forward from v2 baseline
Specialist judges (up to 7-judge panel)	incremental per additional judge
Market simulation engine	fixed per-run compute cost, incurred once per review ahead of Primary Review (5.2)
Embedding generation (opt-in reviews only)	marginal, per-review; skipped entirely for opt-out reviews (2.9)
Corpus write and indexing	marginal, per-review
Infra amortization	Caddy/API/WeasyPrint/cron overhead spread across run volume
Total v3 full pipeline	$0.86/run

9.3 Per-Tier Economics

Tier	Price	Cost per run	Margin
Free	$0/mo (1 review/mo)	$0.07 (single-reviewer, no full panel)	N/A (loss-leader / funnel)
Pro	$79/mo (20 reviews)	$0.86/run	78%
Enterprise	$499/mo (100 reviews)	$1.05/run	79%
White-Label	from $1,999/mo (unlimited cohort)	$1.05-$1.20/run	75-82%

Free tier deliberately runs a reduced single-reviewer pipeline (no full judge panel, score summary only, corpus percentile teaser) rather than the full pipeline at a loss per unit; this keeps the funnel economically sane while still giving prospective customers a real taste of the product.

9.4 Cost Assumptions and Optimization Levers

• AI cost deflation assumption: underlying inference costs are modeled to decline 15-20% annually based on historical trend, which is factored into margin projections for Enterprise and White-Label tiers over a multi-year horizon. This is an assumption, not a guarantee, and margin models should be re-validated against actual provider pricing at each planning cycle.
• Judge panel cost optimization: judge count is a tunable parameter, not a fixed constant. The system supports running fewer judges for cost-sensitive contexts (e.g., Free tier) and more for Enterprise/White-Label, and the accuracy-vs-cost ratio (accuracy dashboard metrics against per-run cost) is the intended basis for deciding whether panel size should grow further, hold, or shrink for a given tier. The three-judge minimum-viable panel and the seven-judge maximal panel bound this tradeoff space; see 7.6 and 8.2 for the operational floor (minimum quorum of 2).

9.5 Cost and Risk of the Timing Side Channel

Noted here for completeness alongside the cost table, since it is adjacent to per-request economics: the timing oracle risk described in 7.6 and 8.14 has no cost impact of its own (it does not consume additional inference spend), but it is worth tracking whether polling-based probing correlates with elevated API request volume from a given account, since that would show up in per-account infra cost before it shows up as a security incident.

---

10. Deployment Topology

10.1 Single-Server Architecture

The platform runs as a single-server deployment: one host running the API server process, the Caddy reverse proxy, the SQLite corpus database file, and the WeasyPrint PDF generation process. This is an intentional simplicity choice given current scale; there is no distributed consensus, no service mesh, and no multi-region failover at this stage. Provider-level failover (5.8) covers model/architecture outages; it does not cover the host itself.

flowchart TB
    subgraph Host[Single Application Host]
        Caddy[Caddy v2 - TLS + routing + static /reports/]
        API[API Server Process - systemd managed]
        DB[(SQLite Corpus DB - WAL mode)]
        PDF[WeasyPrint Process - memory/timeout capped]
        Cron[Prediction Tracking Cron - systemd timer]
    end
    Internet -->|HTTPS| Caddy
    Caddy --> API
    API --> DB
    API --> PDF
    Cron --> DB
    API -.->|15min WAL checkpoint| S3[(S3-Compatible Backup, encrypted)]

10.2 DNS and TLS

• Public entry point is a standard DNS A/AAAA record pointed at the application host.
• TLS termination and certificate issuance/renewal are handled automatically by Caddy v2; no manual certificate management step exists in the deploy path.

10.3 Build and Deploy Pipeline

• Documentation (this site): built with MkDocs (Material theme) and deployed as static output.
• API and pipeline code: deployed via a standard git-push-triggered pipeline. The sanitization gate (Section 2.6) runs as a required check in this pipeline for any change touching public-facing output paths; a gate failure blocks the deploy outright.
• Deploys to the API layer and deploys to the docs site are independent pipelines and can ship on separate cadences.
• Dependency pinning: requirements.txt pins exact versions with hashes (pip install --require-hashes), so a deploy always installs the exact, previously-verified set of packages rather than resolving against whatever the latest compatible versions happen to be at deploy time.

10.4 Backup Strategy

• WAL-based continuous backup: rather than a nightly-only snapshot, the corpus database's WAL file is checkpointed to off-host, S3-compatible object storage every 15 minutes, matching the pattern used for Hermes's own backup strategy. This bounds worst-case data loss on corruption or host failure to roughly 15 minutes of writes, versus up to 24 hours under a nightly-only scheme.
• A full nightly snapshot is retained in addition to the 15-minute checkpoints, giving a clean daily restore point independent of WAL replay correctness.
• Backup encryption: backups are encrypted with a key separate from and independent of host-level disk encryption. Host disk encryption protects the live system against physical media theft; backup encryption protects the offsite copy against compromise of the storage provider or backup credentials, and the two are not treated as substitutes for each other.
• The corpus is treated as first-class backup content, not an afterthought: it is the platform's accumulated institutional knowledge (Tier 1 cumulative intelligence) and its loss would silently degrade corpus search, prediction tracking, and accuracy scoring without necessarily causing an immediately visible outage.
• Restore procedure for corpus corruption is covered in Failure Modes 8.5.

10.5 Native Dependencies (WeasyPrint)

WeasyPrint's PDF rendering depends on several native system libraries that are not installable via pip and must be present on the host as apt packages: cairo, pango, gdk-pixbuf, libffi, and harfbuzz. These are documented explicitly here because a pip-only dependency list (requirements.txt) will install the WeasyPrint Python package successfully while still failing at render time if these system packages are absent, a failure mode that is easy to miss in a deploy checklist that only checks Python dependencies.

• Font availability: branded PDF reports depend on the client's chosen or the platform's default brand fonts being installed at the OS level and discoverable by WeasyPrint's font stack. A missing font does not hard-fail the render; it silently substitutes a fallback font, which is a correctness issue (broken branding) rather than a crash, and is checked for as part of the post-generation validation script (6.3) rather than assumed correct by default.

10.6 Process Management and Graceful Shutdown

• Systemd unit: the API server and orchestrator run as a systemd service with Restart=on-failure and RestartSec=5, so a crashed process comes back automatically without manual intervention, with a short delay to avoid a tight crash-restart loop against a persistently failing dependency.
• Log rotation: process output is captured via systemd's journald (the default under a systemd unit) with journald's own rotation/retention policy, or via logrotate for any component that writes to flat log files instead. Logs are not left to grow unbounded on the single application host.
• Secrets management: credentials (model/architecture provider API keys, database encryption keys, backup credentials) are supplied as environment variables via a systemd EnvironmentFile, not hardcoded in source or committed to the repository. The EnvironmentFile itself is filesystem-permission-restricted to the service's running user.
• Graceful shutdown on deploy: a deploy sends SIGTERM to the running process rather than a hard kill. The process's shutdown handler stops accepting new review submissions immediately, then drains in-flight reviews already in progress, allowing up to 15 minutes for any review that is mid-pipeline at shutdown time to reach completion before the process actually exits and is replaced. This avoids a deploy silently killing a review that was seconds from finishing Phase 3.

---

11. Operations

11.1 Key Metrics

Metric	Description
Review success rate	Fraction of submitted reviews reaching status = complete vs. failed
API latency (p95)	95th percentile response time for synchronous API endpoints (submission, polling, dashboard)
PDF generation failure rate	Fraction of reviews requiring PDF retry or falling back to plaintext (Section 8.4)
Model/architecture latency	Per-role latency for research, primary review, and each judge, used to catch a specific provider degrading before it causes a full timeout
Panel diversity health	agreement_rate / diversity_score trend across the judge panel over time (Section 7.7)
Corpus growth rate	Reviews indexed per period, used to sanity-check corpus and backup sizing assumptions against the 5,000-review / 50GB comfort ceiling (5.7)
SQLite write contention rate	Count of SQLITE_BUSY occurrences per period (Section 8.8); a rising trend is the leading indicator for the write-concurrency migration trigger described in 5.7, well before size alone would suggest one
Queue depth	Current pending-review count against the 50-review cap (5.6, 8.13); alerts at 80%
Report access anomalies	Count of report retrievals flagged by the access logging in 7.3, used to spot URL-guessing attempts

11.2 Alerting Thresholds

• Review success rate dropping below an agreed operational floor over a rolling window triggers investigation (distinguish between a systemic pipeline issue and a single provider outage per Failure Mode 8.6).
• Any sanitization gate block on a production deploy or report generation attempt (8.7) should notify the operator immediately; this is a security-relevant event even though it is functioning as designed.
• Sustained upward drift in panel agreement_rate beyond a defined band triggers a manual review of judge configuration, per the training-data recursion and consensus-drift concerns in 7.7 and 8.14.
• PDF fallback-to-plaintext events (8.4, 8.9) should alert on any occurrence, not just above a threshold, since they represent a customer-visible degradation of the deliverable.
• Queue depth crossing 80% of the 50-review cap (8.13) should alert with enough lead time to investigate before submissions start receiving 429.
• Prediction cron missing-run or failure events (8.11) page the operator immediately; there is no acceptable silent-failure window for this check given how quietly its failure otherwise degrades Tier 1 intelligence.

11.3 Backup Schedule

• 15-minute WAL checkpoint to encrypted, off-host S3-compatible storage, plus a full nightly snapshot (Section 10.4).
• Backup integrity should be spot-verified on a periodic cadence (e.g., a scheduled restore-to-scratch test), rather than assumed functional purely because the backup job reports success.

11.4 Corpus Health Checks

• Periodic verification that corpus_record entries stay in sync with their source review rows (no orphaned or stale index entries), particularly after any manual corpus maintenance operation.
• Periodic audit that corpus_opt_out reviews are in fact excluded from search, comparison results, and embedding generation entirely (7.4, 2.9), as a direct verification of the confidentiality control rather than trusting the flag's existence alone.
• Periodic audit that summary_snippet values in the corpus (4.4, 7.4) remain structural-only and have not regressed toward containing proposal-text-derived content through a template change.

11.5 Prediction Cron Monitoring

Moved here from its prior placement as a passing note in 11.4, and elevated to its own subsection given the failure mode's severity (8.11): the prediction-tracking cron runs as a systemd timer with OnFailure= wired to a dedicated notification unit, and a separate missing-run detection check verifies that a run that should have fired within its expected window actually did, independent of whether a run that did fire reported success or failure. Both conditions page the operator; there is no code path where this cron can fail silently and go unnoticed.

---

Appendix A: Implementation Roadmap Reference

For planning and evaluation purposes, the v3 build is staged as follows:

Phase	Weeks	Scope
Phase 0	1-3	Sanitization gate (both layers, 7.5), automated PDF generation, corpus schema and backfill, red-team testing protocol (7.1)
Phase 1	4-7	Corpus search, embedding generation policy (2.9), prediction tracking cron, accuracy scoring (read-only, dashboard only, Enterprise-tier access per 4.5/8.16)
Phase 2	8-12	Vertical-specific templates, adversarial red-team per vertical
Phase 3	13-18	New specialist judges (reasoning-verification, execution-feasibility, market-reality), public shareable reports, review-as-a-service API beta, market simulation engine (5.2) if methodology remains defensibly specified per Condition E; otherwise re-scoped to this phase by default
Phase 4	19-22	Live accuracy-based judge weighting, general availability, White-Label pilots

---

Appendix B: Review Addendum

B.1 Review Verdict

On 2026-08-10, this architecture document underwent a structured three-judge review prior to build sign-off. All three judges returned an independent CONDITIONAL GO verdict; the panel result was a unanimous 3/3 Conditional Go, meaning the architecture is approved to proceed to implementation contingent on the conditions below being addressed in this document, which this revision does.

Judge	Model	Focus Area
Judge 1	Opus	Operational realism: deployment mechanics, failure recovery, resource limits, monitoring
Judge 2	Claude	Security and failure modes: prompt injection, access control, corpus confidentiality, abuse protection
Judge 3	Gemini Pro	Completeness and coverage: missing sections, edge cases, specification gaps

B.2 Conditions Addressed

The following 18 merged and deduplicated conditions, consolidated across the three independent reviews, are addressed in this revision:

1. Embedding/vector search subsystem specified: model named (all-MiniLM-L6-v2 via sentence-transformers), storage/query mechanism specified (SQLite row-level cosine similarity via numpy, or sqlite-vec as the documented exception to the no-extensions rule), embedding generation skipped entirely (not merely flagged) for opt-out reviews. See 2.9.
2. SQLite WAL mode and write contention: WAL mode declared explicitly in 2.4; SQLITE_BUSY/write contention added to the failure modes table (8.8); concurrent-writer ceiling and single-worker serialization documented (2.4, 5.6); migration trigger re-diagnosed as write concurrency, not corpus size (5.7).
3. Deployment/production hardening: WeasyPrint native apt dependencies and font availability (10.5); systemd unit with Restart=on-failure/RestartSec=5 (10.6); log rotation via journald/logrotate (10.6); secrets via systemd EnvironmentFile (10.6); graceful SIGTERM drain with a 15-minute completion window (10.6); dependency pinning with hashed requirements.txt (10.3).
4. Backup RPO: nightly-only backup replaced with 15-minute WAL checkpoint to S3, retaining a nightly full snapshot in addition; backup encryption documented as independent of host-level disk encryption (10.4).
5. Market simulation specification: methodology specified as agent-driven Monte Carlo simulation with explicit assumption documentation; sequencing clarified as running before Primary Review, feeding a ResearchBrief appendix rather than running concurrently with Phase 2 (5.2); de-scope path to Phase 3 documented if methodology cannot be defensibly specified (Appendix A).
6. Missing sections added: Frontend (2.10), Authentication (2.11), Tenant Isolation (2.12), Billing Integration (2.13), Notification System (2.14).
7. Edge case handling: token-limit chunking and 25K-word hard cap (5.9), non-English rejection with Phase 1 English-only scope (5.9), queue depth cap of 50 with 429 (5.6, 5.9, 8.13), duplicate submission via proposal_hash (5.9), corpus scale threshold restated as write-contention-driven (5.7, 5.9).
8. Failover configuration: per-role provider list [primary, secondary, tertiary] documented (3.8, 5.8), 3-retry exponential backoff before escalation, configuration location specified as a config table (3.8).
9. Prompt injection defense reframed: claim extractor acknowledged as the first point of exposure, not a pre-exposure filter; pre-extraction regex sanitization added; output behavioral anomaly detection added as a supplementary signal; red-team testing protocol added as a Phase 0 deliverable (7.1).
10. Corpus confidentiality: embeddings skipped (not generated) for opt-out reviews (2.9, 7.4); raw DB access model documented as single-operator, all-or-nothing design assumption (7.4); S3 backups encrypted with a separate key from host-level disk encryption (10.4); summary_snippet construction rules defined as fixed-length structural summary, not a proposal-text derivative (4.4, 7.4).
11. Accuracy dashboard access restricted: 4.5 endpoint restricted to authenticated Enterprise-tier access at minimum; Free/Pro tiers have no access to per-judge accuracy data.
12. Report URL security: uuid4()/os.urandom-based unguessable identifiers confirmed; optional expiration (default 90 days) added; access logging added for all report retrievals (7.3).
13. "No training" API terms: contractual requirement for "no training on API usage" terms added as a standing provider condition; agreement-rate monitoring documented as lagging detection, not leading prevention; training-data recursion acknowledged as partially inherent to multi-model pipelines (7.7, 8.14).
14. Expanded failure modes table: six new entries added covering SQLite write contention, WeasyPrint memory exhaustion, sanitization gate false positives, prediction cron silent failure, confident-but-wrong judge analysis, and concurrent queue saturation (8.8 through 8.13).
15. Sanitization gate enhancements: behavioral fingerprint detection layer added alongside string matching, targeting self-identification phrases, training-cutoff patterns, and capability-boundary language across model families; false-positive procedure (human review, allowlist, re-run, mandatory audit log) added; gate confirmed as still blocking by default (7.5).
16. API abuse countermeasures: polling endpoint rate-limited independently from submission endpoint; free-tier submission content monitoring for probing patterns added; timing oracle via polling documented as an acknowledged low-severity side channel (7.6).
17. Inherent risk documentation: a dedicated Inherent Risks subsection added (8.14) stating plainly that training-data recursion, the polling timing oracle, and queue-saturation capacity limits do not have full technical solutions and are accepted, ongoing risk rather than solvable failure modes.
18. Accuracy dashboard, tenant, and billing sections cross-checked against tier structure: Section 4.5, 2.12, and 2.13 were cross-verified against the pricing tiers in 9.3 to confirm no contradiction between stated tier entitlements and the access restrictions added under condition 11.

B.3 Scope Note

This addendum documents that the above conditions were incorporated into this architecture document as a direct response to the review. It does not itself constitute a fourth review pass; implementation should still be validated against this document during and after each build phase in Appendix A, and the red-team testing protocol (7.1, condition 9) and Phase 0 deliverables should be treated as prerequisites for the Phase 0 exit criteria, not optional hardening.