SignalForge is an independent portfolio prototype with illustrative / demo data — not a validated pharmacovigilance system, and not for clinical decision support. All sample metrics are labelled as illustrative demo data. Interpretations are hypotheses for human review.

SignalForge

Therapeutic area

Moderate-to-severe atopic dermatitis

Illustrative demo data · Checking persistence…

Checking session…

Edit onboarding

Decision Records

Architecture decisions that changed the product

Each ADR records context, options, the decision, and the eval cases that justify it. Re-run fixtures from the Evaluation centre.

  1. ADR-001

    Deterministic matcher before hosted LLM

    accepted · 2026-02-10
    Context
    Ask Signal needed grounded answers with abstention, but a hosted LLM without retrieval would invent citations on the demo corpus.
    Options
    • Ship a live LLM with prompt-only grounding
    • Ship a deterministic corpus matcher with honest labelling
    • Block Ask Signal until hybrid RAG + evals pass
    Decision
    Ship the deterministic matcher with explicit “not a live LLM” labelling, and gate any LLM behind hybrid retrieval + CI evals (FR-01–FR-05).
    Consequences
    • Hiring reviewers see working abstention/citation behaviour today
    • AI claim remains partial until embeddings + model key land
    • Eval harness can already fail closed on injection fixtures
  2. ADR-002

    Server-side permission merge on whole-state PUT

    accepted · 2026-10-20
    Context
    PUT /api/state checked only prefs:write, so an analyst session could overwrite brief approval state and bypass briefs:approve.
    Options
    • Remove PUT entirely and rewrite the client to per-entity routes in one pass
    • Keep PUT but authorize each mutation class against the existing permission matrix
    • Rely on UI role chips alone
    Decision
    Authorize PUT with a server-side diff (authorizeStateWrite): approval, safety, evaluation, backlog, and change-control each require their own permission; unsupported-claim gate stays server-enforced.
    Consequences
    • Critical A1 closed without a full client rewrite
    • Per-entity routes remain the longer-term NFR
    • Analyst can no longer approve via document overwrite
    Eval evidence
    eval-unsup-01
  3. ADR-003

    Admin reset password from environment only

    accepted · 2026-10-20
    Context
    Published admin-demo credentials let any visitor wipe the shared Neon demo a hiring panel would open.
    Options
    • Remove admin account entirely
    • Keep published password but rate-limit harder
    • Require ADMIN_DEMO_PASSWORD from env; disable admin login when unset
    Decision
    Admin login succeeds only when ADMIN_DEMO_PASSWORD is set in the deploy environment. README publishes analyst/reviewer/PO only.
    Consequences
    • Shared demo is safe by default (A2)
    • Operators must set the env var to reseed
    • Guest browse remains read-only until a demo login
    Eval evidence
    eval-safety-01
  4. ADR-004

    Record-level deep links required for verified sources

    accepted · 2026-10-20
    Context
    Eleven of twelve sources pointed at site homepages while assets were marked verified — a false provenance signal.
    Options
    • Keep homepage URLs and soften “verified” language in UI only
    • Require PMID/NCT/EPAR/label identifiers for verified_source rows + CI lint
    • Drop verification badges entirely
    Decision
    Replace homepage URLs with record-level deep links; add amlitelimab and rocatinlimab; fail data lint when verified rows lack identifiers (FR-06).
    Consequences
    • Provenance badges become inspectable
    • Corpus refresh needed when identifiers age out
    • Fictional SIGFOR rows stay unverified without URLs
  5. ADR-005

    Measured fixture evals separate from illustrative targets

    accepted · 2026-03-01
    Context
    Evaluation centre showed all cases as “not run”, undermining the claim that SignalForge owns evaluation strategy.
    Options
    • Leave cases as documentation only
    • Run deterministic fixtures and store measured pass/fail with model/prompt/corpus labels
    • Claim ≥90% citation correctness without running anything
    Decision
    RunEvaluationHarness executes fixtures in UI and CI; targets remain labelled illustrative; measured runs never claim production SLAs (FR-04/FR-05).
    Consequences
    • Reviewers see real pass/fail against the seeded corpus
    • Misses get root-cause nextAction strings
    • Live LLM evals still pending provider choice
  6. ADR-006

    Neon Postgres over SQLite for the shared demo

    accepted · 2026-10-19
    Context
    Vercel serverless cannot keep a durable SQLite file; live /api/state returned 500 with missing DATABASE_URL.
    Options
    • In-memory only demo mode
    • Neon PostgreSQL via Prisma with pooled + unpooled URLs
    • Turso / libsql
    Decision
    Prisma 6 + Neon PostgreSQL; seed refuses to wipe populated data; health probe reports DB readiness.
    Consequences
    • Refresh persistence works for signed-in demo sessions
    • Secrets must stay in Vercel env (rotate if pasted)
    • Migrations required before new entity shapes
    Eval evidence
    eval-meta-01