Site Updates
Ongoing improvements and fixes, in the order they shipped. For the current state of known gaps and open questions, see Issues.
- Complete
AI assessment crawler reliability improvements
The background AI assessment crawler (see July 24 entry below) received a round of reliability fixes. It previously slept a fixed 60 seconds between every item regardless of outcome, which alone exceeded the free tier's daily quota — once exhausted, it just kept retrying at the same pace with no timestamps to show when. It now paces itself under the daily cap, backs off exponentially (bounded, using the AI provider's own suggested retry delay when present) on repeated rate-limit errors, and timestamps every log line, including successes, so a healthy run stays distinguishable from a stuck one. It also now stops itself right before a Run All swaps in a new database and restarts fresh afterward — its pagination only moves down through Combined score, so without this, any hypothesis a Run All introduced or rescored above the crawler's current position was invisible for the rest of that run (already-assessed hypotheses are skipped with no extra API calls, so the restart is cheap). Finally, a stale "running" status left behind by an unclean restart (crash, reboot, service restart) — which previously blocked new runs from starting since nothing was left to update it — now clears itself automatically, mirroring the same fix already applied to Run All's own status tracking.
- Complete
Fixed filters resetting on back-navigation, added AI verdict filter
Landing on a URL like /?p=3 — via the browser back button or a shared link — silently reset the page to 1 and stripped the query string 350ms later, regardless of whether any filter had actually changed. The underlying filter-sync effect was firing unconditionally on every mount; it's now guarded by comparing against the last-seen filter values, robust to React Strict Mode's double-invoked effects defeating a simpler one-shot guard. The synced URL is now also mirrored into session storage and restored from there if the query string comes back empty, since the router's own back/forward restoration drops it independently. Separately, the AI column now has a Positive/Negative/Unclear filter, matching the existing Sources column filter pattern.
- Complete
Hypothesis detail page: Scores/Polarity/AI Assessment moved into tabs
The right-hand panel on each hypothesis detail page stacked Scores, Polarity, and AI Assessment vertically, pushing the Cite This Result box far down the page. The three sections are now consolidated into a single tabbed panel, and the Cite box moved above it — same content, less scrolling.
- Complete
AI plausibility assessment — on-demand and background crawler
Each hypothesis can now get a second opinion from an AI model on whether the compound plausibly helps or harms the disease — grounded in the same evidence already shown on the page (the bridge relationship, the compound's effect on it, direct evidence, paper counts), not a bare "would X help Y?" question, which is what made ad-hoc chatbot queries against this data inconsistent. Open any hypothesis and click "Assess with AI" for a verdict (likely beneficial / likely harmful / unclear / conflicting), a confidence score, and the full reasoning — explicitly labeled as an automated second opinion, not a validated finding. A new background crawler (admin-controlled, never shown to regular users) works through hypotheses ranked by Combined score, highest first, requesting an assessment for anything not already cached — so popular, high-scoring hypotheses accumulate real AI assessments over time even without anyone clicking the button. The crawler uses its own separate API key so it never competes with the on-demand button for the same daily quota. The results table has a new AI column showing a color-coded verdict badge, sortable by confidence and filterable to "AI assessed only" — cached assessments never need to be recomputed unless something about the hypothesis's evidence changes, which you can check yourself since the assessment date is always shown alongside the verdict.
- Complete
Withdrawn/discontinued drug tagging
Compounds that ChEMBL flags as withdrawn or no longer marketed now show a "Withdrawn" badge next to the compound name, independent of the existing "Rx" (FDA-approved) badge — a withdrawn drug keeps its Rx badge too, since it really was approved at some point, just isn't currently available. A new three-way filter (All / Withdrawn only / Hide withdrawn) lets you isolate or exclude these from the results table, combinable with the existing Rx-only filter. No new data source was needed — ChEMBL's API already returns this alongside the approval-status field the site already imports; the importer just wasn't requesting it yet.
- Complete
Polarity indicator for the directionality gap (Problem 2 below) — shipped and live
Each hypothesis now shows a Polarity breakdown, laid out as three explicit rows labeled by the A/B/C roles and connected by directional arrows (not the double-headed ↔ used elsewhere for undirected paper counts, since these rows describe an actual effect, not a raw count): Compound → Bridge (C→A), Bridge → Disease (A→B), and an overall best-assessment, Compound → Disease (C→B). Compound → Bridge is shown as an arrow plus a signed value from -1 to +1, weighted by how many independent curated records assert each direction and diluted toward 0 by non-directional evidence (records that merely note an interaction without saying which way it goes) — so the value reflects both which direction dominates and how much of the evidence actually takes a side. Bridge → Disease shows whether the bridge is a confirmed drug target or a documented disease mechanism/marker, relabeled to read the correct direction (an earlier internal version had this backwards for some relationship types, since the underlying records store disease-then-gene, not gene-then-disease — the ABC diagram above this section had the same arrow pointing the same wrong way, now also fixed). For a minority of compound-disease pairs (about 8% in a sample check) where a data source directly curates the relationship, a direct therapeutic-vs-risk-factor verdict is shown in the best-assessment row instead, linked to its supporting citations. A new "✓ Confirmed" badge on the results table (next to the compound name) marks these rows directly, since a confirmed pair can still legitimately show a B-C count of 0 — that count only reflects papers co-indexed together in the local literature index, while the confirmed verdict can come from a source paper that used a different, more specific term. Where the Bridge → Disease direction isn't known — the common case — the best-assessment row says so explicitly rather than guessing, and placeholders mark where a real Bridge → Disease direction and a fully composed Best Assessment will appear once that data exists. A new table filter (and sortable Polarity column) lets you show only, or hide, hypotheses with direct compound-disease evidence. None of this changes any score — Strength and Combined still count evidence volume the same way as before; Polarity is a separate, independent signal shown alongside it.
- Complete
Fixed disease-term fragmentation (Problem 1 below) — sibling-term notes now live
Hypothesis cards and detail pages now show an explicit note when a zero-scoring disease-compound pair already has published evidence under a related sibling or parent disease term (e.g. a genetic subtype like "Amyotrophic Lateral Sclerosis 1" sharing evidence with generic "Amyotrophic Lateral Sclerosis"), alongside the paper count and which term it came from. This does not change any score — it surfaces context so you can judge for yourself whether a connection is genuinely novel, rather than silently merging paper counts across subtypes (which risks hiding a genuinely novel subtype-specific finding, since subtypes don't necessarily share the same disease mechanism). Two rounds of validation against real production data were needed: an initial version matched on any shared CTD-tree parent, which incorrectly linked unrelated diseases sharing only a broad category ancestor (e.g. "Addison Disease" and "Arthritis, Rheumatoid" both fell under a shared "Autoimmune Diseases" ancestor) — this was caught before wide exposure and fixed by requiring actual name similarity, plus a second signal from CTD's own synonym data to catch same-disease pairs with unrelated names (e.g. "Loeys-Dietz Syndrome 2" / "Aortic aneurysm, familial thoracic 3"). The directionality/polarity gap (Problem 2 below) is unrelated and still unaddressed.
- In Progress
Known issue: some hypotheses look novel but aren't (and don't yet account for harmful vs. beneficial evidence)
Two data-modeling gaps were identified. First, some diseases are indexed under multiple separate terms (e.g. "Amyotrophic Lateral Sclerosis" vs. genetic-subtype terms like "Amyotrophic Lateral Sclerosis 1") — a compound-disease connection already published under one term won't reduce the B–C count shown for a differently-worded sibling term, so an already-known connection can appear as a high-scoring, seemingly novel hypothesis. This is now mitigated — see the July 21, 2026 entry above. Second, scoring currently counts how much literature connects a disease, bridge, and compound, but does not yet distinguish whether that literature shows the compound helping, harming, or merely being associated with the bridge mechanism — so a compound known to worsen or cause a condition via a given pathway can score the same as one known to treat it. This remains unaddressed. Until it's fixed, treat every hypothesis as a lead to investigate, not a validated finding, and check the underlying papers before drawing conclusions.
- Complete
Fixed B–C paper counts undercounting drug mentions
The local literature index used to compute B–C (Disease–Compound) paper counts was only checking whether a paper formally listed a compound as a MeSH subject heading. In practice, most specific drugs are indexed by the National Library of Medicine via a separate chemical-substance field rather than as a subject heading — so many well-documented drug–disease connections were showing a B–C count of 0 when hundreds of supporting papers actually existed (verified example: benazepril and hypertension showed 0 despite 265+ papers directly connecting them; after the fix, the local index shows 225). The indexer now also checks the chemical-substance field, and the full literature index has been rebuilt from NCBI's MEDLINE archive against it — citations indexed nearly doubled, from 43M to 83.5M. Hypotheses will reflect the corrected B–C counts as of the next database update.
- Complete
Resilient database updates — staging swap and incremental refresh
The periodic data update process has been redesigned for reliability. Updates now build a complete new database in a staging file alongside the live one. The live database is only replaced once every import step succeeds — if any step fails partway through (e.g. a data source API times out), the staging file is discarded and the live database is left completely intact. Previously, a failed update would leave the database in a partial state with missing data. Additionally, updates are now incremental: each data source clears and replaces only its own records rather than wiping the entire database before rebuilding. The ChEMBL importer also received improved retry logic (6 attempts, 60s timeout, longer backoff on HTTP 500) to better handle transient API failures.
- Complete
Permanent hypothesis URLs, Cite This Tool page, and startup reliability
Hypothesis URLs are now permanently stable across database rebuilds. Each URL is derived from the three UMLS Concept Unique Identifiers (CUIs) that define the hypothesis — standard codes maintained by the US National Library of Medicine and used across NIH, FDA, and major biomedical databases. Because CUIs are internationally standardized and never change, the URLs will remain valid indefinitely regardless of how the underlying database is updated or rebuilt. A new Cite This Tool page (linked as CITE in the navbar) provides ready-to-copy APA and BibTeX citation blocks for academic use, and explains why the URLs are permanent. Each hypothesis detail page also includes a collapsible Cite This Result box with a pre-filled citation for that specific hypothesis.
- Complete
Insights tab: top pair leaderboards for A–B, A–C, and B–C
Three new leaderboard cards on the Insights tab show the top 5 most paper-supported pairings for each side of the ABC triangle. The A–B card ranks Disease × Bridge pairs by B↔A paper count, the A–C card ranks Bridge × Compound pairs by C↔A paper count, and the B–C card ranks Disease × Compound pairs by direct B–C paper count. The B–C leaderboard includes a note reminding readers that high counts reflect well-established connections — this app's primary targets are hypotheses where B–C = 0. All three leaderboards respect the active filters and tranche selection.
- Complete
Scoring formula v1.1, methodology page, and score delta indicators
Strength score formula updated to v1.1: log compression now applied per-side before the geometric mean (S = 1 − 1 / (1 + √(ln(1+B↔A) × ln(1+C↔A)))). The previous formula applied log after the geometric mean, which caused lopsided triangles (e.g. B↔A=2, C↔A=100) to incorrectly outscore balanced equivalents. Symmetric cases are mathematically unchanged — only lopsided hypotheses are affected. All hypothesis scores have been recomputed. A new Scoring Methodology page documents the formula rationale, behavior tables, known limitations, and full version history; it includes a version picker (v1.0 / v1.1) and a Track Changes toggle that overlays a diff of what changed between formula versions. The table page shows a scoring formula version badge (SCORING v1.1) linking to the methodology page, and after a rescore, each affected row shows a ▲/▼ badge indicating how many percentage points its score shifted.
- Complete
Discovery Landscape, Insights tab, and UX improvements
Major update across the main results page. A new Insights tab sits alongside the Table tab and shows aggregate analysis of the current filtered view: literature evidence distribution across five novelty tranches (Untested through Established), top 20 bridge targets by hypothesis count, score tier analysis for the top 1/5/10% of hypotheses, and auto-generated key findings. Clicking any tranche filters the table directly. A B–C column was added to the table showing direct disease–compound paper counts, color-coded by tranche. Pagination now supports typing a page number to jump directly. The 'What is this?' content moved to a dedicated About page linked from the navbar. Mobile layout improvements: a desktop recommendation banner and a 4-column table layout on narrow screens.
- Complete
PubMed novelty verification crawler
All 2,008,644 unique disease–compound pairs have been verified against PubMed. Novelty scores are now exact counts rather than estimates — the ~ prefix no longer appears. The cache survives database rebuilds so re-verification is not needed after a data update.
- Complete
Scoring formula correction
The Strength score was using a simple sum of B↔A and C↔A edge counts, which allowed a lopsided bridge (e.g. 99 papers on one side, 1 on the other) to score the same as a balanced one. Replaced with a geometric mean so both sides of the triangle must be well-evidenced. All 4.7M hypothesis scores are being recomputed.
- Complete
Filter state preserved on back navigation
Filters, sort order, and page are now encoded in the URL. Using the browser back button or the ← Back link on a hypothesis page restores your exact previous view.
- Complete
Score column shows active metric
When sorting by Novelty or Strength, the score pill in each row now shows that metric's value rather than always showing the combined Score.
- Complete
Public launch
Swanson's Apple is now publicly available at swansonsapple.org. The database currently tracks 4,746,394 hypotheses drawn from 6 biomedical sources: CTD, Open Targets, ChEMBL, DGIdb, DrugCentral, and DISEASES.