Methodology
Magistra Side Effects Predictor™ — Version 5.21 (dual-track)
Last updated: September 2026 | Open for peer review
Invitation to researchers
This methodology is deliberately transparent. We invite biostatisticians, epidemiologists, clinical researchers, and ML researchers to critique our approach, propose improvements, or collaborate. All feedback is publicly acknowledged in future versions.
Submit feedback ↓1. Dual-track architecture
Rather than blending clinical and community data into a single number (which hides the evidence hierarchy), we compute two parallel estimates per side effect:
Clinical
Only published clinical trials + regulatory reports. Conservative. All modifiers from peer-reviewed sources.
Real world
Reporting frequency from posts on the community platforms themselves: of all distinct community reports in our corpus, what share mention this effect. News-aggregator search results do not count as community reports (corrected 2026-08-17), and Reddit posts count only if they are in a GLP-1, weight-management or diabetes community we collect from (corrected 2026-08-29, after a failed subreddit restriction let 159 of 185 reports in from unrelated communities) — the live platform list is shown on /en/data and in /api/data. Not a measured incidence — a single anecdote is not a percentage (see "5. Honest limitations" below for why we stopped averaging self-reported rates).
The two tracks measure different things and are deliberately not compared as one estimate: clinical is an incidence percentage, real-world is a reporting frequency (how often something comes up, not how often it happens). A high reporting frequency alongside a low clinical estimate is a signal that patients are actively talking about something — useful as a pointer, not evidence of a higher true incidence. (Revised 2026-08-14: earlier versions of this page described the real-world track as an incidence estimate that should "converge" with clinical data — that rested on averaging self-reported percentages from forum posts, including one X/Twitter post claiming 76% hair loss that pulled a 2-source average to 39%.)
2. Data collection
Daily, 17 scrapers collect data from four source categories. Claude Haiku extracts structured data points with a confidence level (high/medium/low):
- • Clinical: PubMed, ClinicalTrials.gov, Cochrane (PubMed-indexed), NICE guidelines, preprints
- • Regulatory: FDA FAERS (direct API), EMA (direct ema.europa.eu pages)
- • User reports: Reddit (16 subreddits), Trustpilot, Drugs.com, Quora
- • News: Google News search results (incl. coverage of WHO/MHRA/EMA safety actions — not a direct feed from those agencies)
Each data point retains its sourceType for deduplication (sourceUrl + sideEffect) and for filtering into the two tracks. Extraction is conservative: only explicitly stated rates, no inference.
A configured scraper is not the same as a producing source: some have delivered nothing for months (Reddit has been served an HTTP 403 block since 2026-05-28). Live per-source counts, each with the date of its most recent data point, are published at /api/data?q=overview.
3. Statistical approach
For each track:
- Filter data points by patient profile (sex, dose, ethnicity, exercise)
- Weighted average (weight = sample size × extraction confidence; future work: inverse-variance meta-analytic weighting by study quality)
- Winsorization at 5th/95th percentile if n > 10 (bounds logged explicitly)
- Dose scaling if data lacks dose specificity
- Log-odds modifier application (capped at total shift of 2.5 log-odds)
- Random-effects confidence interval (simplified, unweighted τ² — inspired by DerSimonian-Laird, not inverse-variance weighted)
4. Self-evolving model
A daily statistical analysis computes empirical odds ratios for every dimension × effect that has enough citable evidence. As of 2026-09-04 no effect clears that bar: every modifier in production is still the 2026-04-12 hand-coded seed value, with no per-modifier citation recorded, and no auto-update has ever been applied (the model config now stands at v2 following a manual correction on 2026-09-04, not an automated one). Safeguards for when it is:
- • FDR correction (Benjamini-Hochberg) across all hypothesis tests
- • Auto-update only when n ≥ 30, corrected p < 0.01, and OR change < 0.3
- • Larger changes are flagged for human review
- • New parameters (ethnicity, BMI, diet) get 'promoted' when significant for 2+ effects
- • Max 5 auto-updates per day; all changes logged with provenance
- • Every applied or flagged change archives the outgoing config version (no retention cap) — this has never happened yet, since no auto-update has ever been applied or flagged
5. Honest limitations
We publish our limitations because hidden weaknesses are more dangerous than visible ones:
- Evidentiary base (not the corpus size): The corpus holds 2,347 data points across the 15 published side effects, but only 1110 of them state a rate, and after our eligibility rules 887 rates from 29 distinct studies are allowed to support a published estimate. Excluded: 150 seed unverified, 62 spontaneous report share, 11 aggregator result. Rates are collapsed to one entry per source per effect (180 entries in total); since 2026-09-08 the site-wide count is keyed by study URL, so one registry record or paper counts once however many effects it reports. Many effects therefore rest on a single source — which is stated next to each estimate. (Correction 2026-08-31: the April-2026 seeding pass turned out to have also written rows restating our own static literature table under real trial/study URLs that no extraction ever read — rows a URL-based screen cannot catch. All 150 seed rows are now labelled and excluded; the eligible base roughly halved as a result, and five effects fell back to a labelled literature figure. Base figures published before this date should not be cited.)
- FAERS is not incidence: Shares of spontaneous adverse-event reports (FAERS) are reported separately and never averaged into an incidence estimate.
- Demographic gaps: Female bias (81% of the 246 data points that specify a sex), minimal ethnic diversity in sources. Only 246 of 2,451 data points state a sex at all.
- Modifier sources: Every modifier currently applied was hand-coded at the 2026-04-12 seed and no per-modifier citation is recorded for any of them, so treat each odds ratio as an expert-coded prior rather than a sourced estimate — the predictor now prints the modifiers it applied, and their provenance, beside every adjusted figure. Zero have so far been replaced by empirically estimated values, because no effect reaches the threshold of 10 citable stated rates. Target architecture: inverse-variance meta-analytic framework with formal study quality assessment (Cochrane Risk of Bias).
- No interaction terms: Modifiers are applied independently. Interactions (e.g., sex × age) are not modeled — we cap cumulative log-odds shifts to limit stacking bias.
- Calibration testing: Not yet formally validated against independent outcome data. Planned once n ≥ 500 per effect.
- LLM extraction: Imperfect. Gold-standard manual audit is planned on a sample of 50 sources per effect.
- Journey predictor: Since 2026-09-12 the weight-trajectory model takes its total loss from the ClinicalTrials.gov arm matching the selected molecule, dose tier and diabetes status (semaglutide: STEP; tirzepatide: NCT04184622/NCT04657003; liraglutide: NCT01272219/NCT01272232, with lower tiers scaled via NCT00422058); the shape of the curve and every age, sex, exercise and BMI modifier remain calibrated on semaglutide STEP data for all molecules. The muscle-preservation and after-stopping models use STEP trial constants with expert-coded modifiers throughout. None of the three is empirically validated.
- Not causal: These are population-average conditional risks, not individual causal predictions.
6. Publication roadmap
- • Phase 1 (current): dual-track framework, open methodology, community feedback
- • Phase 2: n ≥ 100 per effect, formal calibration testing
- • Phase 3: n ≥ 500, external validation on independent dataset
- • Phase 4: pre-register at OSF.io, submit to peer-reviewed journal (target: Nature Medicine)
We are a small team and our methodology inevitably has weaknesses. If you see something wrong or improvable, let us know.
Researcher feedback
If you're a biostatistician, epidemiologist, clinical researcher, or ML scientist: critique our approach. We acknowledge all contributors in future versions and maintain a public changelog.
Open source on GitHub
The full methodology, source code, and preprint are published under Apache 2.0. Inspect, fork, or submit a pull request.
github.com/saurabhgoyal75/magistra-predictor →Cite this methodology
This page is the canonical, continuously updated version of the methodology. There is no DOI — it is self-published here rather than deposited with a repository that mints permanent identifiers.
Working from the corpus itself? The dated snapshot behind these estimates — the full dataset CSV with a PDF brief and this methodology annex — is available as a one-off purchase, alongside the free public API. Data & API.
Magistra Side Effects Predictor™ — Statistical indicator, not medical advice.