Key points
- With about 300 patients, a private neural generator did poorly: 56% fidelity on a heart failure cohort. Switching to a private Gaussian copula raised that to 81%, against 82% for a second sample of real patients.
- A model trained on the synthetic cohort predicted death with ROC AUC 0.87, against 0.94 when trained on real patients. Useful for building analyses, but not a stand-in for the real data.
- The copula keeps relationships between pairs of variables. It does not keep interactions, so subgroup findings should not be read from synthetic small-cohort data.
- Rare events are deliberately blurred. A privacy method that preserved three unusual patients exactly would be exposing them.
Why small cohorts are hard
Rare-disease studies, early-phase trials and single-site registries often have a few hundred patients or fewer. These are exactly the teams that would benefit from a shareable synthetic copy, and exactly where synthetic data struggles. There are three reasons.
- Neural generators need volume. A GAN learns by seeing many examples. With 200 training rows it has very little to learn from.
- Privacy noise is a fixed size. Differential privacy adds noise scaled to one patient's influence, not to the size of the table. On 5,000 patients that noise is small next to the real counts. On 200 patients it is large.
- Small samples are noisy anyway. When we compared one sample of real patients with another from the same small dataset, the fidelity score was only 82 to 84%. No synthetic method should be expected to beat that.
What we changed
For tables under 1,000 rows, or tables dominated by lab and vital measurements, the MirrorCohort engine switches from its private GAN to a private Gaussian copula. The choice depends only on the row count and the column plan, which are already public or private, so making it spends no extra privacy.
- Each column's distribution comes from a privately measured histogram. Measurements such as blood pressure or creatinine are grouped into up to 40 bins, and a value is drawn within the chosen bin.
- The relationships between columns come from one noisy release of their rank correlations. This is a single, compact summary, so it needs much less privacy budget than training a neural network for hundreds of steps.
- Everything is counted in the same privacy budget as the rest of the run. All results here are at ε = 8.
Results on two small public cohorts
| Measure | Private GAN | Private copula | Real-data reference |
|---|---|---|---|
| Heart failure (299 patients): fidelity | 56.4% | 80.9% | 82.1%second real sample |
| Heart failure: death, ROC AUC | 0.703 | 0.871 | 0.943trained on real |
| Heart failure: follow-up time, R² | n/a | 0.118 | 0.114trained on real |
| Cleveland heart disease (303 patients): fidelity | n/a | 82.6% | 84.4%second real sample |
| Cleveland: heart disease, ROC AUC | n/a | 0.836 | 0.855trained on real |
| Membership attack AUC (0.50 = guessing) | 0.52 | 0.48 to 0.52 |
ε = 8, outcome column declared before training. 70% of patients trained the generator (209 and 212 patients); every score is measured on the 30% it never saw. The Cleveland dataset was only run with the current engine. Single runs; four repeat runs on Cleveland ranged from 81.3% to 82.6% fidelity and 0.815 to 0.853 AUC.
On Cleveland, the synthetic-trained model came within 0.02 AUC of the real-trained one. On heart failure the gap is larger, 0.07, but much smaller than the GAN's 0.24. Follow-up time is weakly predictable even from real data (R² 0.11), and the synthetic data matched that.
What you give up
Small-cohort synthetic data is a simplification, and it helps to know exactly which parts are simplified.
- Interactions. The copula keeps how each pair of variables moves together. It does not keep effects that depend on a third variable, such as a treatment that works differently in men and women. Subgroup analyses on this data will not reflect the real cohort.
- Rare categories. A category held by a handful of patients may be merged or smoothed away, because its count cannot be measured precisely without exposing those patients.
- Rare events. If three patients had an unusual adverse event, the synthetic cohort will not reproduce them faithfully. That is the privacy guarantee working as intended.
- Very small registries. Our smallest tested training set was 209 patients. For a registry of a few dozen, we would test on the real table in a pilot before promising anything useful at ε = 8.
Where it is useful
Within those limits, a synthetic copy of a small cohort is good for work that needs realistic structure but not the real patients:
- Writing and testing the statistical analysis code before database lock or before data access is approved.
- Sharing a realistic table with a collaborator, statistician or software vendor while a data use agreement is still being negotiated.
- Teaching and onboarding, with data that behaves like the study's own.
- Checking that pipelines, data dictionaries and edit checks handle real-looking values and ranges.
It is not a basis for estimating treatment effects, subgroup findings or adverse-event rates. Those belong on the real data.
Data sources
- Heart Failure Clinical Records. Chicco & Jurman (2020), BMC Medical Informatics and Decision Making 20:16. UCI Machine Learning Repository. CC BY 4.0.
- Heart Disease (Cleveland subset). Janosi, Steinbrunn, Pfisterer & Detrano (1989). UCI Machine Learning Repository. CC BY 4.0.
Use of these datasets does not imply endorsement by UCI or the original authors. Full method and all five benchmark datasets: benchmark write-up.
Raja Pal, PhD is the founder of MirrorCohort. Before founding MirrorCohort, Raja spent nearly nine years as a process and quality engineer at Intel. Questions about small-cohort data: raja@mirrorcohort.com · LinkedIn
Working with a small cohort or a rare-disease registry? We can test it in a pilot inside your environment before you commit to anything.
Book a 30-minute call