Documentation
Archivum is a catalog of public AI datasets with a consistent record of what each one documents about itself. This page explains exactly how those records are assembled and how Documentation Coverage is calculated — so you can recompute any figure yourself.
Quickstart
Search is free and needs no account — explore the catalog and open any dataset for its full record: origin, licensing, structure, and lineage as documented at the source.
from archivum import Client
client = Client()
ds = client.pull("clinical-notes-mimic-derived", version="v3.2.0")
print(ds.coverage.total) # % documented
print(ds.license.spdx) # licence as published
ds.export("llamaindex") # provenance travels with the dataEvery record links back to its source platform. Archivum stores metadata only — downloads always happen at the origin, under the origin’s terms.
Coverage methodology v1.0
Documentation Coverage measures one thing: how much of a dataset’s provenance was documented at the source when Archivum checked. It is a factual measure of the record, not a judgment of the dataset. Twenty-eight checks are grouped into four equally weighted sections; each check asks a question with a verifiable yes-or-no answer.
How each check is scored
Documented — 1 point. Archivum retrieved the artifact itself through the platform’s API: a licence field, a file manifest, a schema, a commit history.
Reported — half a point. The publisher stated it in prose — a README section, a dataset card paragraph — that Archivum retrieved but did not independently confirm.
Not found — 0 points. Absent from the published metadata at the time of the check. This is a fact about the record, not a defect in the data. Checks that cannot apply on a platform are excluded from that dataset’s denominator entirely.
A section’s score is its points divided by its applicable checks. The overall figure is the mean of the four sections. That is the entire calculation — no weights to tune, no popularity bonus, no judgment call.
The 28 checks
Origin & Sourcing
7 checks · weight 25Where did this data come from, and who assembled it?
- Publisher identified. Owner or organisation name present on the source record.
- Published by an organisation. Source account is an organisation rather than an individual profile.
- Upstream sources declared. Card metadata or documentation names the data it was derived from.
- Collection method described. Documentation describes how the data was gathered.
- Collection timeframe stated. Documentation gives dates or a period of collection.
- Annotation process described. Documentation describes labelling or human review, where applicable.
- Maintainer contact listed. A contact address, issue tracker, or discussion channel is given.
Licensing & Terms
7 checks · weight 25What terms did the publisher attach to reuse?
- Licence declared. A licence field is set on the source record.
- Licence file present. A LICENSE file exists in the repository tree.
- Licence maps to a known identifier. The declared licence matches a recognised SPDX identifier.
- Commercial terms stated. The licence text or card states whether commercial use is addressed.
- Attribution terms stated. Attribution requirements are stated in the licence or card.
- Redistribution terms stated. Redistribution or sharing terms are stated.
- Upstream licences noted. Documentation notes the terms of the sources it was built from.
Composition & Structure
7 checks · weight 25What is actually inside, and how is it organised?
- Description present. A non-empty summary is set on the source record.
- Field schema available. Column or feature names and types are retrievable.
- Splits documented. Train, validation, or test splits are declared.
- Record count available. A row or example count is published.
- File manifest available. The list of files in the dataset is retrievable.
- File sizes available. Byte sizes are published for the listed files.
- Sample records available. Example rows are retrievable without downloading the dataset.
Maintenance & Usage
7 checks · weight 25Is it still maintained, and how is it being used?
- Last modified date known. The source reports when the dataset last changed.
- Version history available. A commit or revision history is retrievable.
- Release notes available. Tagged releases or a changelog are published.
- Citation provided. A citation block, DOI, or paper reference is given.
- Usage statistics available. Download, star, or reuse counts are published by the platform.
- Known limitations documented. Documentation includes a limitations, bias, or caveats section.
- Intended use documented. Documentation states what the dataset is intended for.
Coverage reflects what was present at the source on the date shown with each record. It describes documentation, not data quality, and nothing on this site is legal advice — read the licence at the origin before commercial use.
Lineage model
Lineage answers one question: what does the source document about this data’s path from origin to the current version? Archivum models the chain as stages — original source, raw acquisition, cleaning, annotation, and current version. Each stage the source documents records the actor, a content fingerprint where available, a timestamp, and whether it was documented or reported.
Stages the source does not document render as explicit gaps — dashed, in warning color — on every lineage graph. Archivum never invents a stage to close a gap. The gap is the information.
API reference
The read API mirrors what the site shows — the same records, the same evidence labels. All endpoints are read-only and return JSON.
| Endpoint | Returns |
|---|---|
| GET /v1/datasets | Paginated catalog with filters: query, platform, modality, licence, min_coverage |
| GET /v1/datasets/:slug | Full record: coverage sections, all 28 check outcomes, licence as published, schema |
| GET /v1/datasets/:slug/lineage | Stage graph with actors, fingerprints, and documented gaps |
| GET /v1/datasets/:slug/versions | Version history with per-version coverage |
| GET /v1/facets | Available filter values with counts |
Responses carry the coverage rules version so pipelines can pin against it.
Corrections
Any publisher or user can submit a correction to a specific part of a record — not the figure in the abstract, but a particular stated fact: this licence entry is out of date, here is the current file. Corrections are checked against the source; when the record changes, the dataset’s history shows the revision. Corrections are free and require no account, because being correctable in public is what makes a reference work worth consulting.
Use Suggest a correction on any dataset page, or write to archivumllc@gmail.com with the dataset slug and the field in question.