Format obsolescence

File formats evolve and sometimes disappear. If you cannot open yesterday’s files in tomorrow’s software, your information is practically lost. The Library of Congress Sustainability of Digital Formats program documents thousands of formats, including “sustainability factors” like openness, adoption, transparency, and self‑documentation that correlate with long‑term viability. IG teams use this analysis to favor formats such as PDF/A, TIFF, and XML for preservation masters. [loc.gov]

The PRONOM registry from The National Archives (UK) complements this by cataloging formats, versions, signatures, and related software, and powering DROID for automated identification. PRONOM’s modernization work (linked data, signature updates) and search services help organizations inventory holdings and spot risky formats early, enabling proactive migration. [nationalar...ves.gov.uk], [nationalar...ves.gov.uk], [github.com]

Media degradation and bit rot

Digital media—HDDs, SSDs, optical discs, tape—fail over time; silent corruption (“bit rot”) can render files unreadable. While NIST’s SP 800 series focuses on information security rather than archival science, its guidance highlights the need for integrity mechanisms and lifecycle controls. Preservation programs implement fixity checks (e.g., SHA‑256) on ingest and on a schedule, verifying checksums and repairing from redundant copies when corruption is detected. Standards bodies and communities reinforce the principle of lots of copies: programs like LOCKSS operationalize peer‑to‑peer integrity polling and repair across many nodes so that corruption in one replica is detected and healed. [nist.gov], [lockss.org]

Software dependency

Many digital objects depend on specific software stacks (codecs, runtimes, plug‑ins) or even complete environments to render correctly. OAIS anticipates this by emphasizing Representation Information—the documentation and software necessary to render and understand content. Preservation planning thus includes documenting dependencies, acquiring or packaging software, or planning migration/emulation routes for complex objects (e.g., multimedia projects, interactive spreadsheets, CAD/3D, or games). [ccsds.org]

Organizational and legal discontinuity

Preservation is a marathon, not a sprint. Funding cycles, mergers, provider shutdowns, and policy changes can interrupt stewardship. NARA’s federal transfer requirements and bulletins (e.g., ERM, Capstone email) try to reduce this risk by mandating standards‑based scheduling, transfer, and metadata across agencies; likewise, LOC’s RFS and FADGI provide continuity for the broader cultural heritage sector. IG’s job is to codify these requirements into retention schedules, legal holds, and disposition overrides so that custody and obligations persist across organizational change. [archives.gov], [loc.gov], [digitizati...elines.gov]

Migration

Migration moves content to newer formats or media to maintain renderability. Routine examples include normalizing office docs to PDF/A‑2 or PDF/A‑4 for fixed‑layout access and migrating master images to TIFF with embedded metadata. Migration is most successful when driven by risk triggers (e.g., PRONOM signals increased obsolescence) and controlled by documented format policies (e.g., Archivematica’s Format Policy Registry). [pdfa.org], [archivematica.org]

Migration does not mean discarding originals. OAIS and NARA practice advise retaining the source (for authenticity and re‑processing) while creating a normalized preservation master and access derivatives, with clear provenance metadata explaining each action. [ccsds.org], [archives.gov]

Emulation

Emulation recreates the original software/hardware environment so digital objects run unchanged. It is valuable for software‑dependent works (legacy multimedia, interactive art, scientific workflows). OAIS frames emulation as a preservation planning option when migration would deform significant properties. Community projects (e.g., Emulation‑as‑a‑Service) use encapsulated environments with documented Representation Information to deliver authentic experiences. [ccsds.org]

Normalization

Normalization converts many incoming formats into a small set of preservation targets with strong sustainability characteristics. Typical targets include PDF/A for documents, TIFF (or JPEG 2000 in some domains) for still images, and WAV/BWF for audio; where structure is key, XML (or JSON with schemas) expresses data in open, self‑documenting form. Normalization is often automated and governed by policy (e.g., Archivematica FPR rules). [pdfa.org], [loc.gov], [w3.org], [archivematica.org]

Favoring open standards and archival variants

• PDF/A (ISO 19005) ensures device‑independent, self‑contained documents. Multiple parts exist—PDF/A‑1 (2005), PDF/A‑2, PDF/A‑3, and PDF/A‑4 (2020)—each building capabilities while constraining risky features. Conformance levels (A/U/B) balance accessibility, text extraction, and visual fidelity. [iso.org], [pdfa.org], [pdfa.org]
• TIFF remains widely preferred for master images because of its openness and transparency, especially when used with baseline features and embedded technical metadata; Library of Congress RFS continues to list TIFF among preferred still‑image formats. [loc.gov]
• XML persists as a durable, human‑ and machine‑readable textual syntax standardized by the W3C, suitable for metadata (e.g., METS, PREMIS) and content that benefits from structured markup and schema validation. [w3.org]

Table — File format longevity (illustrative)

Provenance metadata and content credentials

AI outputs must carry provenance: who prompted/approved, what model produced it, when, with what data/parameters, and where it was stored. The C2PA (Coalition for Content Provenance and Authenticity) standard provides a method to bind tamper‑evident provenance assertions (e.g., “content credentials”) to media using embedded metadata and cryptographic signing; the Library of Congress format workplan explicitly references JUMBF (the metadata box format underlying C2PA) as a preservation‑relevant technology. [nvizionsolutions.com], [wwws.loc.gov]

Watermarking and detection

Watermarking can signal AI origin but should not be the only control: technical research continues to show that many watermarks are fragile under transformations. Preserve both the watermark (if present) and the provenance record within the object’s metadata/AIP to support future authenticity checks. (The RFS and Sustainability of Digital Formats emphasize metadata richness and transparency to bolster preservation even when technical markers fail.) [loc.gov], [loc.gov]

Model and prompt versioning

AI content cannot be fully understood without model context. IG should require:

• Model version identifiers (and provider), training cutoffs, and safety filters applied at generation time.
• Prompt/response archives for significant decisions, captured in XML/JSON with timestamps and user IDs.
• Reproducibility artifacts where possible (e.g., seeds, temperature), recognizing some models are inherently nondeterministic.

Where models are internally developed, preserve model cards, training data documentation, and versioned artifacts using open tools (e.g., MLflow or DVC) linked to preservation packages, even if the binaries themselves are not preserved indefinitely. Align with OAIS by treating the model and its documentation as Representation Information for derivative AI content. [ccsds.org]

Ethical considerations

Preserved AI content can encode bias or misuse. Appraisal should weigh long‑term value against potential harms, applying access controls and use notes where appropriate. LOC’s RFS reminds institutions to consider accessibility, rights, and user communities; OAIS’s Designated Community concept helps tailor documentation and access for ethical clarity. [loc.gov], [ccsds.org]

OAIS (ISO 14721)

The OAIS Reference Model is the blueprint for digital preservation systems: it defines information packages (SIP, AIP, DIP), functions (ingest, archival storage, data management, administration, access), and the Preservation Planning function that monitors risks and triggers actions (e.g., migrations). The 2024–2025 updates reaffirm mandatory responsibilities and clarify terminology used by audits and repository software. [ccsds.org], [iso.org]

NARA guidance and regulations

NARA’s digital preservation guidance covers the lifecycle from agency creation to transfer of permanent records, including format and metadata requirements (e.g., NARA Bulletin 2018‑01 updating format guidance; metadata transfer requirements). The eCFR 36 CFR Part 1235 Subpart C specifies transfer standards for electronic, audiovisual, cartographic, and architectural records. Together these sources inform federal‑grade expectations for format choices, quality, and documentation. [archives.gov], [ecfr.gov]

Library of Congress (LC) RFS and Sustainability site

The Recommended Formats Statement (2025–2026) lists preferred and acceptable formats across content categories (text, still image, moving image, audio, datasets, 3D/design, software/games, web archives, email). The Sustainability of Digital Formats site provides deep documentation and evaluation criteria to guide format selection and risk appraisal. [loc.gov], [loc.gov]

PRONOM and DROID

PRONOM is the public, authoritative format registry; DROID uses PRONOM signature files to identify formats at scale. These are indispensable for inventory, risk assessment, and migration planning and are integrated into preservation tools (e.g., Archivematica). [nationalar...ves.gov.uk], [archivematica.org]

Archivematica

Archivematica is an open‑source, OAIS‑aligned digital preservation system with a Format Policy Registry (FPR) that encodes identification, validation, and normalization rules; it integrates PRONOM signatures and tools like JHOVE and MediaConch, and automates PREMIS event metadata capture. Its Preservation Planning tab lets institutions manage policies as standards evolve. [archivematica.org], [archivematica.org]

LOCKSS / CLOCKSS

LOCKSS (Lots of Copies Keep Stuff Safe) provides distributed, tamper‑resistant preservation via peer polling, repair, and local custody. Networks like CLOCKSS (Controlled LOCKSS) preserve scholarly content; others protect government documents and regional heritage. LOCKSS is actively maintained by Stanford Libraries and recognized for long‑standing contributions to digital preservation. [lockss.org], [cni.org]

Appraisal for long‑term value (what to keep)

Not all digital content merits permanent preservation. Use appraisal to select what supports legal accountability, cultural memory, scientific reproducibility, or business continuity. NARA’s Capstone approach for email illustrates role‑based appraisal for high‑value communications; LOC’s RFS provides preferable format guidance when multiple versions exist. Document appraisal decisions in records schedules and link to OAIS ingest (SIPs) to ensure the archive knows why and how long to preserve. [archives.gov], [loc.gov]

Integrity checks (fixity) and storage

• Generate checksums (SHA‑256 or better) at ingest.
• Recalculate on a schedule; investigate any mismatch; repair from redundant copies.
• Store multiple, independent copies (ideally in different geographic and administrative domains) to avoid correlated loss—LOCKSS operationalizes this model. [lockss.org]

Access and authenticity

OAIS emphasizes authenticity (is this what it claims to be?) and understandability for the Designated Community. Preserve PREMIS events (ingest, migration, fixity verification, rights changes), maintain rights/access policies (including restrictions for sensitive AI outputs), and provide DIPs in accessible formats (e.g., PDF/A, web derivatives) while safeguarding originals. LC’s and FADGI’s guidance help shape access derivatives and metadata. [ccsds.org], [digitizati...elines.gov]

Table — Preservation checklist (condensed)

Case 1 — Format Obsolescence Sinks a Research Archive (Hypothetical)

A university inherited thousands of lab notebooks exported in a proprietary note format from a discontinued platform. Ten years later, the new team could not open them; the vendor was gone. The archive had no Representation Information, and exports lacked embedded metadata.
What went wrong: No appraisal of format sustainability; no migration or normalization at source; no PRONOM/DROID inventory.
Recovery: Working with format signatures and community reverse‑engineering, the library extracted plain text and images, then normalized to PDF/A‑2 with embedded XML metadata for structure. The originals were kept under restricted access; the preservation masters were indexed for search.
IG lesson: Incorporate format policy in records management; require open or standardized exports in contracts; run DROID on new collections; align preservation targets to LC RFS preferences. [nationalar...ves.gov.uk], [pdfa.org], [loc.gov]

Case 2 — Media Failure vs. Many Copies (Realistic Composite)

A regional archive stored a single LTO‑5 tape set of oral histories. Years later, read errors and missing index blocks prevented restoring several sessions. Fortunately, a LOCKSS network for local heritage had ingested access derivatives and checksums; curators used them to reconstruct missing segments and verify integrity.
What went wrong: Single‑copy storage; no scheduled fixity verification of the tape.
What worked: Distributed preservation with integrity polling; enough copies to reach consensus and repair.
IG lesson: For essential collections, maintain multiple, independent replicas and scheduled fixity checks; integrate LOCKSS or cloud/geo‑replication into storage policy. [lockss.org]

Case 3 — AI‑Generated Report Without Provenance (Hypothetical)

A public agency published an AI‑assisted policy brief later challenged in court. There was no record of the model version, prompts, or human edits. Opposing counsel argued the report could not be authenticated or reproduced.
Remediation: The agency’s IG office instituted content credentials (C2PA) for graphics and embedded provenance metadata for text (XML sidecars linking prompts, model IDs, timestamps, and approvers). Major AI outputs now enter the archive as SIPs with PREMIS events and, when appropriate, watermark evidence retained but not relied upon.
IG lesson: Treat AI outputs as records; preserve the who/what/when/how (including model context) and bind provenance at the object level using open standards. [nvizionsolutions.com], [ccsds.org]

Case 4 — Successful OAIS Implementation with Archivematica (Real Organization)

A mid‑sized museum adopted Archivematica to process digitized collections. The team configured FPR policies to normalize TIFF masters and PDF/A access copies, integrated PRONOM/DROID for identification, and enabled PREMIS event capture for every step (virus scan, normalization, fixity, storage). An internal audit mapped the workflow to OAIS functions and found clear evidence of authenticity and chain‑of‑custody.
IG lesson: Open‑source tools and community standards can meet audit needs if policies are explicit and logs are retained. [archivematica.org], [archivematica.org]

Ongoing challenges

• Scale and diversity: Archives ingest petabytes in many formats; appraisal and automation are essential. LC’s RFS and Sustainability site help prioritize. [loc.gov], [loc.gov]
• Evolving standards: OAIS, audit criteria (ISO 16363), and PDF/A parts evolve; repositories must update policies accordingly. [OAIS (ISO...rs updated]
• Cloud & SaaS churn: Vendor lock‑in and API changes threaten extractability; insist on standards‑based exports and contractual exit terms aligned to preservation targets. (Use PRONOM/LOC criteria during procurement.) [nationalar...ves.gov.uk], [loc.gov]
• Human factors: Staff turnover and under‑resourcing can erode practice; OAIS stresses administration and preservation planning as ongoing commitments. [ccsds.org]

Best practices (actionable)

1. Adopt OAIS as your common language; map current processes to OAIS functions and packages (SIP, AIP, DIP) to identify gaps. [ccsds.org]
2. Set format policies using LC RFS and Sustainability criteria; implement identification with DROID and normalization via Archivematica FPR. [loc.gov], [loc.gov], [archivematica.org]
3. Build integrity in: generate checksums on ingest; run periodic fixity; keep independent replicas (e.g., LOCKSS or diverse cloud providers). [lockss.org]
4. Document provenance and rights in METS/PREMIS; for AI outputs, add C2PA/JUMBF credentials and model context. [wwws.loc.gov], [nvizionsolutions.com]
5. Follow NARA transfer and metadata guidance (even outside U.S. government contexts) to ensure completeness and audit‑readiness. [archives.gov]
6. Test your migrations: run side‑by‑side render and significant‑properties checks; record PREMIS events and keep originals. [ccsds.org]
7. Embed preservation in IG: tie retention schedules and legal holds to disposition overrides and OAIS workflows so permanent records never get purged during routine cleanup. [archives.gov]

Table — Migration strategy in seven steps