Adelaide's Digital Archives Are Full of Ghost Images, and Officials Say the Problem Is Getting Worse

South Australian government agencies, cultural institutions and technology operators at Lot Fourteen are confronting a growing backlog of duplicate, corrupted and mismatched images embedded across public-facing databases and internal record systems, and the consensus among those managing the problem is that manual fixes are no longer keeping pace with the scale of it.

The issue sits at the intersection of several forces pressing on Adelaide simultaneously: rapid digitisation of heritage collections, the expansion of defence and space industry data infrastructure, and an interstate migration surge that has pushed new residents toward government digital services faster than those services were built to handle. When images are ingested into a database more than once, through scanning errors, system migrations or human oversight, they create redundant file chains that corrupt search results, inflate storage costs and, in some cases, serve the wrong photograph against the wrong record.

What the Institutions Are Saying

The State Records of South Australia, which maintains physical and digital archives for the state government from its repository on Leigh Street in the CBD, has been working through a digitisation program that accelerated significantly after 2022. Officials with knowledge of the program have indicated, in public presentations, that image deduplication was not built into the original ingestion workflow and must now be applied retrospectively, a more expensive and time-consuming process. The agency has not released a public figure on the number of affected records, but the scope of the retrospective work has been described internally as substantial.

At Lot Fourteen, the former Royal Adelaide Hospital site on North Terrace that now houses the Australian Space Agency headquarters, the SmartSat Cooperative Research Centre and a cluster of defence and data analytics firms, the duplication problem surfaces differently. Operators there working on satellite imagery pipelines and geospatial data products say automated ingestion systems can pull the same image tile multiple times when satellite passes overlap or when processing queues fail mid-run. The South Australian Space Industry Centre, also based at Lot Fourteen, has been expanding its data partnerships since 2024, which increases both the volume of imagery being handled and the likelihood of duplication events if deduplication logic is not applied at the ingestion layer.

The History Trust of South Australia, which manages the South Australian Museum collections and Migration Museum on Kintore Avenue, has publicly acknowledged working through legacy digitisation backlogs that stretch back to scanning campaigns conducted before consistent metadata standards were adopted. Without reliable metadata, identifying and removing duplicate image files requires manual review, slow work when collections run into the hundreds of thousands of items.

The Practical and Financial Stakes

Storage is not cheap and the costs compound quickly at government scale. Commercial cloud storage in Australia currently runs at roughly $25 to $40 per terabyte per month for enterprise-tier services, depending on the provider and redundancy configuration. Agencies carrying duplicate image files at volume, particularly high-resolution heritage scans or uncompressed satellite imagery, pay for that redundancy every billing cycle without any operational benefit. A collection carrying even a 15 percent duplication rate across a 100-terabyte archive absorbs thousands of dollars in avoidable monthly costs.

The solution most commonly advocated by digital archivists and database administrators is perceptual hashing, a technique that generates a compact fingerprint from an image's visual content rather than its file name or metadata, allowing near-identical images to be flagged even if they were saved under different names or at slightly different resolutions. Several open-source implementations exist and have been adopted by institutions in the United Kingdom and Canada, but uptake in South Australian government agencies has been uneven.

The practical path forward, according to discussions at digital heritage forums held in Adelaide over the past 18 months, involves three steps: audit existing collections to establish a baseline duplication rate, implement hash-based deduplication at the ingestion point for all new material, and establish a governance policy requiring image provenance documentation before files enter any public-facing system. Agencies that delay the audit phase typically find the retrospective workload doubles within two years as new material continues to arrive. For Adelaide's institutions, managing everything from colonial-era photographs to real-time satellite feeds, the window for a tidy fix is narrowing.