Adelaide's Digital Archives Are Drowning in Duplicate Images — and the Numbers Tell the Story

South Australian cultural and government institutions are sitting on hundreds of thousands of duplicate digital image files, a problem that specialists say is measurable, costly and — critically — fixable. The issue has come into sharp focus in 2026 as organisations from Lot Fourteen on North Terrace to the State Records office on Leigh Street accelerate digitisation programs tied to defence industry documentation, space sector growth and heritage archiving.

Duplicate image replacement — the systematic process of identifying, cataloguing and replacing redundant digital files with a single master copy — sounds mundane. The cost of ignoring it is not. Storage infrastructure is not free, and when an organisation holds three or four copies of the same high-resolution scan, those redundant gigabytes compound across thousands of records.

What the Numbers Actually Show

Industry benchmarks from digital asset management research published by organisations including the international Digital Preservation Coalition suggest that duplicate files can account for between 20 and 40 per cent of total storage volume in institutions that have not run systematic deduplication programs. For a mid-sized archive holding 50 terabytes of image data — not an unusual figure for an institution like the History Trust of South Australia, which operates out of the Migration Museum on Kintore Avenue — that range implies between 10 and 20 terabytes of potentially redundant material.

Cloud storage costs in Australia have generally tracked between $20 and $30 per terabyte per month for enterprise-tier services through major providers, though government procurement arrangements can shift that figure. At even the lower end, 10 terabytes of unnecessary storage runs to roughly $200 a month, or $2,400 a year — for a single institution. Multiply that across the dozen-plus agencies feeding into SA Government's GovTEAMS and whole-of-government digital infrastructure, and the cumulative drag becomes significant.

The timing matters because 2026 is the year several large digitisation pipelines are generating fresh image volumes simultaneously. The Lot Fourteen precinct, which houses the Australian Space Agency and a cluster of defence and tech tenants, has been producing technical documentation, satellite imagery and engineering schematics at pace. The Australian Space Agency relocated its headquarters to Lot Fourteen in 2019, and the precinct has expanded steadily since. Each new tenant adds its own digital asset workflows — and its own duplication risks when files are shared across teams without a centralised asset management layer.

The Local Deduplication Gap

The South Australian Film Corporation, based on Hendon's Film Studios site in the western suburbs, provides another illustration. Screen production generates enormous image libraries — stills, frame grabs, promotional assets — and productions that pass through multiple post-production hands frequently duplicate files across server environments. A single feature film can produce upwards of 50,000 individual still image files before a deduplication audit is run.

Automated deduplication tools use perceptual hashing algorithms — software that compares image content rather than just file names or sizes — to identify near-identical images even when they have been resaved, reformatted or slightly cropped. These tools have become more accessible since 2022, with open-source options available alongside commercial platforms. The practical challenge for Adelaide institutions is not the technology itself; it is the workflow discipline to run deduplication checks at ingestion rather than retrospectively.

Retrospective deduplication projects — going back through existing archives — consistently take longer and cost more than building deduplication into the front end of a digitisation workflow. A project auditing 100,000 images retrospectively typically requires between 40 and 80 hours of specialist labour depending on the consistency of original file naming conventions, according to published case studies from the Digital Preservation Coalition.

For SA institutions accelerating their digital programs through 2026 and into the lead-up to the AUKUS submarine build's documentation requirements at Osborne Naval Shipyard, the practical advice from the sector is consistent: implement hash-based duplicate detection at the point of ingest, establish a single master file policy before storage volumes grow further, and schedule an audit of existing holdings before the next major infrastructure procurement cycle. The numbers behind this story are not dramatic. They are steady, accumulating and entirely preventable.