Adelaide's Duplicate Image Problem: How the City Stacks Up Against Global Peers

Adelaide's cultural and government institutions are sitting on a growing backlog of duplicate digital images — redundant scans, re-uploaded photographs and copied asset files clogging storage systems and distorting public-facing archives — and the methods being used to clean them up are drawing comparisons, not always flattering ones, with approaches taken in cities that moved earlier on the problem.

The issue has sharpened in 2026 for a specific reason: the State Government's digital infrastructure push, centred on the Lot Fourteen precinct on North Terrace, has accelerated the migration of legacy datasets from agencies including History Trust of South Australia and the State Library of South Australia into shared cloud environments. When analogue collections digitised at different times by different departments land in the same repository, duplicates multiply fast. Archivists at the State Library, which holds more than 750,000 photographic items in its collection, have flagged the deduplication workload as a recurring challenge in annual reporting cycles.

What Other Cities Did First

Amsterdam's Rijksmuseum completed a large-scale deduplication audit of its Rijksstudio platform between 2022 and 2024, deploying perceptual hashing algorithms — software that assigns a fingerprint to each image based on visual content rather than file metadata — to identify near-identical copies across more than 700,000 digitised objects. The process cut redundant storage load and improved search precision for the platform's millions of annual users. Toronto Public Library undertook a similar exercise across its digital newspaper archive in 2023, eliminating an estimated 12 percent of catalogued image entries that were confirmed duplicates or near-duplicates introduced during batch scanning campaigns.

Singapore's National Heritage Board went further, mandating from January 2025 that any new digitisation contract awarded through its Cultural Matching Fund must include a deduplication protocol as a deliverable, not an optional step. Adelaide has no equivalent mandate yet, though the Office of the Chief Digital Officer, based at Lot Fourteen, has been developing a broader data quality framework under the State Government's Digital Strategy 2025–2030 that sources familiar with the process say is expected to address image asset management.

The contrast with Singapore's contractual approach is pointed. Where Singapore baked deduplication into procurement from the supply side, South Australian institutions have largely handled it in-house, institution by institution, with uneven results. The State Records office on Leigh Street administers guidance on digital preservation standards, but deduplication methodology sits outside its formal mandate as currently defined.

The Local Picture

At Lot Fourteen, the Australian Space Agency and several resident technology companies have dealt with the duplicate image question in a narrower but instructive context: satellite and earth-observation imagery ingested from multiple feeds frequently arrives with overlapping coverage, requiring automated triage before data is usable. The workflows developed there — largely built around open-source tools including Python-based image comparison libraries — have attracted interest from the History Trust, which manages collections spanning sites from Tandanya National Aboriginal Cultural Institute on Grenfell Street to the Migration Museum on Kintore Avenue.

The History Trust declined to provide specific figures on its current duplicate image count, but its 2024–25 annual report noted that collection digitisation remained a priority and that data quality improvements were ongoing. The State Library's most recent collection development policy, updated in March 2025, references metadata integrity but does not set a numerical deduplication target.

The practical stakes are not trivial. Cloud storage costs for South Australian government agencies are billed through a whole-of-government contract, meaning redundant image files translate directly into budget line items. Internationally, the Toronto exercise demonstrated that even a modest cull of 12 percent of duplicated entries can produce meaningful storage savings across a collection of several hundred thousand records.

For Adelaide, the clearest short-term path is the one Singapore chose: attach deduplication requirements to digitisation contracts rather than treating them as a retrospective cleanup task. The Office of the Chief Digital Officer is expected to release updated data quality guidelines before the end of 2026. Whether those guidelines will include binding deduplication standards — or leave it to individual agencies — will determine whether Adelaide closes the gap on its more methodical peers or continues managing the problem one hard drive at a time.