Adelaide's Digital Archives Face Duplicate Crisis Other Cities Already Solved

South Australia's major public institutions are sitting on tens of thousands of duplicate digital images across their collections — and the infrastructure to systematically identify and replace them is still catching up with the scale of the problem. The issue affects everything from the State Library of South Australia on North Terrace to the History Trust of South Australia, whose photographic archives span well over a century of the state's past.

The timing matters. Adelaide is mid-way through a significant expansion of its digital public infrastructure, anchored by the Lot Fourteen innovation precinct on the former Royal Adelaide Hospital site. That precinct has drawn data-focused organisations and tech startups into the city, and it has also raised expectations about what government-linked institutions should be capable of delivering in terms of searchable, reliable digital records. When duplicate images muddy catalogue searches or serve low-resolution versions of photographs that exist in higher quality elsewhere in the same collection, the practical cost is real — researchers waste hours, educators get frustrated, and heritage records lose their authority.

What 'Duplicate Image Replacement' Actually Means — and Why It's Hard

Duplicate image replacement is not simply deleting copies. Institutions have to determine which version of an image is the master — highest resolution, most accurate metadata, correct rights status — before anything else can be retired or redirected. A photograph from the 1930s Hindmarsh Square redevelopment, for example, might exist as a faded scan from 1995, a better scan from 2008, and a high-resolution version uploaded in 2021 under a different catalogue number. Without a perceptual hashing system or similar automated tool to flag them as related, staff must find the overlap manually.

Amsterdam's Rijksmuseum completed a major duplicate-resolution program across its online collection by 2023, deploying open-source image fingerprinting tools that processed roughly 700,000 object records. The British Library ran a parallel audit of its digitised newspaper archive, which identified duplicate page scans running into the hundreds of thousands. Both institutions published their methodology openly. Adelaide's counterparts have not yet done the same at comparable scale, though the State Library confirmed in its 2024-25 annual report that catalogue integrity work was ongoing — without specifying the scope of duplicate records under review.

The contrast with cities of similar size is instructive. Helsinki's public library network, Helmet, completed a collection deduplication project in 2024 covering around 1.2 million digital items, using a combination of automated tooling and a small dedicated team of two archivists over 18 months. Wellington's Alexander Turnbull Library in New Zealand launched a structured duplicate audit in early 2025 tied directly to its Te Ara digital encyclopedia refresh. Neither city is dramatically larger than Adelaide in population terms, yet both had formalised programs with published timelines.

Where Adelaide Stands — and What Comes Next

The good news is that the institutional appetite exists. The Lot Fourteen precinct hosts the Australian Space Agency and a cluster of AI-adjacent firms, some of which have informal research relationships with the University of Adelaide and Flinders University. Computer vision tools capable of near-instant perceptual hashing of large image libraries are no longer expensive or exotic. The barrier is less technological than organisational: procurement cycles, staff training, and the decision about who owns the problem across agencies.

The History Trust of South Australia, which manages Migration Museum on Kintore Avenue and the South Australian Museum's social history collections, holds one of the densest photographic records of any state institution. Getting its duplicate pipeline sorted would be a meaningful proof of concept for the broader sector.

For researchers working at institutions like the J.D. Somerville Oral History Collection at the State Library, the practical advice right now is straightforward: always cross-check catalogue entries against the institution's direct Flickr Commons uploads and Trove holdings before assuming the version you find first is the best available. Higher-resolution originals often exist under different catalogue numbers and can be requested directly from digitisation staff.

A formal statewide strategy, if one emerges from the SA Government's ongoing digital infrastructure review, could bring Adelaide level with its international peers before the end of 2027. Without one, the backlog will keep growing.