Duplicate Images in Adelaide's Digital Archives: What Happens Next and the Key Decisions Ahead

South Australian cultural and government institutions are confronting a decision point over how to handle tens of thousands of duplicate digital images sitting inside their collections management systems — files that consume server storage, confuse search results and complicate public access to records built up over decades.

The issue has sharpened this year because several major Adelaide organisations are mid-cycle on digital infrastructure contracts. Decisions made before the end of the 2026 financial year will lock in approaches for the better part of a decade. For institutions already navigating tight budgets and expanding mandates — particularly those tied to the state government's Lot Fourteen tech and space precinct on North Terrace — the cost of getting it wrong is real.

Why the Problem Compounds Quickly

Duplicate images accumulate for mundane reasons. A staff member scans a photograph twice under different filenames. A migration from one database to another copies records without deduplication checks. A contractor delivering digitisation work submits both a raw and a processed version of every image, and nobody deletes the originals. Across a single mid-sized institution running 500,000 records, duplication rates of 10 to 15 percent are not unusual in the sector — meaning upward of 75,000 redundant files can sit undetected inside a single collection.

The History Trust of South Australia, which manages the South Australian Museum collections framework and operates out of the Migration Museum on Kintore Avenue, has been publicly expanding its digitisation program in recent years. The State Library of South Australia, a few hundred metres east along North Terrace, holds photographic archives stretching back to the 1840s. Both institutions face the same structural question: once duplicates are identified, who decides which version is the authoritative record, who carries liability if the wrong file is deleted, and who funds the remediation work?

These are not purely technical questions. They touch on chain-of-custody obligations under the State Records Act 1997 (SA) and on public accountability for how government-funded collections are maintained.

The Decision Framework Taking Shape

Three broad paths are available to institutions working through duplicate image backlogs. Manual review — a human archivist assessing each flagged pair — is the most defensible but also the most expensive, typically costing between $8 and $15 per image decision when staff time, quality control and documentation are factored in. Automated deduplication tools using perceptual hashing or AI-based comparison can process thousands of images per hour at a fraction of that cost, but require a governance policy that pre-authorises deletion thresholds without case-by-case sign-off. The third option is hybrid: automated flagging followed by human approval at defined confidence intervals.

Lot Fourteen, the 7.6-hectare former Royal Adelaide Hospital site now home to the Australian Space Agency headquarters and dozens of tech startups, has become a focal point for conversations about exactly this kind of government-adjacent data challenge. Several Lot Fourteen tenants working in machine learning and data infrastructure have pitched services to South Australian public sector clients in the past 18 months, according to procurement documents available through the SA government's buy.sa.gov.au portal.

The state government's broader digital strategy, updated in late 2024, encourages agencies to adopt reusable platforms rather than bespoke solutions — a principle that applies equally to collections management.

Timing matters for another reason: the July 2026 federal budget cycle has released a tranche of funding under the National Cultural Policy for digitisation and preservation work at state institutions. Institutions that can demonstrate a remediation plan — including a deduplication methodology — before applications close will be better placed to access those grants.

The immediate practical steps for any Adelaide institution sitting on an unresolved duplicate backlog are clear: commission a collection audit scoped specifically to identify duplicate and near-duplicate files; draft a disposal authority under the State Records Act that covers automated deletion of confirmed duplicates; and identify whether existing enterprise licence agreements with vendors like DAMS providers already include deduplication modules that are simply switched off. Many do. The decision ahead is less about technology and more about who inside the organisation has the authority to sign off — and whether they are willing to.