Duplicate Images in Adelaide's Digital Archives: The Key Decisions Ahead

South Australian institutions managing thousands of digitised photographs, planning maps and satellite images face a reckoning over duplicate files that are quietly eating through storage budgets and slowing down public-access databases. The problem is not new, but the scale has shifted: the rapid expansion of Lot Fourteen's data and technology precinct in the city's east, combined with the state government's broader digital infrastructure push, has concentrated the issue in ways that are now impossible to ignore.

Procurement officers and archive managers across agencies have been quietly flagging the same headache — images captured, uploaded and catalogued multiple times as different teams work from different systems. A single aerial photograph of the Port Adelaide waterfront, for instance, might exist in three separate formats across a council GIS layer, a state heritage database and an AUKUS-related planning file, none of them tagged to indicate duplication. Multiply that across years of datasets and the redundancy compounds fast.

Why This Matters Right Now

The timing is not accidental. The SA government's hydrogen jobs plan, which is directing significant public investment toward green energy infrastructure in the Upper Spencer Gulf region, relies on accurate and deduplicated spatial imagery to support environmental assessments and site surveys. Feeding redundant data into those workflows carries real costs — not just in storage, but in analyst time spent reconciling conflicting file versions. The Olympic Dam expansion program faces similar demands, with geological survey imagery central to approvals processes managed partly through databases held at the Department for Energy and Mining on Grenfell Street in the CBD.

Lot Fourteen, the former Royal Adelaide Hospital site on North Terrace, houses multiple tenants — including the Australian Space Agency and various defence tech startups — whose work depends on clean, non-duplicated image libraries. When image management goes wrong at that level, the downstream errors can affect contract deliverables and compliance reporting. The precinct's growth, with dozens of organisations now operating from the site, has made shared data governance a live issue rather than a theoretical one.

The University of Adelaide's digital collections team and Flinders University's geospatial research groups have both been working on deduplication tools, though the approaches differ. Hashing algorithms that generate a unique fingerprint for each image file are the most reliable method for catching exact copies, but perceptual hashing — which catches near-identical images taken seconds apart — adds another layer of complexity when dealing with field photography or drone footage.

What Happens Next: The Decisions That Can't Wait

Three practical choices are coming to a head for agencies and institutions across Greater Adelaide. First, whether to centralise image repositories under a single state-managed platform or allow individual agencies to maintain their own systems with shared deduplication standards applied at the point of ingest. The second is a budget question: enterprise-grade deduplication software licences from vendors such as Wasabi or Cloudian can run to tens of thousands of dollars annually for large collections, a figure that matters considerably when council IT budgets are under pressure following rate-cap negotiations earlier this year. Third, and most consequential, is the question of what happens to the duplicate files once identified — whether they are deleted outright, archived offline, or flagged for human review before any action is taken.

Heritage advocates have a specific concern here. The History Trust of South Australia, based on Kintore Avenue, has argued in previous submissions that automated deletion without curatorial review risks permanent loss of contextually significant variants — images that look identical but carry different metadata, capture dates or provenance notes. That concern has slowed decision-making at several institutions.

The most likely near-term outcome is a pilot program, probably centred on one or two Lot Fourteen tenants and coordinated through the state's Office of the Chief Digital Officer, to test a common tagging and flagging protocol before any deletion policy is formalised. If that pilot gets underway before the end of 2026, it could set the template for every major SA public institution managing visual data. If it stalls, the problem simply grows more expensive to fix — and the files keep multiplying.