How Adelaide's Digital Archives Ended Up Full of Duplicate Images — and What It's Taking to Fix Them

South Australia's public sector is sitting on a digital storage problem that has been years in the making. Across government agencies, cultural institutions, and the technology precinct at Lot Fourteen on North Terrace, IT administrators are now working through backlogs of duplicated image files that accumulated during a decade of rushed digitisation programs, migration projects, and pandemic-era remote work uploads.

The issue matters now because the stakes have changed. With the AUKUS submarine program drawing sensitive defence contractors to Edinburgh Parks and Osborne Naval Shipyard, and with the state's Hydrogen Jobs Plan generating thousands of new technical documents and site photography, the cost of poor digital asset management is no longer just an administrative headache — it is a budget and security concern.

How the Duplication Problem Built Up

The roots of the problem stretch back to roughly 2015, when South Australian agencies began accelerating their shift from paper-based filing to digital repositories. The State Records Act 1997 requires government bodies to preserve public records, but the legislation predates cloud storage and offers limited practical guidance on deduplication. When agencies migrated to new platforms — often under tight deadlines and with limited IT resourcing — files were copied rather than moved, creating parallel libraries that then diverged as staff updated some versions but not others.

The Australian Bureau of Statistics estimated in its 2023–24 Government Finance Statistics release that general government IT expenditure across state and territory bodies exceeded $14 billion nationally in that financial year. South Australia's share of that figure runs to hundreds of millions of dollars annually, and storage costs represent a meaningful slice of it. Industry practitioners typically cite duplicate files as accounting for between 20 and 30 per cent of unmanaged digital repositories, though figures vary widely depending on an organisation's migration history.

At Lot Fourteen, where the Australian Space Agency headquarters sits alongside defence tech startups and the Australian Institute of Machine Learning, the problem has a particular edge. Several tenants operate under data-handling agreements that require clean, auditable file libraries. Duplicate images — especially where metadata timestamps differ between copies — can trigger compliance flags during routine audits.

Adelaide Institutions Now Leading the Response

The State Library of South Australia on North Terrace began a formal duplicate-image-identification project in late 2024, working through its Digitised Collections holdings that span historical photographs, maps, and government reports. The project uses perceptual hashing, a technique that identifies visually identical or near-identical images even when file names and formats differ. Libraries elsewhere in Australia, including the State Library of New South Wales, have run similar programs.

Renewal SA, the agency managing several of Adelaide's major development precincts, updated its internal asset management policy in January 2026 to require deduplication checks before any new image set is ingested into its project documentation systems. The change followed an internal review that found duplicated construction-site photography from the Riverbank Precinct redevelopment was consuming storage that carried a direct cost to the agency.

For smaller organisations — the dozens of technology businesses operating out of Lot Fourteen or defence subcontractors running lean IT teams near Port Adelaide — the practical advice from the South Australian Chief Information Officer directorate has been to prioritise deduplication before any major platform migration, not after. Cleaning a repository of 500,000 files before a cloud move costs a fraction of what it costs to clean 1.5 million files once a new platform is live and staff are already working inside it.

The state government's Digital Strategy 2025–2030, published by the Department of the Premier and Cabinet, identifies data integrity as a foundational priority, though it stops short of mandating specific deduplication schedules for individual agencies. Advocates within the public sector argue that without binding timelines, the cleanup will remain piecemeal — with some departments making real progress while others continue generating new duplicates faster than old ones are removed. The Lot Fourteen precinct's next major infrastructure expansion is scheduled for completion in 2027, which gives agencies a practical deadline: get the libraries in order before another wave of new tenants, new systems, and new uploads arrives.