The National Archives of Australia’s role is to promote the creation, management and preservation of authentic, reliable and usable Commonwealth records and enable ongoing access to the archival resources of the Commonwealth.
Records currently being created by Commonwealth agencies are predominantly digital. Under the Digital Transition Policy, any records that are created digitally after 2015 must be kept in a digital format, and any that are identified as RNA (Retain as National Archives) must be transferred to the National Archives in digital format.
Managing digital records brings a range of challenges, and the National Archives must further develop its business capabilities in order to sustainably harvest, preserve and provide access to its born-digital collection, as well as increase its capacity to provide online access to its analogue collection.
The primary challenges of digital records for the National Archives in implementing sustainable business processes for managing the lifecycle of digital records from their creation to providing access are outlined in the table below:
The Archives End-to-End Digital Business Solution Design has been created to provide an end-to-end system for digital records that will enable the National Archives to effectively meet the needs of Commonwealth agencies and public clients. Based on the Open Archival Information System (OAIS) framework, it will:
The target state architecture envisaged in the Archives Digital Business System will meet the challenges within the archival lifecycle for digital records in a scalable, flexible and sustainable way – with as much automation of business processes as possible.
Commonwealth Agencies will have their record producing and management systems integrated into the Archives Distributed Digital Record System (ADDRS). Records will be exported and/or harvested, batched and transferred to the Archives via an automated process over the most appropriate channel.
The records will then be quality assured and stored within the Store Cloud for further processing.
Figure 1: Digital Records – From Agency to ingest within the Archives
Manage Agency Record Authority & Exception Handling
Monitor Agency Record Keeping Notify of Disposal Freeze Government wide Rule Implementation
Agency Business System, Agency EDRMS, ERP and Agency Websites all feed into ADDRS Pop which consists of:
This feeds between the Internal DMZ, NAA Internal DMZ, ADDRS Net and a Submit Information Package to the ADDRS Store.
Scheduled Scanning of EDRMS & Agency Websites
Item Preservation Format
Transfer Manifest & Schedule
Transfer – Cache and Send - Channel Supports network or physical
Reports flows forward
Transfer Requests flows forward
Metadata flows forward
Items flows forward
Transfer Report flows backward
Governance flows backward
Archive Notifications flows backward
Transfer - Receive & Acknowledge
Clients will search for the records via a find and display function. The records will be retrieved by a workflow running within the workflow controller. The workflow will retrieve the records, converting them into an appropriate access format.
The retrieved records will be published to a delivery platform most appropriate for the client. For example, a small digital file for consumption over the Internet, would be published to the Local Content Distribution Server and be accessible via a web download; whereas a large video may be pushed to the Content Distribution Network for retrieval by the client using a streaming video player.
Figure 2: Digital Records – From Archives to external clients
Retrieve Information Package
Publish Information Package Request
Publish to Offline Copy Creation
Publish to Local Content Distribution Server
Publish to Content Distribution Network (Public)
The Archives has been delivering its services online for many years. Currently 99% of collection access occurs online, in digital format. A key priority for the Archives is to build business effectiveness to deliver innovative, responsive and appropriate services that meet the needs of government, the Australian people and other key stakeholders. These online services will continue to be expanded through:
The scope of the Archives Digital Business System will support the Archives' business needs by:
Benefits delivered by the Archives Digital Business System over time include:
The project is to be delivered in four iterations. Each of those iterations will deliver increased capability over time, before the entire end-to-end system becomes entirely business as usual.
Design and trial using Archives data from its internal Electronic document and record management system (EDRMS)
Extend the trial to transferring data from selected partner agencies to test scalability & automation of transfer & storage
Further extend the trial to ensure all Archives' clients can access digital records in an appropriately controlled environment
Moving into full production, trialling extraction of agency data directly from the agency and/or gateways
System in place as business as usual for transferring, preserving, managing and providing access to digital records
June 2014 – July 2015 was dedicated to undertaking a Archives Digital Business System proof of concept stage in order to identify and mitigate the highest levels of risks that were identified in the Architectural Blueprint. This was broken into five projects, each with its own purpose and scope. Each separate project was scoped. The projects were:
The proof of concept has used the Archives’ data from its internal TRIM EDRMS as the basis for this work, with the addition of other collection and external data sets to test the Access Examination process as well as Search and Discovery.
July 2015 marked a successful end to the proof of concept stage of the project with all major objectives and milestones being met. In précis, the project team and associated partners within the Archive have demonstrated the viability of Chrysalis by proving it is possible to develop a suite of software to assist record keepers in the digital age by:
Lessons learnt from the build of the proof of concept will inform decisions, processes and planning required for the development of the prototype. The end-to-end prototype is scheduled to be completed by June 2016.