A speech given by Director-General David Fricker at the Archives and Records Association of New Zealand (ARANZ) Conference in Wellington, 23 October 2012.
It's a great pleasure for me to talk to you today. The theme today is 'Inside and outside the box' and my contributions to the discussion are my thoughts on the Business Model for the National Archives of Australia, and the reforms we should be looking at as we operate in the digital age.
I will focus my remarks on the basic elements of our business: the records, custody, preservation, services, and the legislative framework that enables all of our activities.
Every archive around the world has its own unique purpose or mission that defines its roles and responsibilities, within a particular scope of records, policies, regulations and, of course, legislation. These factors notwithstanding, we are all operating on the basis of common archival concepts.
We speak of 'records', indeed 'unique record' which is the 'authentic, reliable and usable' thing that can be relied upon as the 'evidence' of business activity.
The records are 'transferred' to us by agencies, donors, governments, etc. They are then in our 'custody', for our 'appraisal', 'arrangement, description', with regard to their 'provenance' and 'original order'.
We 'preserve' the records in their proper context, with all necessary arrangements to ensure they're usable by all future generations.
And of course we do all of this to ensure we can provide 'access', for without access there is no purpose to preservation or in collecting the records in the first place.
We provide services to researchers, genealogists, historians, government agencies and individuals, who by and large come to us for the records we hold. This could be in person or online, but essentially they are making a decision to visit the Archives to access archival records.
Our value proposition? Our differentiator? We guarantee the long-term availability of the authentic source record, the original – the reliable, irrefutable evidence.
At the National Archives of Australia we think about these basic concepts a lot – because every one of them will be seriously challenged by our digital society – and this will happen well within the next decade. Which means our preparations should have already started by now!
The National Archives of Australia was established following the passing of the Archives Act in 1983.
A program of 'major change' led to the launch of the National Archives' e-permanence range of products in the year 2000. The e-permanence suite of tools and guidelines, based on the concepts and strategies recommended in AS 4390 and later ISO 15489, provided the framework for government agencies to develop systems to make and keep good records – including records 'born digital'.
From 2002 to 2006 the Australian National Audit Office released reports advising on government recordkeeping, the last of which made recommendations resulting in the 2007 launch of Check-up and then, in 2011, Check-up 2.0 – a practical example of the role of the Archives in assisting government departments to self-assess the state of their records and information management.
Also in 2011, the Australian Government established the Digital Transition Policy. For many government agencies the new policy means moving from paper-based records management to digital information and records management. To support the new government policy, the Archives developed the Digital Continuity Plan, which provides guidance to government agencies on the management of digital information for as long as it is needed.
In accordance with this policy, the National Archives, in consultation with other Australian Government agencies, has set a target to limit the creation of paper records.
Our expectation is that, by 2015, information that is'born digital' – that is, information that is created using computers or other digital devices which means anything created other than with a pen or a chisel – will be stored and managed digitally and subsequently transferred in digital formats to the National Archives.
Of course, none of this work is done in isolation. Collaborations, both direct and indirect, are crucial to the ability of the Archives, and I would say the profession generally, to develop new products and to improve skills and abilities.
I would like to specifically acknowledge our collaborations across Australian State and Territory borders, including across 'the ditch' here in New Zealand and beyond, to our Pacific region, that have helped position this part of the world as a standards setter.
But 'what got us here, won't get us there'; there's still plenty to rethink and re-engineer if we are to take every opportunity offered by the digital age, and meet the expectations of the digital natives who are our next wave of clients.
So what are the megatrends that are disrupting our current business model, and how are we framing our position as Australia's National Archives?
Let's consider the environment we are operating in, the environment our business model has to be robust enough to survive.
The Australian Government will be releasing its National Cultural Policy later this year.
One of the cultural policy's proposed goals focuses on technology, innovation and participation – using emerging technologies and rich information sources to enable more people to access and participate in the cultural development of the nation.
In 2009, the Government 2.0 Taskforce presented its Engage: getting on with government report to the government. The government's response the following year confirmed a commitment to open government, calling for openness and transparency, for example through a 'pro-disclosure' culture and greater engagement and participation of the public in agency and policy activities, particularly through online sources.
The Australian Government's support for openness and transparency was expressed in our Declaration of Open Government, and in this context we are seeing increased use of a 'co-design' approach to government policy and programs, using Web 2.0 technologies to put the citizen at the centre of planning, and inviting more participation in the formulation and indeed delivery of those policies and programs. We are saying to the citizen that we are delivering services 'the way you want it' as opposed to the way the government departments are organised.
This move to a more inclusive, open government is matched by increasing levels of scrutiny of the actions of organisations – particularly government organisations. People are less tolerant of poor governance, they expect to find answers in government records and rightly expect to be able to trust that those records will be authentic and have integrity.
In 2010 the Australian Government published Ahead of the game, a blueprint for reform of Australian Government administration. This focuses on building a culture of independence, excellence and innovation in policy advice and service delivery. Information sharing, information access and effective information management underpin many of the recommendations.
There is also an expectation of easy access to all information, whether official records or not, in an age where information is available 24/7 from a myriad of sources.
Government information is increasingly regarded as public information – not the sole property of government but 'democratised' and 'socialised' through developments such as Information Disclosure policies, social media, Wikipedia, crowdsourcing, and blogs.
This is a globalised information marketplace, and it's not just being shaped by government institutions like mine – it's being shaped by market forces.
And of course in a heated and dynamic marketplace, the right to privacy and requirements of national security must be reconciled and balanced with principles of freedom of information.
The business of government is increasingly conducted in cyberspace, with a consequential fragmentation of information holdings across many digital platforms including email servers, social media accounts, electronic business systems, shared drives, mobile devices and Electronic Document and Records Management Systems (EDRMS) – and the list is growing. For more than a decade, government policy around the world has preferred that services and information be provided to citizens by digital means. Recent policy initiatives such as the Australian Government's Declaration of Open Government and legislative changes are accelerating this requirement. Government information is certainly not neatly 'inside the box' any more.
These are exciting times and there is no doubt that huge leaps are being made in day-to-day efficiency and utility of information, but we are at grave risk of losing any substantive archival memory of our time: this fragmented information could evaporate as easily as it was created.
In Australia, the Australian National Audit Office has just released a report into records management in several government departments. This strongly supported the Archives' earlier findings that Australian Government agencies create a substantial amount of electronic information and records as part of their normal operations. However, in 2009 less than 30 per cent of these agencies and bodies managed the majority of their records digitally, even though more than half reported having an EDRMS and using other electronic business systems to manage records. Many other electronic business systems that were not identified and functioning as 'records management systems' were also used by the agencies to create, capture and manage records. These systems did not generally meet legal requirements relating to the management and destruction or transfer of records.
So how does this challenge our business model? Let's consider some of those basic concepts I mentioned earlier.
At the heart of everything we do is this concept of 'the record'. Since taking up my position with the Archives, I have been most surprised by how mercurial this concept is, and how subtle differences in our definitions can lead to major policy dilemmas. So what is a 'record' in our business model for the digital age?
My simplistic, practical view is that the record is the 'thing' that provides evidence of business activity.
A digital record exists in a virtual space. It is the information, the knowledge, or the experience – but it is not the medium that carries it.
In the digital environment a record is 'made' as part of a business process. As far as the record's creator is concerned, it might be produced deliberately, incidentally, accidentally, or even invisibly. But my view is that it doesn't wait to 'be made' by a separate act. It exists and has meaning well before anyone has separately 'put it to file'.
It has a digital context that is global, interconnected and commercialised – comprising collections which have no meaningful physical presence. A business transaction can traverse an email, a tweet, a business system, a voicemail, a website. The provenance of the record is clear; however, in a globalised, joined-up world, historians will interpret provenance in different way, and may present new challenges for 21st-century archivists.
The digital record typically comes loaded with metadata offering many new opportunities for more comprehensive and accurate appraisal, description, arrangement, discovery and retrieval.
A digital record may move from one medium to another, as it cycles through ICT maintenance and data migration projects, from disk to tape and back again. In order to use the record, it will also be rendered many times over – sometimes on a computer screen, projected on a wall, or printed on a sheet of paper. This movement or viewing of the record does not alter it, nor is it 'reborn' – the magnetic tape, computer screen, wall or paper does not become part of the record.
Our business model must recognise this, both in practice and in law. It is essential that the record maintains its unique identity as it moves from one medium to the next, ensuring that:
If you accept this definition of a record, a somewhat mercurial thing that maintains its identity but changes its form, then you must be clear about what it is we are preserving.
We are framing our thinking around the concept of the 'essential performance'. This is a concept that accepts that the digital information must endure well beyond the utility of the medium that carries the information. It requires a formal mechanism for determining the characteristics that must be preserved for the record to retain its meaning over time, and, therefore, those attributes that are only serving the technology of the day and can be left behind.
In our Digital Business Model, preservation must start well before the originator even thinks about the Archives. It must start the moment the record is created.
The issue across the Australian scene is that records are initially created in digital form, but are often printed to paper for records management and preservation purposes. This presents many inefficiencies and costs for the agency, but also loses much of the important metadata originally integral to the digital record. To address this, in 2011 the Australian Government announced the Digital Transition Policy, which will see all Australian Government agencies moving from paper-based records management to digital information and records management. To support this policy, the Archives developed a Digital Continuity Plan, which provides guidance to government agencies on the management of digital information for as long as it is required.
The Digital Continuity Plan (which is published on our website) has at its core six principles which should be incorporated into the governance and strategy of all government institutions:
Another aspect of the digital business model is the 'transfer' or 'accessioning' of records into our custody.
Custody in the virtual world is more about 'control', less reliant on physical location or proximity. Data belongs to an enterprise but is usually housed in a remote data centre, a dark room many kilometres away. Or indeed in a cloud, which may itself draw upon a geographically diverse array of servers and data stores. The point here is that 'custody' means protecting, monitoring and controlling records – not building a brick wall around them and commanding exclusive use of the record.
So how will this shape our business model?
For one thing, we may not have to wait for an agency to transfer records to the Archives. Based on the metadata and the business rules that we embed within the agency's systems, the record could 'transfer itself'.
More and more, an organisation's information management is conducted in the digital environment – whether in a specific recordkeeping system or any of the electronic business systems running within the enterprise.
Our future in this environment – in particular, to guarantee preservation of the record and proper and timely custody of the record by the Archives ¬– is to embed business rules within those systems; in effect, to automate as much as possible so that the sound practices and proper information governance are a natural by-product of the business process, not a separate activity occurring several years after the event.
In this model, the Archives could 'have' the record as soon as it's created. Why wait until it's too late?
Taking the digital record means that we're not preventing the ongoing use of the record by the originating agency and there are no constant requests to lend or transfer the records between physical locations.
An image of the record may still be in administrative use; however, its preservation would be assured. In the digital world, a record can be in many places at once – the important thing is to ensure that the authentic version is secure, preserved and accessible.
The embedding of business rules in the source systems will also ensure collection of the optimum level of metadata. This will provide enormous benefits as we are able to automate many aspects of accessioning, arrangement and description of our archival resources.
Access examination, to properly consider privacy, security and public interest issues, will be far more efficient and effective, supported by advanced data analysis and retrieval tools – taxonomies and ontologies that provide for concept-based searching to fully appreciate the significance and interrelationships of the information in our care. I'll return to this in a moment.
And of course the originating agency will have a more complete and accurate repository available whenever it requires access to its preserved records. Current 'lending' and 'transfer' processes, which are necessary for the physical movement, tracking and care of paper documents, will be replaced with continuous online access to electronic data, provided with no risk to archival material.
In the digital world, making information accessible means rendering the essential performance.
The expectation is that this will not require a trip to a reading room or a wait until an exhibition is mounted.
From our citizens' point of view, it must be 'when I want it' and 'where I am, right now' – be it at home, in the office or riding on the train.
In this environment, our records will be intermingled with all the other information out there, and the consumer will take the easiest, most convenient path to satisfy his or her query or research requirement.
Queries won't always start with a virtual visit to that archive's website, but instead will be serviced by third-party federated searches and concept-based algorithms for discovery and assembly of information into highly customised factsheets or result sets, by commercial portal services influenced by advertising revenue or a paywall.
This is, as I said earlier, an information marketplace. A crowded marketplace at that, with more and more choice and power to the consumer.
I think the majority of people in this marketplace won't be the so-called 'serious' researchers – they won't want to retrieve a series, or perhaps even an item or a record. They will want a 'factoid', a piece of information that fills a specific gap in their knowledge or answers a narrow question – a date, a name, a place or a quote. They might stop looking as soon as they believe they have what they want.
A really interesting development is augmented reality. Anyone with a smart device can simply point at a document, a building, a monument or a landscape and enjoy a rich array of audiovisual information that brings the object to life.
This means that people will not even have to type a query, or think of a question to ask. They won't have to know that the Archives has relevant and interesting information. It will be possible for us to augment the object itself with our archival resources, presenting it in an interesting and engaging way – perhaps inviting the person in to explore our archives even further.
Another key development in expanding access is crowdsourcing. As many of us are now seeing, by allowing the public to contribute to the description of archival resources we are enhancing the ability of future generations to discover and learn from our archives. I also think it is a wonderful opportunity for the public to be more engaged with us as archives and to share in the work we do – preserving the memories of our nations.
This is a type of the co-design, citizen first activity I mentioned earlier: drawing on the interest and enthusiasm of the community to bring more of our archives into view – discoverable and retrievable.
The slide I have here shows the National Archives of Australia's most recent adventure into this area, Destination: Australia.
Unofficially implemented in June this year, Destination: Australia is now building a user community through word of mouth and some mainstream media attention. We have around 50 people adding tags, with around 150–200 visits per day.
The featured photographs come from a promotional series of photographs taken by the Department of Immigration, which are now stored as the Immigration Photographic Archive collection.
The series contains more than 25,000 photographs and over 21,000 of those are featured on this site. With nearly six million migrants to Australia since 1945 it would be challenging to try to identify everyone who might appear in the photographs, and to collect and share their stories. But with the help of all of Australia, we are going to try.
Increasingly, our information and our services are contestable; that is, the records we hold will be held in any number of other places, and will be discoverable and retrievable by any number of other content providers. Because of contemporary legislative developments in Freedom of Information and Open Government, not to mention greater scrutiny of government institutions by investigative journalism, much more of the information we hold will be publicly available well before the statutory periods that control the release of archival records.
In this environment it simply won't make sense for one part of the government to withhold information under a 20-year rule, while another part of government has already released the information into the public domain, either through a proactive information publishing scheme or by an FOI request. All parts of government will need the same view of the information and handle it accordingly. Again, we need to organise ourselves around the way the citizen works, not the way we've organised government departments.
In this era of a networked, joined-up government I think it's also time to revisit the most basic principles of the archivist's trade: provenance and original order. I think these fundamentals will endure the digital age; however, our practical application of them has grown up in the paper domain, and could never have contemplated the possibilities of digital technology.
It's a world of contestable information and competitive information services, and this world is inhabited by people who consider themselves to be 'time poor', lacking perhaps the motivation to go that extra step to corroborate their findings or understand the historical or business context of events.
And when there is a desire by a researcher to examine context and original order, it is quite possible that they will rely less on the arrangement and description done by the archivist and examine instead the source metadata within the data. A digital record will have many simultaneous provenances. For example, is a record created by an established department of state, a taskforce, a policy hub, an inquiry or a particular official? This depends on what the researcher is looking for, and in the present paper world we would have reference staff and finding aids that direct the researcher to various series with their own distinct provenance and order, from which the researcher would select then re-arrange the required records around the particular topic.
In the digital world, with the benefit of rich metadata and data mining tools, the researcher can specify the provenance as an agency, an issue, an inquiry or any other concept, while the international date-time stamp on each item will provide the original order. In short, the arrangement and description of records may be moved to the other end of the archival process, done at the time the researcher ponders the issue, and consequently the result will draw on the full resources available at the time of the research, not the more constrained view that may have prevailed at the time the records were first deposited in the archives.
In this information marketplace, our brand, our differentiator, is everything – we may not be the first port of call, but we will be the last port of call when the authenticity of the fact is more important than the convenience of the search; and when the evidence of an act or decision is more important than simply the understanding or knowledge of it.
It will be important, therefore, to make our archival resources discoverable in as many portals and search results as possible, but the next step of opening the details should, in my view, bring them in to an online archive, using rich data-visualisation techniques to encourage a broader view of the information and the context in which it was created.
Productive partnerships with like-minded organisations from the public and private sectors will be a key to our success. An example I'm showing here is the Trove portal provided by the National Library of Australia. There's a lot going on in the content delivery industry, and we need to be riding the wave, not swimming against it.
We will be embedded in business systems to ensure complete, authentic records are properly managed from the time they are made.
We will have legislative and regulatory frameworks that differentiate between a record's 'essential performance' and the medium upon which it is held. Those frameworks will also recognise that an organisation's records exist across many data storage platforms, potentially across many legal jurisdictions.
Our relationship with originating agencies will be changed – business rules embedded in agency systems will allow the maintenance of records from the time they're created, without constraining the ongoing administrative use by the owning agency.
We will exist alongside, and intermingled with, many other information portals offering much of the same information that we hold, sometimes in formal partnerships. Uniquely, however, we will be a rich content provider in a global information marketplace, ensuring that the 'whole story' is remembered and can be retold.
And of course access will be online and everywhere, improved by rich new data visualisation techniques and expanded descriptive contributions from an engaged citizenry.
But I am equally confident that some things will not change: in a world full of information sources, archives will be consulted for the authentic, complete record – that can truly be regarded and relied upon as evidence.
And the 'true record' will preserve the cultural heritage of the nation, support the rights and entitlements of our citizens and provide accountability and transparency of government.