Challenges for Web Archiving

The keynote speaker this morning is Malcolm Gillies, the DVC (Education) at Australian National University. He provides a brief history of cultural transmission, from remembered oral tradition to the emergence of the written word (which suffered its first tragedy with the demise of the library at Alexandria). Massive duplication through printing made text less vulnerable to loss, and gave information a tangible form. Now, however, digitisation has made information intangible again, as well as flexible and ephemeral.

Forms of communication have multiplied and diversified with the new electronic and digital networks, and only a few of these are being covered by archives and libraries so far. This may constitute an 'archival dark age', Malcolm suggests. (This, indeed, already started with earlier electronic forms - the thermal fax, early digital music recordings, etc.). The Web only intensifies this problem - and there is also a problem of overlap between it and other forms of communicaton.

Malcolm suggests that this century is the century of design, and the Web the fashion magazine of our times - it's the primary means of cultural transfer, scientific exchange, government interaction, etc., but especially also a tool for shared interest groups and reinforced niche-based cultural exchange and the interaction between the niche groups. Why, what, and how should we archive this medium?

Our daily loss of content is now huge - significant amounts of information now exist only in electronic form, and to lose them would be a tragedy (this was pointed out in the launch of the NZ digital repository). It is also important not only to archive cultural resources, but also important governmental and other official documents, for legal and ethical reasons. and further, taking Web snapshots (a la Internet Archive) regularly will provide an important history of human endeavour much as the archiving of newspapers and other everyday publications does.

But what should be archived? National libraries have generally argued from positions of 'national significance', and perhaps focussed on specific forms - but today the lines are blurring. Multimedia capabilities enable the combination of very different forms, and significance is not necessarily clearly establishable at all points. It is also now possible not only to archive the outcomes (e.g. of scientific work), but the data and other incidental materials as well. And additionally, of course, national collection is made problematic by the largely non-national nature of Internet content.

How can archiving be made to work, then? There is a long list of potential problems here. The major issue is one of auspices: there is a compelling need for strategic alliances between both like and unlike institutions. Archiving of the Web may be the largest archiving initiative of all time, and calls for large-scale cooperation across state and national boundaries. It is through international auspices and strong national leadership that the Web can best be archived; it is also important however, to keep track of how the archive may be used, and by whom. The rise of the creative industries is leading to the emergence of a new and highly skilled digital workforce which will both drive the expert and experimental use of the Web and its archiving and preservation, Malcolm suggests.

Further key problems then are: who archives the archives? Who ensures that the archives remain in existence and accessible? How can materials be kept accessible through technological changes, and kept safe from physical damage or political interference? This again points to the importance of multilateral cooperation in archiving, then.

It is also important not to select too sharply only what may be considered 'important' resources: this may lead to the freeze-drying of archived material - an exercise in nostalgia rather than a continuing development of content (music's increasing focus on archival recordings - with Elvis still a bestselling artist - is a worrying trend here). By archiving content, we may be placing it in shackles; in capturing an ephemeral artefact, it is by necessity changed.

Legal problems also abound - largely also because of the varying legal frameworks on a global scale. As we archive the Web, do we apply moral or commercial censorship, or do we capture what the Web is really like - spam, porn, copyright piracy and all? Current copyright extension trends lead to a privatisation of content and a bypassing of libraries; however, countertrends also exist in this field.

It's also important to remember the fact that the 'world-wide' Web is highly unevenly distributed, of course. Therefore, archiving alliances and cooperatives need to reflect this diverse environment, and be sensitive to differing regional requirements and cultural contexts. Chiefly, this also requires awareness by the various national governments which are driving archiving initiatives.

Governments still have some way to go towards realising the national and economic importance of library and archiving services. There may be a need to institute a Chief Information Officer in analogy to the Chief Scientist or other positions, and it is important also to recognise the importance of archives to those citizens who are not yet online. Overall, then, the challenges are many, but the step-by-step road is now mapped out, and its steps are achievable. It remains important to ensure that information is free and remains so.