Bibliotheque nationale de France

Story Details

Bibliotheque nationale de France (BnF) resulted from the merger of Bibliotheque Nationale and the public Bibliotheque de France in 1994. The library has the job of preserving France's written heritage and making it available to members of the public from any one of six sites across the country and on the Internet. The amount of information that BnF has to preserve and make available goes way beyond the kind of libraries that most people are used to. Apart from 13 million of books and newspapers, BnF also stores 250,000 volumes of manuscripts, 350,000 periodical collections, about 12 million posters, more than 800,000 maps, 2 million musical compositions, 1 million audio documents, and tens of thousands of videos, print documents, photographs, and multimedia items, as well as 530,000 coins and medallions, among other items. Added to all this, there is the BN-Opale Plus catalog, which has more than 10 million bibliographic references.

The 1990s highlighted the challenge of continuing to preserve printed material when the quantities were growing exponentially. Managers saw that the only way to cope with the increases was copy the content onto computer disks and launched a series of digitization projects that also included archiving the French Web sites. Adote Chilloh, assistant director, Information Systems Department, Systems and Networks Division, summarizes why digitization was the perfect answer: "The BnF's mission is not merely to house and catalog documents — we also want to be involved in preservation. This is not just a matter of archiving in the traditional sense (for about 100 years). All documents must be 'readable' forever."

In 2004, BnF looked at how much content was being digitized and whether it was possible to make the process any faster. Findings quickly showed that the amount of stored data would exceed 1 petabyte by 2012 and that demand for digitized content by the French public was outstripping supply. To meet these needs, the existing storage infrastructure and method of digitizing material had to be replaced. Consequently, the institution launched an internal study to create a whole new system.

The Information Systems Department at BnF decided to use the Open Archival Information System (OAIS) reference model as the basis for developing a Distributed Archiving and Preservation System (S.P.A.R-Système de Préservation et d'Archivage Réparti), with tape technology for main storage and the backup system, and disk technology for accessing. "We decided that tape technologies were still the most competitive for the expected volumes," said Mr. Chilloh. All aspects of preservation had to be integrated in parallel, using special software to verify the data format before archiving and to manage the metadata, legal rights, and tape life cycle.

BnF didn't plan on making one petabyte of storage available from the beginning. The idea was to scale the architecture over time as the need for more data storage arose and so maximise cost effectiveness. The solution also had to be flexible enough over its life cycle to incorporate any advances in technology, and BnF required any proposed vendor solution to go through a test phase before receiving the green light. "Our final decision would take account of the technical aspects of the bid, the test results, and also the costs," comments Mr. Chilloh.

The Sun proposal was judged best in terms of economics, functionality for hardware and software architecture, and startup services. To ensure the longevity of the system, BnF also wanted the tape management software to include APIs and programming interfaces, without depending on encryption functions or irreversible compression. The Sun-based architecture that the institution deployed consists of the following technology and services:

• Sun Fire V890, V490, and V240 servers that include processing units and storage management units that run on the Solaris 10 Operating System, using Solaris containers
• Sun StorageTek SL8500 Modular Library Systems for each site, comprising 6500 slots equipped with T9840C high-performance tape drives
• Sun StorageTek T10000 tape drives providing a nominal rate of 100 megabyte per second, and an uncompressed media storage capacity of 40 gigabyte for the main site and 500 gigabyte for the backup site
• Sun StorageTek FlexLine 240 and 380 storage systems with storage area network (SAN) and network attached storage (NAS) access for database storage, along with workspaces and reference files
• Sun Services for advance installation, a three-year support agreement, and onsite and instructor-led product training

The software portion of the new architecture is based on Infotel's Arcsys Manager, an archiving and storage management engine that provides an interface with BnF's specialised applications. Based on a multisite architecture and synchronous or asynchronous data replication, the system ensures data security via a backup site equipped with the same equipment as the main site, which is the base for BnF's disaster recovery plan.

The new architecture makes digitizing content incredibly simple — and rapid. Documents are digitized in a high-quality mode by a service provider, then loaded onto disks and inspected visually. The documents are then placed on tapes in Tiff format, while compressed copies (for example, in .jpg) are stored on disks to be accessed via Gallica (noncopyrighted documents) or at BnF (documents that can be accessed onsite). "This process now makes it possible to digitize, preserve, and make available to the public about 100,000 documents per year, versus 6000 prior to 2005," says Mr. Chilloh.

“The actual performance meets our expectations, even though utilization is minimal at present. The system operates 24 hours a day, 7 days a week, and moreover is accessible to the public, so we have to provide the highest-quality service. Beginning in 2008, we will reach optimum utilization, and we will be better placed to evaluate the performance of the Sun system," concludes Mr. Chilloh.