As regular readers know, I used to work at the National Archives. Since starting this blog, I’ve avoided writing much about NARA because I worried that if was too critical people would think I was trying to get back at someone for something, or that if was too supportive people would think I was just a shill for my old employer. But I read a post a few days ago called “The NARA/TGN contract as a bad precedent” on a blog I admire, Free Government Information, and I felt I needed to write a response.
The authors at FGI are advocates for, clearly, free government information. As an archivist and a former employee of NARA, I am an advocate for the broadest possible public access to NARA’s holdings, as well as for the general welfare of NARA as an institution. I am also keenly aware of the challenges NARA’s mission presents and the limited resources it is being given to carry out its mission. I am also a pragmatist. I am not, however, an expert in digitization. If you’re looking for a technical discussion, you won’t find it here.
Many of you may remember an excellent article published last March in the New York Times, “History, Digitized (and Abridged)“which described the challenges NARA and other repositories face in digitizing their enormous collections. The article estimated that, given the expected annual rate of digitization, it would take 1800 years to digitize all of NARA’s textual holdings and 576 years to digitize all its non-textual holdings. Clearly they have to seek ways to speed up their digitization process. Without a large increase in their budget or a drastic shift in their institutional priorities, they cannot digitize their materials any faster. To make more materials available online more quickly NARA, like many other national archives across the world, has chosen to pursue corporate partnerships. In such partnerships, the corporate partner has to get enough out of the deal to make it worth their while. This leads to agreements, such as the one with The Generations Network (TGN) (available here) that limit free public access to the digitized materials for a given period–in this case five years. It’s a trade off, and it’s one I can live with. I think most archivists, based on their own experience with the challenges of finding money for digitization, would agree.
I will not address all of the concerns raised in the FGI post, but think I will cover most of the substantial ones. Now, let’s get to the specifics of this agreement and FGI’s concerns with them, and I’ll share what I think people should really be worried about.
TGN will be creating digital copies from either original archival materials or from microfilm copies of materials, if available. The most confusing parts of the agreement concern the distribution of the digital copies made from microfilm publications. I’ll discuss those after we look at the basics of the deal. What NARA gets out of this are:
- Digital copies of high-use, high-demand archival materials
- After five years, NARA can use these copies without any limitations (except for some restrictions on those created from microfilm–see below). This will mean that users can be required to use the digital copies (similar to requirements when materials have been microfilmed), decreasing the preservation risks to the materials. Note that the terms of the agreement give NARA, after five years, “full and unrestricted rights to use [the digitized materials], including the right to sell, make available for downloading, or otherwise provide in electronic form, the entire contents of the Digitized Materials or segments of them.” FGI latched on to the “sell” in this statement, but the terms don’t prohibit NARA from making the images available for free (except for the ones produced from microfilm).
- Detailed, often item-level metadata and indexing that would otherwise never be produced and which will be freely available to the public (and searchable by Google, etc) on both NARA’s website and TGN’s
- This metadata will enable all researchers to identify (at no cost) materials that they never would have known existed. This appears to me to be a significant public service. Yes, if people want access to the digitized copy, they will have to pay a fee, but without this service they might not have ever known about the document or they might have had to go through many many pages of irrelevant documents (in person or by paying for copies) in order to find it.
- Conservation and preservation activities for materials to be digitized will be paid for by TGN
- This could be a significant cost savings for NARA. Again, the materials that will be selected for digitization will probably be high-demand and therefore high-use materials, and so in need of treatment.
- Free access to the digitized materials in all NARA reading rooms and for all NARA employees
- Is this ideal? No, it would be better if everyone could have free access from everywhere. But TGN is in business to make money and so I respect their right to create conditions that allow them to recoup their investment. Getting access in the reading rooms is helpful to those who can travel to them (and yes, people will be able to hit the print button on the computer in the research room at no charge, as long as they’re not abusing the privilege). Free access for all NARA employees will allow them to use the resources in responding to reference requests and I would guess that this would improve speed and quality of those responses. In FGI’s comments, I think they were not taking into consideration that there is a distinction between access (which will be free in the reading rooms) and reproduction (which, by pre-existing NARA policy, is almost never free).
- Potentially, increased revenue for producing reproductions
- NARA collects fees for creating reproductions of its materials–whether they are photocopies, digital copies, or microfilm. Under the terms of the agreement with TGN, NARA will charge the appropriate fee for a reproduction of one of the new digital images. Again, TGN doesn’t want NARA giving away what it (TGN) is charging for. It seems reasonable to me. NARA will charge its standard fee, and so it seems likely that this will result in increased revenue for NARA. I am sure this is not a motivating factor for NARA to enter into this agreement, but I think it might be a nice side affect.
- TGN will clearly identify all NARA materials on its site as being from NARA and will link back to the appropriate descriptions in NARA’s online catalog.
- FGI saw the (restricted and limited) ability for TGN to use NARA’s trademarks as a concern. I understand their point, but all uses must be pre-approved and I know NARA will be very careful about this. I see it more as an opportunity to expand public knowledge about NARA’s holdings and bring more people onto the NARA site. Yes, NARA’s online catalog will link out to the TGN site, but again, that doesn’t bother me so much. Maybe I’m just too jaded, but sites link out to each other all time. This is giving users the option to get more information and I think on the whole it’s a net gain for our user community.
Now, about the microfilm issue. I found this very confusing, so I looked into it as best I could, and honestly, I still don’t think I get it. I believe there is general agreement, including within NARA, that this section of the agreement is confusing and the issues need to be explained and clarified. The first issue seems to revolve around giving an unfair competitive advantage. As most of you probably know, NARA has microfilmed over 3,000 series of Federal records and makes these microfilm publications available for purchase. Anyone who purchases microfilm may do anything they want with it–including digitizing it and selling the product. Therefore creating digital copies of the microfilm is a competitive area–many people can do it. This is not true of digitizing archival materials (that is, series of archival materials). Once a group of archival materials has been digitized by NARA or a partner (such as TGN), it will not be digitized again. Therefore, this area is not competitive. If I am following the logic correctly, the rationale for continuing to charge (after the first five years) for digital copies made from microfilm publications is that providing such copies for free would give TGN’s competitors an unfair advantage in a competitive area (since they would not incur the costs of digitization). NARA is not willing to apply this same rationale to the digitized images made from archival materials because no other vendor will ever be given the ability to create their own digital copies, and so the “playing field” is not fair. I believe this to be the logic. I am not sure I agree, but I believe this is the basis for the additional restrictions on these digital copies.
In addition, I believe the agreement does not make clear whether the requirement to continue to charge for reproductions of digitized microfilm materials applies to microfilm publications (meaning the a reproduction of the entire publications) or to any subset of the publication (such as a few pages). Based on my research, I think the intent is that these fees would only be charged for a reproduction of the entire microfilm publication, not for reproductions of small parts of a publication. But, again, this is not clear. If NARA’s intent is what I have described, the agreement needs to make this clear.
So, let us assume that five years have passed (and NARA has resolved the sticky wickets about microfilm). Under the terms the agreement, NARA now has the right to post, without restrictions, on its website all the digital images and metadata that TGN has created and transferred to them. People could download all of it. For free. We would now have the free public access that the FGI blogger wants, and that we all want.
But, here is what I think you should worry about–will NARA be able to do that? The answer is no. Given NARA’s IT infrastructure and (to the best of my knowledge) their plans to supplement that in the future, they will not be able to host anywhere near all the images they will be receiving back from their partners. We the citizens, through NARA, are giving TGN the right to exclude our free access to government records on the web in exchange for TGN’s services in digitizing and describing them. We are making this exchange with the understanding that in five years we will have free public access to all the materials. But we will not have that access unless NARA makes them available to us. I am sure that NARA will make some of these digitized records available, as many as their infrastructure will allow. But it won’t be anything near the total amount digitized. And for the materials that NARA can’t make available, yes, you will still have to pay to access them through the partner’s site, even after the five years is up. That’s what concerns me.
What can be done? Well, if you dislike these partnership agreements, you could encourage NARA to be more aggressive in seeking out donors who will fund digitization (and the infrastructure needed to support access) with no strings. The Library of Congress seems to have been pretty successful with this. NARA has one such arrangement, I believe, with the EMC Corporation to digitize materials at the Kennedy Library. Other than this, I think most of their large corporate contributions have gone to the remodeling of the downtown building, the National Archives Experience and other educational resources.
Given that we probably have to live with these partnership agreements, you could ask some serious questions about NARA’s investments in its IT infrastructure and what those investments will do to support access to these digitized records. If making these records available to the public as soon as legally possible is not a NARA priority, you could start asking some questions about those priorities. If you want free access in five years, you’d better get started now.
That’s my perspective. I’d like to hear yours.