Scanning microfilm back to the early 1900s to preserve paper

Every newspaper has faced the ever-demanding challenge of how to preserve back issues.

From microfilm storage to copying, binding and housing, in recent months newspapers have been seeking new ways to maintain their old news. Today, that challenge has been made easier with the latest electronic imaging and new storage technologies in an effort to digitize useful clippings.

The labor-intensive task of newspaper indexing is reduced with this recent introduction, converting text into an electronic form. Larger newspapers are more able to afford this transition while smaller companies realize the cost considerations, preventing them from seeing this as a readily available option.

"We are interested in introducing a system like that here but we have not gone through the experience of buying it yet," said Kathy Foley, assistant managing editor at Texas's San Antonio News-Express. "I have seen the technology at conferences and we have even put some money into doing a pilot program. We want to find the easiest way to go from microfilm to digitizing with as little keying-in of the information as possible. Not everyone can afford programs like what the Chicago Tribune has done."

The Tribune has embarked on a major collaborative effort to install a digital database that will contain all stories that have been published since Jan. 1, 1900 (yep, 100 years ago). Some material that will not be contained in the database includes horoscopes, most financial and sports tables and some advertising.

"This has been a continuous three-year project for us at the Chicago Tribune," said John Jansson, the paper's editor for information systems. "We began the project in June 1999 and will be finished in June 2002. By the end of the project, we will have digitized about 15 million news clippings that date back into the early 20th century from 1900 to 1984. Users will be able to conduct a full text search and view the images of specific clippings."

The Tribune had been discussing several options before it implemented the program last year.

"We had been looking at some options of preserving clips since 1983 or 1984," Jansson added. "We got really involved in the project once we had seen how successful the Baltimore Sun was in their digitization program back in 1997. We put together an in-house study group to see the specific aspects of looking at this as an option and put together a pilot program at that time. It seemed like something that we should do and we went forward with the project."

Obtaining good quality scans from microfilm was a concern.

"We are taking images from the microfilm exactly as it was when the paper ran, say, in 1949. But the [scan] is coming from the microfilm, so the quality sometimes isn't quite as good as an original copy," Jansson said. "We are having some of our copy re-keyed," he said, so that the database will contain the cleanest possible copy.

Jansson said that the paper started out by trying to find a way to preserve some of its clips that had been deteriorating over the years with age -- they were literally disintegrating.

"We started out thinking that we wanted to preserve the information on the clippings for our reporters but then realized there was a market for this information," Jansson said.

The paper is planning to market the information in two ways: through a third-party reseller, Newsbank, who will sell subscriptions to public libraries, and also on the Web. Users who want to check a family tree or see the front page of the paper on the day they were born will soon be able to access all this information.

The public library model will be looking at an annual subscription fee and the Tribune may charge Web users a per-image fee or a fee based on their usage. "We haven't quite worked out those details yet," said Jansson.

Scanning microfilm
Before the mid-1800s, newspapers were published on paper made of cotton rag fiber. Many of these publications have survived over time but the cost of production at that time was very costly. By the 1800s, publishers were looking for cheaper ways to produce paper information. Newspapers were being published on more expensive rag paper but now included substances to prevent discoloration and deterioration.

Although this provided for a better solution for publishers, they also faced the challenge of print being published on a paper that would eventually fade over time. Newspapers today are now over the newsprint's susceptibility to damage by heat, light or dampness.

Today, scanning microfilm into a digital format attempts to ensure that all the information will be preserved, while enhancing access capabilities. One of the major concerns raised in providing digital solutions is the ability to thoroughly transfer all of the data. Some transformations have been difficult because they do not include photographs, advertising sections or legal and death notices.

"We introduced this program in-house about three years ago," said Sandra Levy, director of library and information services at Baltimore's Sun. "We digitized most of the byline clips, but then we ran out of money. We are not planning a complete conversion right now. We had one company scanning the clips, while another was inputting 760,000 images."

Levy said that it was a "difficult task" when the scanning was being done, but now it is an "excellent system, allowing us to do a full text search. It has been a big help to us."

The London Evening News was the first newspaper to be microfilmed in 1853. By the 1930s, newspapers were realizing the need to better preserve their information. Boxing newspapers -- encasing the newsprint in boxes to shield them from light and humidity -- provided support and protected the publications from light while providing easy accessibility, but space constraints remained an issue. Clipping files presented a preservation challenge and more and more newspapers began to look at microfilm as their immediate solution.

In the 1980s, the Library of Congress and the National Endowment for the Humanities embarked on a campaign to make newspapers throughout the country aware of the need to convert their newsprint archives to microfilm.

Companies like MediaStream Inc. provide consumer access to newspaper archives and tools to refine a specific search.

"We offer a service called News Library, which is a web-based service that posts archives of 90 different newspapers," said Joseph DiMarino, president of Philadelphia-based MediaStream.

"We are able to make available archives through our web site and also from the newspaper's web site," said DiMarino. "If a newspaper has a Save system [the in-house archive MediaStream sells to publishers], we do not have to convert the data and there is no cost to the newspaper for this service. Right now, we have the ability to archive articles on our site through a Windows-based program and any sales from our site are split 60-40 with the newspaper with us receiving the higher percentage."

DiMarino said that if a paper does not have the Save system, MediaStream charges it to convert the data. As a division of San Jose's Knight Ridder, MediaStream customers include all Knight Ridder publications, as well as papers such as the Washington Post and Boston Globe.

Take back the files
Some companies like Southwest Micropublishing Inc. (SMI) of Texas are looking to in-house solutions.

"We currently provide microfilm services for newspapers," said James Park, president of SMI. "We are getting into the digitizing end of archives. We are working on introducing this by the first of the year. Right now we are hoping to take back files and scan them to create a searchable database with full text capabilities. A user can receive this information on CD, through the Internet on a pay-per-view basis or the information can be accessed on our server or the newspaper's."

Digitizing the issues has also been helpful to corporations like the Gale Group, a division of Thomson Corp. that offers publishing assistance for libraries, schools and businesses. The company creates and maintains more than 600 print, on-line and microform databases.

"We have been working with the Times of London by digitizing the Times supplement for the past year and one-half and have now just started to begin digitally producing the entire newspaper from 1795 to 1985," said Mary Sauer-Games, program director for the Gale Group, which is based in Farmington Hills, Mich. "We take all the images from microfilm and convert it to digital format by cleaning the images and electronically clipping the articles. The final product can provide a search of individual articles. We are also OCRing [Optical Character Recognition scanning] text and incorporating fuzzy logic searches for period products for the library market."

Sauer-Games said the end result will allow the user to see the specific terms highlighted in an article and can place context around that article and "even see what ads appeared around that article on a certain date."

Libraries have seen numerous changes in the digital market over the past few years.

"There has been generally a lot of interest in the marketplace in having material digitized," Sauer-Games added. "Libraries are seeing a tremendous value in historical newspaper preservation. Digitizing back issues provides the user with different options instead of microfilming, where you have to know how to index, etc."

Through micropublishing, companies like Bell & Howell are able to offer content management solutions by taking customer data in multiple formats, producing a paper, microfiche, CD-ROM or Internet end product.

"We have not yet released our product for newspaper libraries but we are planning to release it by first quarter 2001," said Tina Creuger, an executive with Skokie, Ill.-based Bell & Howell Co. "We currently have over 4000 full-text periodicals that we digitize from general reference to medical journals and other sci-tech publications. Typically, academic and public libraries subscribe to our database."

With its on-line research databases, digital-imaging solutions, Internet billing applications and electronic parts catalogs, Bell & Howell offers direct solutions for media outlets.

"When Bell & Howell started out, we started microfilming information that was endangered and we wanted to preserve that information," Creuger said. "We got into microfilming newspapers and periodicals and then started providing information on CD-ROMs and on-line to academic and public libraries. Next year, we will be introducing a new software to the newspaper industry, which will be very helpful to them."

Digitization may soon become the wave of the future for newspapers.

"The justification in larger newspapers is that they think digital access is a good revenue source and that is something they can count on," said Nora Paul of the New Media Institute at the University of Minnesota.

"It is a fabulous reporting source," said Paul, who spent 10 years at Florida's Poynter Institute and previously had been a newspaper librarian at the Miami Herald. "A user can easily find relevant back issues, which is a very powerful thing. It ensures that readers can get the most information available to them. There are very few places which can justify the cost of this transition but others are looking at measured ways of doing this by getting the information onto digital format."

-- Kellie K. Speed, kkspeed@colepapers.net

Bell & Howell Co.,
(847) 470-7100;
Gale Group,
(248) 699-4253,
e-mail: galeord@galegroup.com;
MediaStream Inc.,
(215) 239-4100,
e-mail: membership@krmediastream.com;
Southwest Micropublishing Inc.,
(800) 367-1273,
e-mail: smi@whc.net.

From THE COLE PAPERS, December 2000, Copyright © 2000, All Rights Reserved.