Streamlining Delivery of Online Oral History Metadata through LibGuides

Background

Founded in 1968, the University of Louisville Oral History Center (OHC) houses over 2000 interviews of people from politicians to everyday citizens. Collectively, the oral histories represent an incredibly rich source of historical information. The challenge with a collection of this type and scope is making that information accessible to the people who might use it. Some of the material has been transcribed; some hasn’t. Some of the interviews have been digitized; others are still on cassette tapes. Having full-text or full-sound of the entire collection online is just not possible at this time; the work of transcribing or digitizing the materials would take an enormous amount of labor. Creating hierarchical finding aids would not accommodate the item-level description necessary for meaningful access to the oral histories. So we looked for ways to make information about the collection available: who was interviewed, who conducted the interview, when, what topics were covered, etc. Over the years access to this metadata evolved from typed lists available at the reference desk to records in the library’s catalog.

In the past ten years we’ve made progress in moving this data to digitally accessible formats. In 2007 Caroline Daniels, then co-director of the Oral History Center, created an HTML table to display item level records on the library website. She exported the data via a query of the Access database she used to keep track of the collection. A number of issues factored into the decision to create an HTML document rather than machine-readable cataloging (MARC) records. Creating individual MARC records for over 2000 interviews was not feasible due to the amount of labor it would require and the lack of staffing to get it done. Creating collection-level MARC records wouldn’t provide meaningful access to the interviews – also not every interview was part of a collection. The Co-director of the Oral History Center essentially creates item-level descriptions for our oral histories as part of the accessioning process. The OHC uses a controlled vocabulary for personal names and corporate entities and follows the archival descriptive practice of writing dates as YYYYMMDD when creating metadata, allowing for easy export and repurposing of the data. All potential donors are asked to complete a summary form post-interview that we use to populate the database, thus minimizing the time we consume constructing descriptions and maximizing the number of records we can put online. By making a webpage the metadata would be findable via search engines like Google. Additionally, adept computer users would be able to search the webpage using a simple CTRL-F to find information in fields such as interviewer, interviewee, date, collection, and description.

Over the years our website has transferred more and more of its content into the LibGuides content management system and in 2009, Daniels moved the oral history data into LibGuides: “with really minor tweaking I was able to migrate data from the existing database to LibGuides. I could just export a text file output from a query, paste in tags in an Excel spreadsheet, and cut and paste it into the LibGuides template.” However, LibGuides content boxes have a maximum character limit which added to the labor-intensive process as the archivist needed to create extra boxes, and cut and paste sections of the HTML code into individual boxes. While this system was feasible for several years, the time-consuming process for updating the information made updates difficult and they often fell to the bottom of the “to do” list. The guide went unchanged from 2013 through 2016.

In early 2016 the new Co-director of the OHC approached the web services librarian and digital technologies librarian about this project to investigate how we could streamline the updating process. The goal was to eliminate the need to handle the data multiple times. The ideal system would update the information in the LibGuides webpage whenever the database was updated.

Literature Review

In an American Archivist article from 2003, Ellen Swain traced the evolution of the relationship between archives and oral history noting debates within the profession about the value of oral history as “non-transactional” documentation and the role of archivists in producing oral histories. (Swain, 147) Swain considered oral histories as a valuable way to fill in the gaps of documentation in traditional archives and encouraged archivists to “familiarize and engage themselves in its practice.” She traces the emphasis on access to oral histories to Bruce Breummer’s 1991 article in which he lays out a national agenda that proposed creating MARC records for each interview and developing an interlibrary loan system for sharing oral histories. (153) (Swain, 158). “One of the key findings of a survey of collections undertaken by the American Folklore Society, the Society for Ethnomusicology, and the American Folklife Center in 2000 was that much of what has been recorded is poorly controlled.” In a footnote she reports that over 90 percent of the respondents did not provide access to their oral history collections via the Web.

In 2004 Nancy MacKay reported on her survey of access methods. Her survey revealed three approaches to describing oral histories: 1) creation of finding aids, 2) creation of MARC records, and 3) simply not cataloging them.(MacKay, 58) Following up on MacKay’s work, S. C. Wynne conducted her own survey in 2007. Wynne’s survey found few people used the Oral History Cataloging Manual, many of them opting instead to follow the common bibliographic standard AACR2 (Wynne, 566).

Looking for solutions to our particular problem in the literature yielded no articles on integrating an external database into LibGuides nor were there articles on exposing oral history holdings in this way; however, the existing literature did provide insight on how archivists have used LibGuides to improve access to special collections resources in general.

In 2011, Lewis and Griffin argued that LibGuides uniquely possesses the necessary features for creating a 2.0 subject guide. (Lewis and Griffin, 24)[1] They reported that when they began their own implementation of LibGuides as a means of providing access to special collections materials, very few archival institutions employed LibGuides for this purpose. (Lewis and Griffin, 25). Their implementation of LibGuides provides access to digitized content rather than to purely descriptive information. As part of their process, they transcribe and post the audio files of interviews to their LibGuides site. A web search for repositories using LibGuides [2] shows that most of them use LibGuides in a similar manner.

Developing the Process

As an existing solution was not found in the literature, we experimented with systems that would enhance access to the oral history metadata without duplicating effort behind the scenes. We wanted the researcher to be able to search the database or sort materials by certain fields, such as the date, interviewee, interviewer, and collection. The oral history records would need to be created in a local system that could be displayed directly on a LibGuides page. At the time of this project, however, LibGuides did not include the ability to upload data via its Application Program Interface (API). LibGuides primarily supported the use of the JavaScript programming language for outside developers. A work-around was needed. How could we get the data into the LibGuides page using JavaScript (JS)?

The solution included three parts:

1. A MySQL database to house the oral history metadata. This provided multiple data entry options such as direct database entry, the use of local online forms, or metadata pulled from online sources;
2. A PHP script to output formatted data with an added header allowing LibGuides access; and,
3. A JavaScript code added to LibGuides that would retrieve and present the data.

While LibGuides served as the target display site, the solution designed could be used with any system using JavaScript.

The original Microsoft Access database developed in 2007 was migrated to MySQL using Access to MySQL (http://www.bullzip.com/products/a2m/info.php), a free software developed by Bullzip (http://www.bullzip.com/about.php). Only minor changes, such as modifying headings and removing unnecessary tables, were needed to establish a convention and simplify the database. The resulting MySQL database was added to a local server for hosting as was the PHP script written by the digital technologies librarian. The script queries the database and processes the data resulting in a document in JavaScript Object Notation (JSON) format. LibGuides then accesses the JSON file using a jQuery script. [Appendix 1] DataTables (https://datatables.net/), an open source plug-in for jQuery, provided the structure for an interactive table display. [Appendix 2]

Styling the Oral History Display

The original display of the oral history metadata was designed as a table. While we considered using a list we decided to keep the table approach. One of the reasons was ease of layout. The LibGuides Content Management System (CMS) comes with a layer of Bootstrap 3 that creates a very clean table layout by simply adding class=”table table-striped” within the table tag. The table layout also provides the structure to allow columns to be sortable which can be very helpful for patrons looking for interviews with certain criteria. For example, it made it possible to order the oral histories by year, by transcript availability, or by collection.

In addition to the LibGuides style sheets, the jQuery we used to convert the data to page elements came with some basic style settings in its own style sheet. Unfortunately, the script also injected some settings directly into the code in the form of inline styles. Since inline styles override styles added via stylesheet or even styles written in the LibGuides Custom JS/CSS for the page, this created layout problems that needed to be addressed. For example, the script injected style code to set table cells to fixed widths. The script determined the largest amount of content in each column and used that to assign a fixed width to the column using an inline style. This made the Interview Number column unnecessarily wide, for example, while reducing the amount of space for the “Summary Information” column. With the narrower description column the content stretched out requiring excessive scrolling, and for printing, required many more pages of paper. By removing the part of the script that generated the inline styles we were able to adjust the column widths via CSS and improve both the screen display and print formatting. We also tweaked the style sheet so that it would print to landscape mode rather than the default portrait mode which allows the content to print more compactly.

Part of our workflow model for customizations in LibGuides is creating separate style sheets that can be uploaded through the Admin Look & Feel section. This section allows system administrators to upload customized JS, CSS, XML, and JSON. Using this approach provides a safeguard against accidental coding errors that could potentially break the page. For example, styling added in an individual page’s Custom JS/CSS could create a situation that inhibits the use of the page editing options on the back end. If the styling is in a separate stylesheet, the stylesheet could be taken down, corrected and replaced, making the content available again; whereas, if the erroneous code is in the page’s custom JS/CSS the page may become completely inaccessible and need to be rebuilt.

The style sheets used for this project included the base style sheet we use for the entire LibGuides site, a custom style sheet for the oral histories page, a style sheet from Data Tables, and a style sheet to animate an image that appears while the data loads. [Appendix 3]

In developing the page we noticed that some of the description sections were much longer than others creating an awkward display and requiring a lot of scrolling to get through the results. To remedy this the digital technologies incorporated jQuery Shorten (Patel, 2016), an MIT-licensed script by Viral Patel. For long descriptions, the script limits the summary to 200 characters and includes a “More” link that shows the rest of the summary when clicked.

Conclusion and Future Enhancements

In the end we created a system that 1) allows the archivist to update their metadata in a single database and have it automatically update the website, 2) creates an interactive table for researchers to search or sort by interviewee, interviewer, interview number, or collection, and 3) provides a way to incorporate external database information into the LibGuides framework. In the process, however, we lost something as well. While the content can be searched using the search box generated with the data and can be found by Google, it cannot be found via LibGuides’ search function. It was a trade we were willing to make to improve the researcher’s ability to manipulate the list and improve our archivist’s workflow for keeping the information current, but improving the findability of the data via LibGuides’ search is one of our next steps. Another future enhancement under exploration involves implementing a method to directly link to audio files and transcripts. The new system has been up for too short a time to glean meaningful information from the usage statistics or to get feedback from users, but this approach allowed us to post 1500 new records instantaneously when we went live, a three hundred percent increase in access from the previous configuration of the system.

Appendix 1: PHP

# Header allowing LibGuides access to local PHP output
$ header("Access-Control-Allow-Origin: http://library.louisville.edu");
[...]

# Create an array from the database query results
$ databaseArray[] = array('interview_number'=> $interview_number, 'interviewee'=> $interviewee, 'date'=> $date, 'interview_number_display'=> $interview_number_display, 'summary_information'=> $summary_information, 'transcript_available'=> $transcript_available, 'interviewer'=> $interviewer, 'length'=> $length, 'collection_name'=> $collection_name);
foreach($databaseArray as $element){
  $hash = $element['interview_number'];
  $uniqueArray[$hash] = $element;
}
$result = array_values($uniqueArray);
[...]

# Output JSON formatted data
header('Content-Type: application/json');
echo json_encode($result);

Appendix 2: LibGuides jQuery DataTables Configuration

$('#dataTable').DataTable({
  ajax: {
    url: 'http://webservices.library.louisville.edu/test/oral_history/history.php',
    dataSrc: ''
  },
  columns: [
    {"data": "interviewee"},
    {"data": "date"},
    {"data": "interview_number_display"},
    {"data": "summary_information"},
    {"data": "transcript_available"},
    {"data": "interviewer"},
    {"data": "length"},
    {"data": "collection_name"}
  ],
  "columnDefs": [
    { "orderable": false, "targets": 1 },
    { "orderable": false, "targets": 3 },
    { "orderable": false, "targets": 4 },
    { "orderable": false, "targets": 6 }
  ],
  language: {
    searchPlaceholder: "Search oral history descriptions"
  },
  "pageLength": 20,
  "lengthMenu": [ [10, 20, 50, 100, -1], [10, 20, 50, 100, "All"] ]
});

Appendix 3: Style Sheets

http://s3.amazonaws.com/libapps/sites/775/include/base_e.css

http://s3.amazonaws.com/libapps/sites/775/include/oral.css

http://s3.amazonaws.com/libapps/sites/775/include/jquery.dataTables.css

http://s3.amazonaws.com/libapps/sites/775/include/customloader.css

References

DataTables script. [Accessed Oct 18, 2016.] http://s3.amazonaws.com/libapps/sites/775/include/jquery.dataTables.js.

Lewis, Barbara, and Melanie Griffin. 2011. “Special Collections and the New Web: Using Libguides to Provide Meaningful Access.” Journal of Electronic Resources Librarianship 23 (1):20-29.

MacKay, Nancy. 2007. Curating Oral Histories: From Interview to Archive. Walnut Creek, CA: Left Coast Press.

Patel, Viral. jQuery Shorten script. [Accessed Oct 18, 2016.] https://github.com/viralpatel/jquery.shorten.

Swain, Ellen. 2003. “Oral History in the Archives: It’s Documentary Role in the Twenty-first Century.” American Archivist 66 (1):139-158.

Wynne, Susan C. 2009. “Cataloging Oral Histories: Creating MARC Records for Individual Oral History Interviews.” Cataloging & Classification Quarterly 47 (6):561-582.