In this Chapter:
- Getting Started
- Use Cases
- Batch Processing Data
Getting Started
Libraries, today, receive metadata from a number of different sources and in a multitude of formats. While many vendors still provide metadata as a single, large MARC download, other vendors may provide data in MARCXML, ONIX, or a myriad of other metadata formats. In these cases, libraries will likely have to transform these non-MARC metadata formats into MARC if they wish to load the data into their ILS systems. For the odd metadata file, pre-processing one file outside of the normal MARC workflow isn’t that big of a deal. However, if your organization receives 100, or 1000, or in the case of one particular vendor that I’ve come across, 17,000 individual metadata files — the ability to touch and process these records becomes excessive. This is what the batch processing tooling was designed to address — those messy cases where libraries have lots of individual files or metadata records that need to be processed before something else can be done to them.
Use Cases
As noted above, the batch processing tooling was designed to address a very specific set of use cases. While the questions and data formats of the requests very, the primary two use cases can be summed up below:
My vendor has provided me 17,000 individual MARCXML files. I need to turn these into MARC for processing and loading into my ILS system.
I’ve exported a set of records from my ILS and split them into separate files. Is there a way that I can turn these into MARCXML or Dublin Core records without touching each file.
Because MarcEdit provides tooling that allows users to join and split records once they are in the MARC format, the batch processing tool has been created to provide an automated method to perform large scale format transformations data in folders and at the folder/subfolder level.
Batch Processing Records
The batch processing tooling can be accessed from the Tool menu from the Main Application Window. This tool can also be added to the Main Application Window by selecting the tool on the Preferences window.
Figure 1: Batch Processing Dialog
Processing Records
When utilizing the Batch Process Tool, the application exposes all the available data transformation functions registered within the application. This means that all the data processing tools available within the MARC Tools window are available as part of the Batch Processing Tool.
The tool requires the user to provide a few pieces of information in order to perform the requested operations. This information is as follows:
Source Directory: The Source Directory represents the top level directory with the files that will be processed. For example, if a user had 3 records in the following directory structure:
The source directory would be the full path to folder 1. The same would be true if this example had included subfolders. For example:
- folder 1 - file 1 - file 2 - file 3