[table] [attr style=”width:90px”],”People ask me what the largest file I’ve ever worked with in the MarcEdit has been.  In 2016 – it was 32 GB.  Overall, 1.7 TB using the Preview paging mode.  So, the moral of the story is, if you have a large file, don’t worry – MarcEdit can handle it.” [/table]

To enable the program to handle such large file sizes, some though has gone into how MarcEdit loads files.  The reason is that you simply cannot load a 1 GB file into a text editor.  On Windows, for example, it costs the system 4 times the size of the file to render the data in the graphical environment.  So, if I have a 32 MB file – loading that file completely into memory would cost me approximately 120 MB of memory.  Not bad – but let’s say your file is 200 MB.  Well, now you are looking at 800 MB of memory, and that is just to render the data.  There is a significant cost as well any time an action happens to the data that has been loaded.  So, while a 64 bit version of Windows, may in theory, be able to address up a gigabytes worth of virtual data – the practical limits make the actual data that can be managed in this fashion much smaller.  And, for an application like MarcEdit, this is a problem.

To allow MarcEdit to support files of any size, the application provides two paging loading modes: Preview Mode and Paging Mode.

Preview Mode

The preview mode is optimized for working with very, very large files.  This mode essentially by-passes the need to load the file, instead, loading just a snippet of the file into the MarcEditor window.  When in preview mode, the MarcEditor is set to read-only, but all the global editing tools are enabled, and work over the entire file.

Paging Mode

The paging mode is designed for files that are ~500 MB and smaller.  When using the paging mode, MarcEdit will load a user defined number of records on a Page.  Users can then jump between pages or records using the controls in the footer of the MarcEditor or tools found in the Edit menu.  Within paging mode, users can edit the records on the page, as well as add new record or delete records from the page, as well as utilizing the global editing functions – which perform edits across all pages.

Together, these two modes offer MarcEdit users a range of options for working with files.  By default, the preview mode is disabled, and the paging mode is enabled.

Logging Data

One of the new features implemented within the MarcEdit 6.2 series is the ability to create change logs when working with data in the MarcEditor.  By default, this option is disabled, as this function does minimally impact the speed of a global editing function, but has quickly become a popular feature of the application.  Using a log manager, users can view all the actions that have occurred on a MARC file during an editing session, when run through a task – and then use the log to extract only the records that were changed as part of the editing process.  For more information about MarcEdit’s logging function, please see the Chapter 6: Logging Changes.

AutoSave Options

So, let’s face it, sometimes things happen.  As a user, you are working with the application, and maybe you stumble upon an error and the program crashes, or maybe your computer reboots – or freezes, or whatever.  Suddenly – you’ve lost work…it’s a pain.  To help protect users, MarcEdit includes an autosave mode.  When working within the MarcEditor, MarcEdit will automatically save the working file every 5 minutes.  This way, if a crash does happen, MarcEdit will allow you to automatically recover your autosaved files and retrieve a portion of your edits.