Migrating from Innovative Interfaces' Millennium to Koha: The NYU Health Sciences Libraries' experiences

by ByWater Staff on Feb 7, 2011

[Originally posted by Ian Walls]

Abstract

Purpose , This paper aims to explain the circumstances, methods and outcomes of the New York University Health Sciences Libraries’ (NYUHSL) migration from their previous Integrated Library System (ILS), Innovative Interfaces, Inc.’s Millennium, to the open-source ILS Koha.

Design/methodology/approach , Over the course of three months, the NYUHSL installed, tested, and configured Koha to match with its existing policies and procedures. Migration was performed at the close of the fiscal year, to reduce the amount of acquisitions data to migrate, as well as for contractual reasons. Training was purchased from a Koha support vendor, ByWater Solutions. After migrating, the staff of the NYUHSL identified areas of improvement for Koha, and began discussing how to implement enhancements.

Findings , The paper identifies several areas of development for Koha, including electronic resource management, course reserves, and cataloging client enhancements. It proves that a migration from Millennium to Koha can be done very quickly, if the library is properly motivated.

Originality/value , This paper provides libraries considering the move to open-source with a real-world success story, as well as many factors to consider. In particular, libraries considering the move from Millennium to Koha stand to benefit from the NYUHSL’s experiences.

Originally Published in:

Ian Walls, (2011) “Migrating from Innovative Interfaces’ Millennium to Koha: The NYU Health Sciences Libraries’ experiences”, OCLC Systems & Services, Vol. 27 Iss: 1, pp.51 – 56

DOI:10.1108/10650751111106564 (Permanent URL)

Introduction

The New York University Health Sciences Libraries (NYUHSL) had been using their previous Integrated Library System (ILS), Innovative Interfaces, Inc.’s Millennium, for over 15 years. During that time, staff members had come and gone, and institutional practices changed, including how some data was entered into the system. Additional modules, including Millennium’s ERM, had been purchased. Attempts were made to synchronize Millennium with other library systems, including the SFX Open Link Resolver and a homegrown Electronic Resources Database. Workflows were adapted to work within the limitations of the systems, and the library was able to provide core services with minimum difficulty.

On 6 October 2008, it was discovered that Innovative’s tech support was no longer able to connect to the local Millennium server. A recent change in network topology, required to comply with the New York University Langone Medical Center’s network security policies, had removed the secure connection method Innovative Interfaces had been using. All external parties were now required to use the Medical Center IT’s (MCIT) Virtual Private Network (VPN) solution to connect from off-site. Unfortunately, the particular VPN client chosen by MCIT was not supported by Innovative Interfaces, and a seven-month negotiation period began, during which time the library could not receive ILS service or support.

The library’s IT department, the Library Medical Informatics Group (LMIG), had long been creating and supporting in-house tools. The LMIG had released several of these tools as open source in previous years, and the library was in the process of migrating their websites to the open source content management system, Drupal. Moving to an open source ILS had been discussed for some time, and with the current connectivity crisis with their ILS vendor, migration seemed like a much more desirable option.

Any open source ILS platform chosen would need to have a robust community of users surrounding it, so the chances of the software becoming unsupported and stagnant were minimized. Additionally, the library required their new ILS to be able to perform all the essential functions that Millennium did, particularly acquisitions. Only two open source ILS’s met the first criteria, Koha and Evergreen. Of those two, only Koha had a functional acquisitions module at the time.

A test installation was performed in-house by the Systems Integration Librarian on a decommissioned desktop machine. The test system was put on the library’s network for anyone to experiment with. Since Koha is entirely web-based, it was easy for anyone interested in demoing the system to do so, without needing to install anything on their local machine. After gaining enough experience with the software to be able to present it to the senior management team, the systems integration librarian was given the go-ahead to install Koha on a production level server and begin migration preparations.

Implementation

One of the initial difficulties encountered by the systems integration librarian in preparation for the migration was mapping the bibliographic data from Millennium into Koha. Since Millennium had been in place for so many years, and had been maintained by a multitude of different staff members, the records had accumulated numerous errors and idiosyncrasies. Expired systems codes, typos, outdated MARC subfields, and stub records littered the system, necessitating a major data clean-up initiative by the bibliographic control department. This department, consisting of only two people, was able to eliminate most of the errors in the bibliographic records in a matter of four months.

In addition to dealing with errors in the records, there was also a need to remap some of the codes in Millennium to their best-fit Koha counterparts. This did not always occur in a one-to-one manner; for example, the location code in Millennium was used to capture data about branch, shelving location within the library, item type, and collection. In Koha each of these elements has its own subfield. Teasing out the key data and remapping it was done primarily by the systems integration librarian, under the guidance of the Koha implementation team.

Once the records were cleaned up and ready to be migrated, there came the difficulty of actually extracting them from Millennium. The library had purchased the data exchange module for Millennium, but this module had not been completely set up before the connectivity with Innovative Interfaces was lost. Exporting as a list was not feasible due to the complexity of completely capturing all the MARC fields in a comma-delimited format, and downloading MARC records from the OPAC omitted all item-level and administrative data. Using record numbers, however, it was possible to extract the complete records from the OPAC in XML. The Systems Integration Librarian was able to develop a script that would take in a series of record numbers, download the records, convert them to MARCXML using a custom XSLT, and then finally upload them into Koha. The script and sample XSLT’s have been released as open source, under the GNU Public License.

The migration of patron records posed a separate, but equally tricky, challenge. Millennium had accumulated numerous records for patrons who had since left the medical center, and it was decided this migration was an opportunity to do some cleaning. Only patron’s deemed “active‚Äù were migrated, where “active‚Äù meant either being created or having activity within the last two years, being in possession of library materials, or owing the library money. In an added layer of complexity, migrated accounts were bounced off an authoritative people database to determine their correct status with the medical center (i.e. active, graduated, terminated, etc.), as well as updated contact information. The script to govern this process was developed by the Associate Director for Systems.

Migration schedule

The connectivity issue with Innovative Interfaces was resolved, with Innovative adopting MCIT’s VPN for a monthly fee. As the end of contract with Innovative Interfaces drew nearer, however, it was decided that the library would not renew for another year; rather, the staff would push ahead with a migration at the turn of their fiscal year, at the beginning of September. In just three months, the library would need to finalize the data migration, make any customizations and configurations necessary, and train staff on how to use the new system. To add an additional wrinkle, the Systems Integration Librarian, the leader of the Koha implementation team, would be taking time off right before the migration to get married.

Three months prior to migration, a test system, on a production-grade server, with a first round of migrated data was made available to the staff. The various departments took advantage of this resource to learn the capabilities of the new system, compare it with the existing one, and anticipate and prepare for necessary changes. Practicing on the new system allowed staff to gain a level of proficiency, which eased the actual implementation. One library assistant had used the previous system for over 15 years. She tested Koha extensively to compare its capabilities with those of the existing system. Staff recorded a list of features and functions that seemed to be missing in Koha. Some of these functions were indeed in the system, but were done differently than in Millennium. Some changes were made in the system, prior to implementation. Other desired changes were set aside to be worked on locally or in conjunction with the Koha community.

Experimenting with the test system enabled departments to identify necessary changes in procedure, and prepare for their implementation to ease the transition. The process for printing spine labels was going to be significantly different, but would result in more legible labels, produced in less time. The old dot matrix printers that had been required for label printing in Millennium could be replaced with any networked printer, since labels were exported in a PDF from Koha. New label stock was chosen and the necessary templates were prepared and tested prior to the migration. Once cataloging commenced on Koha, the templates were constructed on the production system and staff was trained. Within a short time, the staff became proficient in the new procedures and materials were quickly processed.

Ten days before migration, a group of staff and librarians from the departments most affected by the change in system visited their sister library of NYU Polytech. This library has been using Koha since 2008, and was able to provide real-world experience and feedback on what living with Koha was like. They described how Koha is used in their library, explained some of the difficulties they encountered, and made some suggestions. One particular problem the Polytech staff mentioned was that the screens in Koha look the same in both public and staff mode. It is helpful to be able to identify the mode with a quick glance at the screen. As a result, staff mode in the health sciences libraries was given a blue background by specifying a different cascading style sheet in the systems preferences menu of Koha.

A week prior to migration, the staff received training provided from ByWater Solutions, a Koha support vendor. Trainers flew in to provide three days of training on the various modules of the system. Staff members were expected to attend the sessions that concerned their job functions. All were required to attend the training on using the OPAC. Unfortunately, it was not possible to have hands-on training sessions, though the instructors made much use of prepared visual material and projected actual use of the system. Staff who had used the test system were able to ask complex questions, which reflected their familiarity with Koha. For some of the staff this was their first contact with Koha and the change seemed overwhelming. Subsequent in-house training sessions, both group and individual, formal and informal, were conducted to help staff acclimatize. Documentation on local practices has been and will continue to be prepared by the library staff.

The most difficult element in training the staff was having them understand and accept that Koha is different from the previous system. It is a highly-integrated system, has new features that can be very helpful, and initially lacks features that are needed. NYU Health Sciences Libraries have the ability to tailor the system to meet their precise needs. Staff had to realize and accept that the new system should be approached and searched differently to take advantage of its strengths, and to use it effectively.

The migration itself took place on a Sunday, when the library usage was minimal. Circulation was put on manual operation, and the systems integration librarian pulled in the latest bibliographic, patron, and circulation information from Millennium in the span of several hours. The library went live with Koha as their primary production ILS on 1 September 2009, becoming the first documented library in the world to perform that particular migration.

Changes to workflow

Adjustments in workflow were needed in several areas of technical services, as Koha and Millennium handled certain core functions differently. Since orders must be linked to a bibliographic record upon creation, Acquisitions now creates a brief bibliographic record, containing just title and ISBN, for new non-electronic materials. Acquisitions also barcodes new materials prior to sending them to bibliographic control for cataloging, since creating an item record requires the presence of a barcode. The resulting changes to the acquisitions workflow began to blur the lines between various technical services departments, and the NYUHSL has considered redefining these departments to better align with the new workflows.

Unlike Millennium, Koha does not have an OCLC Connexion Gateway for uploading records from OCLC. Cataloging procedure has become a little slower, since records can no longer be directly exported from Connexion to the ILS. Catalog records can be added through a Z39.50 search or by downloading a batch of records from OCLC to a file and then staging and uploading them to Koha. This increase in time for cataloging is more than compensated by the significant reduction of time needed for materials processing, which resulted from the improved procedure for the production of spine labels, described earlier.

Koha gave the library a more powerful module for creating reports, using SQL rather than proprietary report-writers. Any information in the system could be captured and reported upon, given enough familiarity with the database structure to write the SQL query. Additionally, the number of reports that could be saved was unlimited; Millennium had allowed for only a limited number of reports to be saved at any given time, with each report having a fixed maximum number of results. The new module has no such size restrictions, removing the need to guess the approximate hit count of any given report before running it. This greatly reduces the overhead of managing canned reports, since space does not have be to cleared to run new ones.

Identifying areas for development

Out-of-the-box, Koha lacks several functionalities that the NYUHSL had made use of in Millennium, the most important of which was the electronic resources management (ERM) module. While it is possible to add links to the electronic versions of titles into the MARC records, and make those links accessible to patrons, there was no systematic way to insert and update the coverages available for a title from a package or database. Fortunately, the library has both the SFX OpenURL Resolver, as well as a homegrown electronic resource database, which combined provide accurate, searchable holdings information for the library’s electronic resources. The acquisitions portion of the ERM, which would keep track of payments and licensing restrictions, was also missing. The electronic resources staff have made do with other tools to track licensing. The NYUHSL has begun investigations into what developing such functionality would require, but are also considering third-party ERM tools.

Another missing function to Koha is course reserves. While items could be given specific shelving locations and item types to keep track of whether they are “on reserve‚Äù or not, the key problem is that Koha currently does all circulation in units of days, rather than hours. This makes circulating course reserves particularly difficult, as the established policy is for a two-hour loan period. Course reserves are being actively developed by several Koha support vendors around the world, and should be available within a year.

The current cataloging interface for Koha is considered to be rather awkward, especially for librarians used to dealing with OCLC’s Connexion. Some required fields, like the MARC leaders, will automatically calculate, but depending on which browser is being used, may need to be clicked on first. Further, MARC tags are grouped into different tabs, making it difficult to see all the applicable fields in a record at once. Either modifying the staff client to more closely emulate Connexion, or building an OCLC Gateway Client for Koha, would both make cataloging run more smoothly.

Conclusion

The transition of the New York University Health Sciences Libraries from Millennium to Koha was overall a positive experience, albeit one with its challenges and difficulties. With an aggressive timetable and limited staff resources, the library was able to pull off an unprecedented migration with minimal interruption to patron services. Workflows were modified, and some functionality was lost, but the potential for the library to fully control their system and data, and to integrate it with practically any other library system, is well worth the change. The NYUHSL, having identified some key areas for development, has the potential to be a significant contributor to the international Koha community, making Koha a better ILS for libraries the world over.