Maintenance - when to expect it and why it's important!

by Evelyn Hartline on Feb 28, 2023

As ByWater has grown, our systems and procedures have grown with us. With this growth comes the ongoing and perennial need to continue to keep all of our systems current with the latest best practices for security and system operation. This need leads directly to what some might consider a scary phrase: Planned Maintenance.

When we talk about planned maintenance for your system, there are two main types: maintenance to the product we're supporting (Koha, Aspen, Libki, Metabase, etc.) and maintenance to the infrastructure and systems where those products reside.

Maintenance to the product we support is typically done more often, and can be subdivided into two broad categories: maintenance for all partners and maintenance for specific partners.

Maintenance for all partners is specific to the product we're supporting - for example, Koha gets monthly updates the third full week of every month. These updates to Koha are applied at night after 9pm local time over the course of a week to ensure that minimal interruption is introduced during usual library hours, and with the aim of not impacting normal day to day operation. We are able to accommodate different partners in different timezones because every Koha is individual, even while getting the same updates as other libraries in the same timezone. These updates typically include quality of life improvements, bug fixes, and other changes provided both by ByWater staff and the Koha Community.

There may be a time when you ask us to do something specific to your Koha installation that could be disruptive to your operations. For these situations, we have a planned maintenance window set aside on Tuesday nights, from 21:00 to 02:00 (local time), so if you are in Pacific Time, your window begins at your 21:00, not Eastern Time's 21:00. For example, if your library has moved to Elasticsearch, you'll know that the procedure is done after hours so that searching is only impacted during your library’s quiet hours during the reindexing process. This maintenance window for individual partners can include both kinds of maintenance: it could only have to do with the application(s) you use at your library, or be maintenance to the infrastructure that is specific to your installation. Please note that in the specific case of reindexing, we do not guarantee that all searching will be restored within the window - some libraries take several hours to reindex - what we do guarantee is that it will, barring some other kind of issue, be ready for you in the morning when your library opens.

If product maintenance is like calling a plumber to take a look at a leaky faucet in your kitchen, then infrastructure maintenance is like the city working on the water lines in your neighborhood.

Infrastructure maintenance has to do with maintaining the technologies, both hardware and software, that actually run your Koha, Aspen, or other application. These are the "pipes" that get water from the city to your home. This maintenance is incredibly important, but can pose a challenge to plan out. Because the infrastructure is where these products live, separating the work by time zone isn't an option.

To help minimize the impact, we have an established, publicly documented maintenance time for this kind of work: Sunday morning between 6 am and 10 am Central Time. Of all the windows, this is the least busy time for the majority of our partners, and won't interrupt any other processes that partners use overnight (crons, backups, overnight reports, and the like). To follow our earlier metaphor, this is the time frame the city provides your neighborhood when the water will be turned off so they can work on those pipes. While it, unfortunately, will never be entirely convenient for every household in the neighborhood, the time is chosen to be the least impactful for the most people.

Infrastructure maintenance is performed less often than product maintenance, but just like the city's water supply, it is vitally important to the health of the infrastructure to perform it. Often, these include important security updates and upgrades that keep your products running safely and efficiently.

Lastly, there are infrastructure maintenance times that are outside of the normal Sunday morning window. These are the least frequent, and when they do occur we do our best to ensure you're alerted to them. For these kinds of maintenance for Koha, you will see news items in your staff client letting you know the time and date. Here again, we attempt to do these types of maintenance during the least busy times for our partners, however sometimes there are additional constraints that affect when these can happen. We’ll still do our best to avoid as much downtime as we can. Oftentimes these types of maintenance are performed on our redundant systems and the outages are classified as “bouncy” as the clusters are rearranged to allow for the maintenance to be completed. Normally these outages are a few minutes each a couple of times over (once to move services out of a node, once to move services back into their normal node). These last two types of maintenance normally can not be rescheduled for individual libraries, as the work to be performed normally will affect most hosted partners.

Where do I find the information?

Published details for our maintenance schedule can be found on our website here at Scheduled Maintenance, and if any infrastructure maintenance is occurring you'll also see that noted in our status tracker here Status Updates. As always, if you have questions, we're always available through the ticketing system Submit a Ticket or on our helpline at 888-900-8944.