Migrating data from Ubuntu 14.04/openEMR-4.2 to Ubuntu 24.04/openEMR-8.x: urgent assistance required

Day-to-Day Users
1

Dear OpenEMR community… Been away for a long time… I am back with an issue that requires urgent assistance:

I have an Ubuntu 14.04/openEMR-4.2 server that hasn’t been updated through the software updates since 2018. Running on a SATA hard drive.

I’ll want to upgrade the server to run on a new 1TB SSD with Ubuntu 24.04/openEMR-8.x. What are the recommended OpenEMR upgrade path to migrate the 4.2.0 data to 8.x while retaining the data integrity…

Will be eternally grateful for any assistance you can provide for a successful data migration

Thank you

Olayinka Raji

Tel: 0803 782 2496
0802 321 0496

2

Hi Olayinka, welcome back.

This is certainly possible, but please do not upgrade the existing Ubuntu 14.04 server in place and do not work from the only copy of the SATA drive. That system is now your source archive and should remain untouched until the migration has been completely tested and verified.

My recommended approach would be:

  1. Make at least two verified backups of the current system:

    • A complete database dump using mysqldump

    • The entire OpenEMR directory, especially sites/default/documents, edi, era, letter_templates, sqlconf.php, custom forms, custom reports, modules and any locally modified source files

    • Preferably also create a full disk image of the old SATA drive

  2. Restore a copy into an isolated test machine or virtual machine using versions of Ubuntu, PHP and MySQL/MariaDB compatible with OpenEMR 4.2.0. Do not expose that old system to the public Internet.

  3. Upgrade OpenEMR sequentially using the official upgrade guides:

4.2.0 → 4.2.1 → 4.2.2 → 5.0.0 → 5.0.1 → 5.0.2 → 6.0.0 → 6.1.0 → 7.0.0 → 7.0.1 → 7.0.2 → 7.0.3 → 7.0.4 → 8.0.0 → 8.1.0 → 8.2.0

At each stage, install the appropriate OpenEMR source, copy only the required site data and configuration, run sql_upgrade.php, select the actual version being upgraded from, and confirm that the database upgrade completes without errors before continuing.

Because PHP and database requirements changed considerably over these releases, you may need more than one temporary VM. For example, perform the older upgrades in a legacy PHP environment, then move the upgraded database and documents into a newer environment for the later releases. Trying to make OpenEMR 4.2 run directly under Ubuntu 24.04 will create unnecessary problems.

  1. After reaching the current release, install a clean Ubuntu 24.04/OpenEMR 8.2 system on the SSD and transfer the fully upgraded database and site documents into it.

  2. Before going live, verify:

    • Patient and encounter counts

    • Documents and scanned records

    • Appointments

    • Billing and insurance data

    • Prescriptions

    • Users, ACL permissions and facilities

    • Custom forms and reports

    • Printing, document paths and any external integrations

Keep the original server available but disconnected as a read-only fallback until the new system has operated successfully and the data has been validated.

Also, do not copy the complete 4.2 source tree over the new OpenEMR source. Old customizations need to be identified and reviewed individually. Copying old PHP files wholesale into version 8 will likely introduce compatibility and security problems.

This is a large upgrade spanning more than ten years of database changes. If the records are operationally urgent and there is no experienced Linux/OpenEMR administrator available locally, this is one case where obtaining professional migration assistance would be well justified. Whatever route you take, perform the entire migration as a rehearsal first and document every command before touching production.

The official sequential Linux upgrade pages are listed here under the OpenEMR Upgrade category:

Be advised that this is the super safe process when following. Experienced users may want to skip some of the drudgery and skip some parts.

However, it cheers me to recently have seen several folks upgrading recently. Now you’ll get to enjoy all the great new features we’ve added.

Good luck,
-Jerry

1 Like
3

Dear Jerry P, thanks so much for your detailed response.

I’ve made 3 copies of the backup of the data on to 3 different secure devices.

Pls see below the steps I intend to take arising from your suggestions. I am an absolute novice in openEMR. :cry::joy::hugs:

Deploy a new SSD as the “new production server,” think of it first as my migration laboratory.

Original SATA (Ubuntu 14.04 + OpenEMR 4.2)
│
│ ← Never modify
│
├──────────────► Emergency fallback
│
▼
New SSD
(Mirror of SATA)
│
├── Test OpenEMR 4.2
├── Verify hospital data
├── Upgrade to 4.2.2
├── Verify
├── Upgrade to 5.0.2
├── Verify
├── Upgrade to 6.x
├── Verify
├── Upgrade to 7.x
├── Verify
├── Upgrade to 8.x
└── Full acceptance testing
│
▼
GO LIVE

Pls let me know if these steps are sufficient for a fast migration…

Thanks in anticipation of your usual prompt response..

With best regards

Olayinka Raji

Tel: 0803 782 2496
0802 321 0496
09050590366

4

First, a quick note concerning private data. I’d remove your phone numbers from all your post. While we do a pretty good job vetting users of the forum posts can still be visible to the public at large. I recommend adding contact information you want available to our users in your profile.

Second thanks for being detailed in your response. Very helpful.

It’s been a very long time since I have tested doing upgrade covering so many versions but if I recall, any upgrade that may have an issue it’d be v5.0.0 because that version has a lot of migration concerning the eye forms and converting several tables to InnoDB engine.
I believe with v6.0.0 we introduced UUID to database and that upgrade will be a much longer run time as it goes through and adds UUIDs to most all tables.

I would bet that you could install latest and run entire upgrade however, if it was me I would

  • Install 5.0.2 and upgrade DB to it.
  • Install 8.0.0 upgrade
  • 8.2.0 upgrade then done.
    You can run sql upgrade as often as you need to verify via on screen log for errors or issues.

If you have the time you may want to increment more often e.g. 5.0.2 → 7.0.0 and so forth.

Lastly, I think it’s HIPAA requirement that you maintain and store a backup to an offsite secure location. Or it may be insurance company requirement. Whichever is the case, it’s just a good policy.

I hope this helps.

2 Likes
5

Dear friends, greetings… I brought my challenges to the forum, indicated the steps I was taking, following from the advice I got from friends on the forum.
I bring a progress report of what’s happened so far:

I cloned the old server’s 1tb SATA hard drive on to a fresh 1TB SSD drive.

:white_check_mark: The computer booted successfully from the new 1 TB SSD.
:white_check_mark: Ubuntu loaded normally.
:white_check_mark: Firefox opened.
:white_check_mark: OpenEMR loaded.
:white_check_mark: A patient record opened correctly.
:white_check_mark: Historical encounter data is present.
:white_check_mark: The desktop and applications are behaving normally.

What you have now is:
Original SATA (untouched)
│
└──► Perfect sector-by-sector clone
│
▼
New 1 TB SSD
Boots successfully
Ubuntu OK
OpenEMR OK
Patient records OK

The old SATA drive has been kept aside. The new SSD server has been put in operation to confirm we have a functional server.
We begin all repairs and subsequent updates/upgrades tomorrow on the new SSD drive. Multiple data backups have been saved on a USB device, an external hard drive and on a workstation.
Keeping the Ubuntu at 14.04, serial upgrade/update from openemr-420 -->5.0–>5.2–>6.0–>6.2–>7 Testing the integrity of the data at each point and making backups. Then Ubuntu 16–>18–>20–>22…

My gratitude goes to the whole community…

Thank you all

6

Jerry has it pretty much nailed. Having done some of these, and have another one in progress right now, you will need to create a temporary server environment, either physical or virtual, and deal with that first upgrade step from Version 4.X to Version 5. Getting through that database transition to InnoDB is key, and depending on the size of your V4 database, it may take hours or even days, depending on the hardware resources available.

One thing that will help, is leaving your old log table behind in your V4, just re-create the empty table structure after you dump and migrate your database into your upgrade environment. It likely has millions or potentially tens of millions of rows in it, and the version upgrade will burn significant time on the storage engine conversion for that table alone.

Chances are, trying to do OS/platform upgrades and OpenEMR version upgrades all on that platform at the same time will prove to be rather frustrating. If you have control of physical hardware, you’d be better off building a clean Ubuntu 26 LTS server platform, and establish a temporary VM on top of it running an Ubuntu 16 or 18 environment to get you from Version 4.X to Version 5.x, then move everything to the physical host and completely the rest of the sequence from V5 to V8.2.

As Jerry mentioned, if you had any customizations to your V4 environment, it’s going to be critical to have a running copy of your V4 to compare with if you wish to re-create any for them, because none of the old application code from V4 is going to function anymore in the V8.x infrastructure.

You didn’t mention the underlying LAMP stack, but if your V4 platform was running MySQL, this is also a great time to migrate your database back end to MariaDB.

It’s extremely important to pay close attention to the output during the version upgrade processes, you may even want to print/save the output for troubleshooting if & when you run into problems or data anomalies.

Running a full database check via the built in tools in MariaDB is also highly recommended, because if you have any table level data issues, those can and will trip up the version upgrade process. Newer versions of MySQL/MariaDB, have changed standards and handling for things like date fields, and there may be data rows in your V4 database that need to be manually addressed and corrected before the version upgrade manipulations will execute correctly.

One last thing to point out, if your old server was vintage 2018, you’ll probably benefit from having newer/faster hardware to grind through all the version upgrade operations, and as Jerry already made clear, you really need to maintain your V4 environment for data rentention, reference and QA both during & after your version upgrade.

1 Like
7

Thanks, I must confess that most of the response sounds like Greek to me… Because, as we speak, I am having data integrity issues with the working Ubuntu 14.04/OpenEMR-4.2 server, currently running on the new 1TB SSD. The old SATA has been set aside as suggested by Jerry and the Penguin.

The issue is random missing prescriptions and clinic encounter narratives. This is quite strange and I am at a loss for solutions. On a patient’s file, while there are dates for encounters, narratives and prescriptions are missing for some patients. It is so random. Users complain that this happened after some unscheduled power outages affecting the server.

Is there a solution to this? This has affected any idea of an update/upgrade.

Thanks in anticipation of your favourable response.

D’Raj

1 Like
8

Power outages on databases are bad, even worse with the old MyISAM storage engine from OpenEMR V4.2.

You really don’t want to be trying to do all this on the same hardware, you should be following Jerry’s original advice.

Did you run any database table health and integrity checks on the system before/after cloning to to the SSD ? That is critical whenever a database is moved or copied, regardless of how it was done.

Short term, you need to keep the V4.2 stable and running, but you have a potentially serious issue there now that you’ve had activity on that system after cloning the drive and before discovering the data corruption issues. Depending on the volume of data entry, it might be best to note the exact missing prescriptions/encounters and the corresponding patients, then boot your system from the old drive and verify if the issue was caused by the cloning process. If the issues aren’t there on the old drive, then you know that either the cloning process or the subsequent power outages are to blame. I can tell you from past experience that the old MyISAM tables were prone to damage from system crashes due to loss of power or hardware issues, and that’s why a production EMR host should never be running without a UPS to afford some protection.

Did you by any chance make a backup of the database after cloning to the SSD, but before the power outage events? That would be the point of reference that you really need to define where the data problems started. My guess is you’ve got corrupted or missing rows in your data tables, but it’s essential to determine if the damage was done during the cloning process or it’s due to the power outages affecting the host. Once you answer that question, you can decide what needs to be done next. Shopping for a UPS should be pretty high on your priority list too, you really don’t want to be halfway through a version upgrade and then lose everything because of another brownout or blackout event.

Again, review what Jerry outlined for you and pay attention to his suggestions, this is not a scenario where “upgrade in place” is a viable option, you need to leave your V4 alone and do the migration/upgrade testing on a different platform. It would also be wise to consider finding a local resource who is familiar with Linux system administration, database maintenance, and LAMP stack implementations to help you through the process. You haven’t mentioned anything about your backup/disaster recovery preparedness, so if that’s not already been addressed, you should be prioritizing that before you start experimenting with Version 8 and prototyping your upgrade process.

1 Like
9

Dear All, I cannot thank you all enough for your community spirit and the volume of advise I am getting…

Perhaps a few clarifications:

  1. The missing and disappearing prescriptions and encounter/clinic notes happened on the old SATA drive…;

  2. (1) was followed by the JSON parse error that grounded the server;

  3. After the JSON parse error was solved it was decided to update/upgrade the Ubuntu 14.04/openEMR-4.2 to Ubuntu 24.04/OpenEMR-8.x…;

  4. Migration and safety of the hospital’s almost 10 year data necessitated a replacement of the old 1TB SATA to a 1TB SSD… And the cloning…;

  5. Tested and run the old SATA. Made at least two verified backups of the current system:
    A complete database dump using mysqldump;
    The entire OpenEMR directory, especially sites/default/documents, edi,``era,``letter_templates, sqlconf.php, custom forms, custom reports, modules and any locally modified source files…
    as suggested by Jerry… In fact I have 4 verified backups: a MacBook workstation, an external SATA drive, a USB flash and a Windows Workstation…

  6. The old SATA now sits as our ‘museum’ of historical events… :blush:

  7. The hospital is now running on the new 1TB SSD that was cloned from the old SATA drive. All observations are now from the new SSD. As mentioned earlier, the missing and disappearing prescriptions and clinic notes existed on the old SATA drive. Probably caused by the power failures. The installation is now protected by a UPS and an inverter.

  8. The missing/disappearing data is rather random. The hospital has been asked to let all terminals engage the openEMR system maximally for the next few days. And record any incidents that indicate an anomally.

Instead of chasing isolated issues as they appear, let’s keep a simple log. For every problem, note just a few things:
Date, Patient, Encounter…

What was expected?

What actually happened?

After a few days, we can ask questions like:

  • Do missing narratives occur only for encounters from a certain period?
  • Are missing prescriptions/clinic notes confined to specific providers/entries/doctors?
  • Does patient search fail only for certain names?
  • Are the same modules involved repeatedly?
  • Is there a pattern in dates, facilities, or users?
  • do a list of very important patients for whom the total prescriptions and encounter notes are very very necessary… Go through their files and note what’s missing.

I shall return to site on Wednesday for further details…
So far no updates/upgrades have been done. Making sure of the integrity of almost 10 years of operational data.

The next step I intend to take is to make another disk image of the SSD, keep the SSD as a more recent data and do any updates/upgrades on the latest disk image..

Pls advise… Thanks

D’Raj

1 Like
10

Again, I cannot emphasize this enough. DO NOT attempt to do an upgrade in place on your old hardware. Your version ugprade path from V4.x to V8.x needs to be thoroughly planned and tested, multiple times, before you’ll be ready to do your production migration. I’ve seen sites where this process takes weeks or even months to get it right, so in the meantime, the facility needs their existing system in order to continue to function.

Also, look at your backups from before the “missing data” incidents, you should be able to retrieve details of the missing encounters, prescriptions, etc from backup.

1 Like