Data migration across multiple versions with docker

pedro · May 31, 2025, 7:55am

6 years ago, while visiting family who are volunteer doctors working through an NGO in rural South America, I set up an instance of openemr 5.0.2 running on a Windows laptop with WAMP to assist them in their work of primary health care. It has worked well. They have come home for a visit and brought the old laptop which needs an update. I have managed to successfully build a Raspberry Pi5 with an NVMe drive to run openemr 7.0.3 in a docker container using the docker-compose.yml from the openemr dockerhub site. This is working very well.
My problem now is how to get the hundreds of data records from the old laptop (version 5.0.2) to the new setup on the new hardware (version 7.0.3). I have the mysql database extracted using myqldump, but clearly have to do quite a bit more than just replact the database contents. Some on this forum have suggested using “sql_upgrade.php” which sounds great, but this file does not appear to be in the docker image.
Do I really need to install and upgrade the 6 versions (Upgrade 7.0.2 to 7.0.3 Upgrade 7.0.1 to 7.0.2 Upgrade 7.0.0 to 7.0.1 Upgrade 6.1.0 to 7.0.0 Upgrade 6.0.0 to 6.1.0 Upgrade 5.0.2 to 6.0.0)? Or is there some way to do this in a more ef
ficient way?
I would be gratful for any suggestions so that I can give them a working Pi5 with their patient data on it to take back with them next month.

jesdynf · June 1, 2025, 12:53am

5.0.2? That’s… Alright, here’s the deal. I can’t solve this for you – but I can give you a quick overview of all the moving parts, and hopefully put you in a position to decide if you can pull it off or not in the time you’ve got.

5.0.2 was the first docker we put in our devops repo. It’s not the first container we built but every version that followed we got better at understanding what we’re doing. Could you get your installation from 5.0.2 WAMP into 5.0.2 containerized? Maybe. Should you? Maybe. However, it happens that our first Raspberry ARM8 build was 6.0.0, so you absolutely won’t be able to go directly into your Pi in that state.

So I think you’ll upgrade WAMP in place to 6 or 7 before importing it. Once it’s in the right container, the containers handle upgrades very sensibly. So don’t import a 6 or something into a 7 container, import it into a 6 container and then upgrade the container and re-up the docker-compose. The higher the version number you upgrade your WAMP to, the smarter the container you’re upgrading to is.

Of course, how you should get your WAMP into a Docker is a good question, since it’s not just the database but the patient records, the documents, all that, right? That’s where were we need to talk about my ingestion script. openemr-devops/packages/lightsail/ingestion.sh at master · openemr/openemr-devops · GitHub was my first crack at trying to build a tool to take OpenEMR’s backup tarball (not your mysqldump but the backup/export you make from inside the OpenEMR application menus) and slurp it into a container. I didn’t write it to work on containers that old, and I never get much feedback about the thing so I don’t know if it works well or if anybody uses it, but it is at the very least an example of the kinds of things you want to be thinking about if you want to containerize an existing installation.

That’s basically how I think this is going to go for you.

Create some form of in-place backup, since this process could be destructive.
Stepwise upgrade your existing installation from 5.x to as high as seems sensible.
Export your installation with the internal backup tool and import it with (or by following along) the ingestion tool into a container of the correct version.
Launch the new container and verify stability and correctness.
Stop the cluster, set the final version of OpenEMR in the compose file, and re-up the application.

It’s entirely possible, thinking this through, that you might have to perform two export and import pairs. One export from the WAMP system to a 5.0.2 backup, and from there into your personal workstation into a 5.0.2 Intel container, then perform containerized upgrades, then export from the 7.0.3 Intel to a 7.0.3 backup, then import that into a 7.0.3 ARM container. This process, while unwieldy, is safer than trying for an upgrade to 6 on the master copy of production when I don’t know how often your backup regimen has been tested.

Hope this helps.

pedro · June 2, 2025, 1:24am

Thank you very much @jesdynf for such a comprehensive reply. I will attempt what you suggest (I’m not an IT expert, but a retired engineer and IT hack). I will post how it goes.

Penguin8R · June 2, 2025, 7:01pm

@jesdynf is much wiser than I on the containers, so I’m not even going to try to make any suggestions there.
However, I might have an idea or 2 about the version upgrade part, and I have actually done quite a few version upgrades from version 5 to version 7, and actually even some from version 4 to version 7.
I’d like to hear @jesdynf thoughts on this, but since the existing data is already running in 5.0.2 WAMP, would it make more sense to pull that data into a 7.0.3 environment, on WAMP/XAMPP, run the software upgrade there (just to take the container variable out of the upgrade equation), and then migrate everything from WAMP/XAMPP into a 7.0.3 container, knowing that the version upgrade is done and the application/database are healthy and happy ? That would mean you only have to export & import once, although it would add the extra overhead of building a newer WAMP/XAMPP stack to do the heavy lifting on the version upgrade(s), but you have to do that anyway? Having been through a fair amount of these, some of which take 30+ hours of data processing just to bring the database up to the new version, my biggest concern in this situation would be having something ambiguous go wrong, and then be left trying to isolate if the source of the problem was the version upgrade, or the container setup. We don’t know the data volume, maybe it’s small enough to make everything easier, when you’re dealing with some of these databases that have 100K patients, half a million encounters, and millions of documents, it extends the cycle time for every experimental run as you’re testing a phase of the overall version upgrade process. So in this case, to simplify, it’s on WAMP now, keep it there until it’s current, and that way the last step is to bring everything online in the newest/smartest version of the Docker container structure that’s available.
I could be way off, in which case I deserve a virtual head smack for my logical failure!

jesdynf · June 2, 2025, 7:13pm

Well, you’re right – if the upgrade goes right. I don’t know if it will or not, and I don’t know what state local backups are in, or the prospects of recovery from them.

The reason I put the protocol together the way I did is because it’s guaranteed non-destructive. The 5.0.2 backup is the source of all the work that follows, and any failure of process never touches production. Stepwise upgrading production to 7 in-place skips a lot of steps, but it skips a safeguard too.

Penguin8R · June 2, 2025, 7:39pm

Oh yes, backups are essential. The original post, it sounds like they’re working on a “new setup on new hardware”, and the production/existing “old” system isn’t being touched yet anyway. The new computer, or the Pi5, wherever it ends up, until the data goes through the upgrade and the result is a working environment of V7.0.X, it’s all just a testing/experiment anyway. But it’s as good a time as any to point out how important regular backups are, and if the only time that someone makes a backup is because they’re about to do a version upgrade, that’s not a situation you want to find yourself in.

pedro · June 3, 2025, 12:32am

Thank you, @Penguin8R and @jesdynf for your comprehensive input. Yesterday I started along the lines suggested by @jesdynf. I have regular backups made using mysqldump. It looks like there are 6 years worth of records for 931 patients.
However, when I try to do a full backup from the GUI, I get a message “There was an error on the backup.”, which I haven’t been able to trace ( and no backup is produced) - I suspect there is some script missing in the tree somewhere.
The doctors are working in a remote part of South America (in Australia for a 6 month break), and the laptop (Windows10 WAMP 3.1.9) is off-grid and has not been connected to the internet for 6 years (since I first installed it). I am hesitant to break a working instance of OEMR, so I thought I would try to duplicate the installation on my linux desktop in a KVM/qemu VM running Windows10. That way I could be free to proceed without risk of destroying the only working copy. I transferred the data using a plug-in USB drive. This process has not been straightforward.

The current version of WAMP throws PHP errors due to deprecated php functions.
I tried various versions of PHP, but get various errors about PHP needing to be >7.1, >7.3.0,… I’m still working through that.
I tried to get a clean version of OEMR 5.0.2 running but got errors about missing files. It turns out that the clean version did not have a /vendors directory, so I just copied /vendors from 7.0.3 and the instance ran. I had some issues getting the sql data over and then fell asleep and went to bed (I don’t have full time available to do this till its done - so don’t feel I’m ignoring your responses if it takes a while to get back to you).

I have not been able to find earlier versions of OEMR docker images ( I’m relatively new to docker) otherwise I would try to get the earlier versions running in a container as suggested by @jesdynf earlier. I suspect I will successfully get to that point eventually.

I like @Penguin8R 's suggestion to pull the 5.0.2 data into 7.0.3 (which is what I originally tried to do when I first started before I came to this forum - but it didn’t work (most likely because the sql schemas have changed over the version evolution and I was going directly from WAMP to docker). It might be worth a try again if I get the desktop VM system working so I don’t permanently break anything.

stephenwaite · June 3, 2025, 1:43pm

hi @pedro , you could do a mysqldump of the database and make a copy of all the files in the webroot (that’s what comprises the backup).

Then try the easy dev docker where you can import the 5.0.2 database into the easy dev mariadb docker and run sql_upgrade at http://localhost:8300/sql_upgrade.phpto practice upgrading.

pedro · June 5, 2025, 7:22am

Thanks for your input @stephenwaite . I got a little success in getting easy dev docker up and running - though I am still pursuing the previous methods. I now have a linux workstation running the easy dev docker environment -which works (7.0.3).
From within the mariadb container, running mariadb, I successfully imported the 5.0.2 database (source backup_of_5.0.2.sql). I can query the database from the command line and see the data using phpMyAdmin.
I then ran http://localhost:8300/sql_upgrade.php which completed successfully (without throwing any errors).
However, when I then try the web interface (https://localhost:8300) I get errors. The errors in the webserver log in the openemr container are:

[Thu Jun 05 06:55:09.823721 2025] [php:notice] [pid 1042:tid 1042] [client 172.20.0.1:54600] OpenEMR Error: Decryption failed HMAC Authentication!\nPossibly Config Password or Token. Error Call Stack:\n#0 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (106): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;coreDecrypt()\n#1 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (513): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;decryptStandard()\n#2 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (160): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;collectCryptoKey()\n#3 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (72): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;coreEncrypt()\n#4 /var/www/localhost/htdocs/openemr/sites/default/config.php (23): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;encryptStandard()\n#5 /var/www/localhost/htdocs/openemr/interface/globals.php (556): require_once()\n#6 /var/www/localhost/htdocs/openemr/interface/login/login.php (44): require_once()\n
[Thu Jun 05 06:55:09.823811 2025] [php:error] [pid 1042:tid 1042] [client 172.20.0.1:54600] PHP Fatal error:  Uncaught OpenEMR\\Common\\Crypto\\CryptoGenException: OpenEMR Error : Key in drive is not compatible (ie. can not be decrypted) with key in database - Exiting. in /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php:525\nStack trace:\n#0 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php(160): OpenEMR\\Common\\Crypto\\CryptoGen->collectCryptoKey()\n#1 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php(72): OpenEMR\\Common\\Crypto\\CryptoGen->coreEncrypt()\n#2 /var/www/localhost/htdocs/openemr/sites/default/config.php(23): OpenEMR\\Common\\Crypto\\CryptoGen->encryptStandard()\n#3 /var/www/localhost/htdocs/openemr/interface/globals.php(556): require_once('...')\n#4 /var/www/localhost/htdocs/openemr/interface/login/login.php(44): require_once('...')\n#5 {main}\n  thrown in /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php on line 525

This page isn’t working

**localhost** is currently unable to handle this request.

HTTP ERROR 500

I then edited the …/sites/default/sqlconf.php to provide the 5.0.2 password to openemr and get a different error:

[Thu Jun 05 07:09:10.506880 2025] [php:error] [pid 38:tid 38] [client 172.20.0.1:50744] PHP Fatal error:  Uncaught TypeError: mysqli_query(): Argument #1 ($mysql) must be of type mysqli, false given in /var/www/localhost/htdocs/openemr/vendor/adodb/adodb-php/drivers/adodb-mysqli.inc.php:1454\nStack trace:\n#0 /var/www/localhost/htdocs/openemr/vendor/adodb/adodb-php/drivers/adodb-mysqli.inc.php(1454): mysqli_query()\n#1 /var/www/localhost/htdocs/openemr/vendor/adodb/adodb-php/adodb.inc.php(1613): ADODB_mysqli->_query()\n#2 /var/www/localhost/htdocs/openemr/vendor/adodb/adodb-php/drivers/adodb-mysqli.inc.php(1195): ADOConnection->_Execute()\n#3 /var/www/localhost/htdocs/openemr/library/ADODB_mysqli_log.php(60): ADODB_mysqli->execute()\n#4 /var/www/localhost/htdocs/openemr/library/sql.inc.php(66): ADODB_mysqli_log->ExecuteNoLog()\n#5 /var/www/localhost/htdocs/openemr/interface/globals.php(301): require_once('...')\n#6 /var/www/localhost/htdocs/openemr/interface/login/login.php(44): require_once('...')\n#7 {main}\n  thrown in /var/www/localhost/htdocs/openemr/vendor/adodb/adodb-php/drivers/adodb-mysqli.inc.php on line 1454

… and the web page response is

This page isn’t working

**localhost** is currently unable to handle this request.

HTTP ERROR 500

I feel that I am close, but lost. If I can get this running on the linux workstation I hope I can successfully reproduce the process on the raspberry Pi5.

stephenwaite · June 5, 2025, 3:09pm

hi @pedro, no need to edit sqlconf.php, you have to copy the keys from the backup of the filesystem, look in sites/default/documents/logs_and_misc/methods/ into the docker.

pedro · June 8, 2025, 11:19am

A big thank you, @stephenwaite and also @Penguin8R and @jesdynf !
Using the developer-easy environment suggested I have been able to try all the advice given without busting the only working instance on the old laptop (WAMP). I then shut down the developer-easy platform and started from scratch on my linux desktop using the production dockers.
I noted the following observations:

Use mysqldump to get the openemr (5.0.2) database off the laptop.
I took a copy of the whole original …/sites/default/documents folder
Start up the openemr 7.0.3 environment on the workstation using docker-compose. I added an additional volume of a host directory mounted under /mnt/OEMR_xfer on both the openemr and mariadb containers. This made it easier to move files into the containers from outside. I’ll probably remove this when the application goes into final production.
Log into the mariadb container using docker exec -it <container_number> /bin/bash.
Log into mariadb using mariadb -uroot -p
Drop the openemr database
Create a new openemr database
Ingest the original data using source “original_sql_dump.sql” from 5.0.2 which had been copied to the /mnt/OEMR_xfer directory
Exit the container session
Log into the openemr container using the same method as above
Overwrite the ..../sites/default/documents folder tree with a copy of the original 5.0.2 version which had been copied to the /mnt/OEMR_xfer directory
Exit the openemr container
Move a copy of sql_upgrade.php from the git repository into /var/www/localhost/htdocs/openemr in the openemr container (the docker image does not have this file).
Point the browser to localhost/sql_upgrade.php and select upgrade from 5.0.2 and wait till the process finishes.
point the browser to localhost and log in with the original credentials from 5.0.2.
Viola!

An observation I made is that the keys sixa and sixb contents of in the openemr container directory /var/www/localhost/htdocs/openemr/sites/default/documents/logs_and_misc/methods are DIFFERENT to what is shown for sixa and sixb in the mariadb openemr database keys table. I don’t know if this is right or it makes a difference. On one of the test runs in the development-easy environment I changed the sixa and sixb keys in the database to match what was in the openemr container and didn’t notice anything different. In the end I left them different.
As a final task I backed up the running instance using

docker exec -i <container_number> mariadb-dump -uroot -proot --all-databases >mariadbDUMP.sql

… the GUI option to backup doesn’t work because the configuration mysql binaries point to c:\xampp… which are in the globals table of the database copied from 5.0.2. Also, mariadb doesn’t have mysqldump anymore, but usesmariadb-dump - I guess I could change something in the openemr source to fix this, but the docker exec option above worked fine.

Again a big thankyou to the 3 people who contributed to remedy my knowledge deficiency!
Next I will replicated this process on the final hardware (Raspberry Pi5).

pedro · June 9, 2025, 8:02am

I’ll write this up better when I get the final version going.

pedro · June 12, 2025, 7:20am

I would appreciate any knowledgeable insight into the following problem.

I have installed the development-easy environment.
I have run OEMR 7.0.3 on it
I have imported the old openemr5.0.2 database into the mariadb container.
I have run the sql_upgrade.php script.
I have copied the 5.0.3 …/sites/default/document folder to the 7.03 instance
It works.
HOWEVER…
If I do exactly the same on the raspberry Pi5 instance using the production-arm docker rather than the development-easy environment…
openemr 7.0.3 works
migrating the 5.0.2 mariadb database works
recursively copying the …/sites/default/document works.
running sql_upgrade.php seems to work, but doesn’t finish doing the final ACL fixes.
openemr gui logs the following errors:

[Thu Jun 12 06:50:19.375154 2025] [php:notice] [pid 320:tid 320] [client 172.18.0.1:52070] OpenEMR Error: Decryption failed HMAC Authentication!\nPossibly Config Password or Token. Error Call Stack:\n#0 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (106): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;coreDecrypt()\n#1 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (513): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;decryptStandard()\n#2 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (160): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;collectCryptoKey()\n#3 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php (72): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;coreEncrypt()\n#4 /var/www/localhost/htdocs/openemr/sites/default/config.php (23): OpenEMR\\Common\\Crypto\\CryptoGen-&gt;encryptStandard()\n#5 /var/www/localhost/htdocs/openemr/interface/globals.php (557): require_once()\n#6 /var/www/localhost/htdocs/openemr/interface/login/login.php (44): require_once()\n
[Thu Jun 12 06:50:19.375280 2025] [php:error] [pid 320:tid 320] [client 172.18.0.1:52070] PHP Fatal error:  Uncaught OpenEMR\\Common\\Crypto\\CryptoGenException: OpenEMR Error : Key in drive is not compatible (ie. can not be decrypted) with key in database - Exiting. in /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php:525\nStack trace:\n#0 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php(160): OpenEMR\\Common\\Crypto\\CryptoGen->collectCryptoKey()\n#1 /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php(72): OpenEMR\\Common\\Crypto\\CryptoGen->coreEncrypt()\n#2 /var/www/localhost/htdocs/openemr/sites/default/config.php(23): OpenEMR\\Common\\Crypto\\CryptoGen->encryptStandard()\n#3 /var/www/localhost/htdocs/openemr/interface/globals.php(557): require_once('...')\n#4 /var/www/localhost/htdocs/openemr/interface/login/login.php(44): require_once('...')\n#5 {main}\n  thrown in /var/www/localhost/htdocs/openemr/src/Common/Crypto/CryptoGen.php on line 525

so… it works on a linux docker, but not on a Pi5 docker
Any insight appreciated.

pedro · June 13, 2025, 6:20am

Thank you, @stephenwaite . I am still getting errors and have a question:
should the contents of the sixa and sixb entries in the keys table be the same as the contents of the sixa and sixb files in “…/sites/default/documents/logs_and_misc/methods/” directory?

stephenwaite · June 13, 2025, 2:41pm

hi @pedro, yes, when you move the database you have to move those keys which should come along with the openemr documents directory.

edit: actually no, sorry, they are different keys. The keys in the database unlock any encrypted files on the drive (filesystem) while the keys in the filesystem unlock the ecrypted entries in the database.

checkout Brady’s explanation