Are upgrade instructions AWS Standard Edition 5.0.2(5) to 6.0.0 available?

Support
#52

Actually, storing the backup in an S3 bucket would be even better and not limited by the file system on the Webserver.

#53

How much disk space does your EC2 instance have? I bet it does not have more than 120gb which iw what you would need in order to make a backup.

Even if OpenEMR backup from the admin menu does not work you could be able to back it up manually by going into the mysql container and copying the backup to your EC2 instance.

You would need to at least double the size of your disk space in order to make the backup, so here’s what I would do.

  1. increase the disk size of your EC2 instance to double of what it currently has to ensure you can make a proper backup.
  2. Make a manual database backup by using an interactive docker session in your mysql docker container
  3. copy the database backup to your EC2 instance
  4. Download the database backup to your computer (most likely via ftp)
  5. copy your open emr folde3r from your open emr docker instance
  6. restore your database and open emr files to a local docker instance
  7. ensure your backup is working locally
  8. update your local docker instance

If you update goes well then I would make a copy of your updated database and restore that to your EC2 instance along with the updated open emr files.

The idea is to do the upgrade locally and restore your updated files in your EC2, the reasoning behind this is your computer will most likely be much faster than the EC2 instance since it has better hardware specs in most cases.

Good luck.

1 Like
#54

Thank you for your fast and thoughtful reply!

Remember that OpenEMR Cloud (AWS) Standard has the database on its own RDS instance, and, thus, it’s not necessary to access it via an interactive docker session.

I will try the general procedure that you have outlined. I can manually mysqldump the entire database pretty easily, and I have been using 200 GB volumes … no problem with storage space on the non-docker part of the disks. The issue I noticed is that the docker portion of the volume is ~40GB and pretty close to full no matter what the total disk storage is …

I will try this after the clinic is closed today because a mysqldump locks access to the database for a while, and I cannot do this while we are seeing patients in clinic.
–RBL

#55

I was able to create a mysqldump > file.sql
However, when I connect the RDS instance interactively via MySQL-client and then source file.sql, I eventually get a failure with the following error:
Timeout, server ec2-aa-bbb-ccc-dd.us-xxx-2.compute.amazonaws.com not responding :frowning:
Evidently, there is some kind of timeout parameter that needs to be changed.
This long restore with some very large tables that do take a while to process.

Thus, I created a parameter group for my RDS instance and put in the following:
connect_timeout 31536000
interactive_timeout 31536000
wait_timeout 31536000

And tried the restore again … it’s running again, and I will call it a night for now. I’ll check in the morning to see if the restore (source file.sql) completed.
–RBL

1 Like
#56

The source file.sql (restoring the database) keeps failing before completion … this last time, AWS disconnected my terminal session and terminated the restore … “client loop: broken pipe.” I cannot get the restore of this large database to complete.
–RBL

#57

Are you making a backup or restoring? If making a backup I would contact AWS support if there’s nothing in the logs that can help indicate what is going on.

#58

Okay. I will ask AWS support.
RBL

#59

You don’t need to use mysqldump at all – you’re using RDS, and that means you’ve got access to database snapshots. All I ever intended here was for you to just fingersnap and take a snapshot of the database and put it to the side. No need for long timeout-fighting techniques or nightly locks.

(It would be nice if you also understood how to use this snapshot – the process of restoring it and then reaching into the OpenEMR instance’s sqlconf.php and assigning the new hostname of the restored snapshot.)

#60

There is an even easier way using RDS read replicas. I create an exact replica of the production database in the test region. Now, I just need to connect the web server to the new RDS database instance … how do I do this?

#61

Look for /sites/default/sqlconf.php and directly edit it with the new hostname. (Also note that if you’re using a read replica I’m not 100% logging on is possible, since any attempt to write a last-visited time or audit rows to the database will fail.)

#62

You can promote the read-replica to stand-alone easily (5 min).
–RBL

#63

When I try to log on using the (promoted to stand-alone) read-replica RDS instance, I never get the log-on screen … it’s just blank … very strange. I worked with AWS support to get rid of the SET commands (–set-gtid-purged=OFF) in the mysqldumpfile.sql and I’m going to try to import that into the test v502 system.
–RBL

#64

Blank page means Apache and PHP both spun up successfully, so check the logs to see what PHP’s mad about.

1 Like
#65

this is from the tail of the error.log

[Wed May 05 03:41:50.434358 2021] [php7:notice] [pid 23] [client qq.zz.yy.xx:52794] Module (Open EMR Fax Module\n) could not be initialized.#0 /var/www/localhost/htdocs/openemr/vendor/zendframework/zend-modulemanager/src/ModuleManager.php(175): Zend\\ModuleManager\\ModuleManager->loadModuleByName(Object(Zend\\ModuleManager\\ModuleEvent))\n#1 /var/www/localhost/htdocs/openemr/vendor/zendframework/zend-modulemanager/src/ModuleManager.php(97): Zend\\ModuleManager\\ModuleManager->loadModule('Open EMR Fax Mo...')\n#2 /var/www/localhost/htdocs/openemr/vendor/zendframework/zend-eventmanager/src/EventManager.php(322): Zend\\ModuleManager\\ModuleManager->onLoadModules(Object(Zend\\ModuleManager\\ModuleEvent))\n#3 /var/www/localhost/htdocs/openemr/vendor/zendframework/zend-eventmanager/src/EventManager.php(171): Zend\\EventManager\\EventManager->triggerListeners(Object(Zend\\ModuleManager\\ModuleEvent))\n#4 /var/www/localhost/htdocs/openemr/vendor/zendframework/zend-modulemanager/src/ModuleManager.php(120): Zend\\EventManager\\EventManager->triggerEvent(Object(Zend\\ModuleManager\\ModuleEvent))\n#5 /var/www/localhost/htdocs/openemr/src/Core/ModulesApplication.php(48): Zend\\ModuleManager\\ModuleManager->loadModules()\n#6 /var/www/localhost/htdocs/openemr/interface/globals.php(567): OpenEMR\\Core\\ModulesApplication->__construct(Object(OpenEMR\\Core\\Kernel), '/var/www/localh...', 'interface/modul...', 'zend_modules')\n#7 /var/www/localhost/htdocs/openemr/interface/login/login.php(34): require_once('/var/www/localh...')\n#8 {main}



[Wed May 05 03:27:59.858463 2021] [ssl:warn] [pid 19] AH01906: xxx.18.0.2:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[Wed May 05 03:27:59.859656 2021] [ssl:warn] [pid 19] AH01909: xxx.18.0.2:443:0 server certificate does NOT include an ID which matches the server name
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.18.0.2. Set the 'ServerName' directive globally to suppress this message
[Wed May 05 03:27:59.898088 2021] [ssl:warn] [pid 19] AH01906: xxx.18.0.2:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[Wed May 05 03:27:59.898102 2021] [ssl:warn] [pid 19] AH01909: xxx.18.0.2:443:0 server certificate does NOT include an ID which matches the server name
[Wed May 05 03:27:59.898207 2021] [core:warn] [pid 19] AH00098: pid file /run/apache2/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Wed May 05 03:27:59.899932 2021] [mpm_prefork:notice] [pid 19] AH00163: Apache/2.4.39 (Unix) OpenSSL/1.1.1c configured -- resuming normal operations
[Wed May 05 03:27:59.899949 2021] [core:notice] [pid 19] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

I’m going to guess that the issue is that I need to copy all the webserver site files over from the production to the test instance … I’m going to try that before I try to login.
–RBL

#66

Just a quick update. I created another v502p5 OpenEMR AWS Standard test system (duplicate of my production v502p5 system), and attempted the ./docker-compose up -d in the devops standard directory. The command seemed to run successfully. However, after this, the webserver no longer responded. This is now my 3rd attempt at the docker-compose upgrade, and each time I wound up with a crippled OpenEMR web server that would not respond at all (cannot even get a blank white screen or a login form).

Since docker-compose is not working for me, I will try a different upgrade method documented here: https://www.open-emr.org/wiki/index.php/Linux_Upgrade_5.0.2_to_6.0.0

Linux Upgrade 5.0.2 to 6.0.0

Contents

[hide]

Upgrade

  1. Move current (5.0.2) /openemr directory to a backup directory openemr_bk
  2. Unpackage 6.0.0 openemr into web directory and rename the directory to openemr
  3. Copy following directory from old 5.0.2 version to /openemr directory:

openemr_bk/sites/default/documents (COPY TO) => openemr/sites/default/documents

  1. Copy the openemr/sites/default/sqlconf.php file from your old 5.0.2 version into the new openemr version
  2. In the openemr/sites/default/sqlconf.php file, set the $config variable (found near bottom of file within bunch of slashes) to 1 ($config = 1;)
  3. Open openemr/sql_upgrade.php (http://localhost/openemr/sql_upgrade.php) in web browser, choose 5.0.2 and click ‘Upgrade Database’
  4. Configure optional settings in openemr/sites/default/config.php files

Download and install most recent patch

After finishing this upgrade to OpenEMR version 6.0.0, it is recommended that you check if there are any patches to apply. If so, install the most recent patch for version 6.0.0. Instruction to do this can be found on the OpenEMR Patches page.

I hope this works …
–RBL

#67

Just to better understand the problem here … why are we using ./docker-compose up -d rather than docker-compose up -d ? How do we know which yml file the docker-compose up is using? Note that the current upgrade to 6.0.0 process makes both the EC2 and RDS instances of an OpenEMR Cloud STANDARD 5.0.2 system unusable with respect to openemr end-user functions. Even restoring the EC2 instance does not restore the system. You have to restore the database instance also.

–RBL

#68

the ./docker-compose specifies the docker-compose in your current working directory so it will use the one that came with the openemr aws package not the system installed binary

think the easy way to upgrade openemr on aws is much simpler although this linked document needs an update, am pretty sure the process is in place for v6, right @jesdynf?

1 Like
#69

That model is using a downloaded Docker because the one available in Ubuntu 16 was problematic and I hadn’t yet worked out the repo manipulation I use in other cases. I still need to tear everything down and move to Ubuntu 20 for underlying structure and that won’t happen any longer.

1 Like
#70

By the way, I have become quite proficient at making my EC2 and RDS non-functional using the ./docker-compose up -d command on a OpenEMR Cloud 5.0.2p5 system. I’m not sure if there is any way to trouble-shoot this crash-and-burn process (test dummy) to understand why this fails, I’m game.
–RBL

#71

Okay, here is the process that I am currently following for the AWS OpenEMR 5.0.2(5) Cloud STANDARD to 6.0.0(1) upgrade. I have successfully upgraded twice using this procedure. Note that I cannot upgrade the existing EC2 because the docker volume is not big enough to accommodate 6.0.0. Thus, I had to create a duplicate system and then upgrade it. For some reason, when I create a new system using the cloudformation stack, the docker volume now has plenty of space.

Note that if the docker volume has enough room, it isn’t necessary to create a test system to upgrade, but it may be desirable to create and upgrade a test system just to make sure the desired upgrade procedure works.

Also, I’m not always 100% on whether to put a trailing / on the copies … if the files go into a subdirectory of where they should be, go in there and copy them out cp * …/

Anyway, here is my draft procedure. I was able to get it to work twice, and I am quite confident it works … although there may be minor errors that intelligent users should be able to compensate for. This will also allow you to test the upgrade (if it doesn’t work, you still have your relatively untouched production system).
One thing that is very touchy (can cause the webserver not to respond) is what the file permissions and ownership are on the openemr site. Compare the site on the production system and test system when in doubt.

Preparation:

  1. Make sure no one is currently using the system or will try to log into it for several hours.
  2. Back everything up. Create an AMI (image) of the production EC2 instance. Create a snapshot of the database RDS instance.
  3. Get authorization from AWS to have more than the default number of VPC 's so that you can create OpenEMR systems in other regions.
  4. Choose a new region (such as Ohio or any region you are not using currently on your continent)

Creating a duplicate system of your production OpenEMR STANDARD system

  1. Run cloudformation Stack from the Marketplace Openemr Standard edition, which you should already have a license for.
  • Be careful to choose a region OTHER THAN THE CURRENT REGION of your production OpenEMR system (the system you are using now).
  • Choose the latest 5.0.2 (or whatever version you are upgrading from)
  • Make careful note of the passwords for the administrator and database you use in your cloudformation stack.

  1. After the cloudformation stack has successfully completed, “apply immediately” every recommendation for the new RDS database instance (upgrading to 5.7.xx, GP2 instead of magnetic storage, etc.).

  2. Patch the system to the patch level you used (such as v5.0.2p5) in your production system so that everything is as similar as possible to your production system.

  3. Encrypt and replace and resize to 200GB the root volume of the duplicate system EC2. This involves making a snapshot of the root volume, copying the snapshot with encryption enabled, creating a new volume with the encrypted copied snapshot (and enlarging it to 200 GB at the same time), noting the device /dev/sda1 or whatever of the root volume, stopping the duplicate system EC2, detaching the current root volume, and attaching the encrypted, enlarged (new) root volume to your instance, and starting the duplicate EC2 back up.

On the duplicate OpenEMR system:

  • Add inbound ssh ip address you will need to connect to your new EC2 (security group change)
  • ssh to the test EC2 instance. Reboot if required. Then do a sudo apt update, and a sudo apt upgrade.
  • Consider changing the RDS instance to a type t3 instance type - it should have better performance at an identical or better price than the t2s. Unfortunately, the EC2’s can’t be upgraded hardware-wise this way.
  • Enable RDS storage autoscaling
  • Change rds backup retention to 14 days.
  • Enable enhanced monitoring.
  • Enable EC2 all log exports
  • Enable RDS performance insights
  • If necessary, make the following change so that the sudo apt update is successful: https://stackoverflow.com/questions/61074587/why-do-i-see-failed-to-fetch-error-while-dping-apt-get-update
  • Go to the new EC2 instance IP address with a browser (probably will need to use HTTP:// to connect). Make sure that the test system allows users to log in and seems to work.

Creating the mysqldump backup of the production database

  1. Perform a mysqldump of your existing database. NOTE THAT THIS WILL FREEZE ANY USERS OF THE SYSTEM UNTIL THIS IS COMPLETE.
  • Here is an example template of the command you should use (make sure your system is large enough to hold your database, which could be huge. If not, you will need to enlarge the root volume of your production EC2 first – I recommend at least 200GB, but yours might even need to be larger - check your database size!)

mysqldump -h fr1xhyxxxxxxx.cydpyyyyyy.us-east-1.rds.amazonaws.com --set-gtid-purged=OFF -u openemr -p openemr > mysqldumpfile-2021-06-06-2130h

where fr1xhyxxxxxxx.cydpyyyyyy.us-east-1.rds.amazonaws.com is the endpoint address of your PRODUCTION RDS database instance (you are copying your production data into a SQL file),.

  1. Make sure that your Production and Duplicate EC2 instance both have AmazonS3FullAccess permissions so that they can copy and write to S3 buckets. If not, go to the IAM role of your instance(s) and add a full permissions S3 policy (just search for S3). Otherwise, you won’t be able to use S3 to copy your system data.

  2. Copy the mysqldumpfile-2021-06-06-2130h to an S3 bucket of your choice (you may need to set one up if you don’t have any ready).
    The command will probably look something like this:
    sudo aws s3 mv mysqldumpfile-2021-06-06-2130h.sql s3://openemr-v502p5-2021-06-06-migration-data/

  3. Copy the openemr directory from the production instance to the duplicate instance EC2 file system to copy the documents from the EMR
    The command is as follows:
    sudo docker cp standard_openemr_1:/var/www/localhost/htdocs/openemr/ openemr-v502p5/

Then move this directory to an s3:
sudo aws s3 mv openemr-v502p5/ s3://openemr-v502p5-2021-06-06-migration-data/openemr-v502p5/ --recursive

Install the database on the duplicate OpenEMR system

  1. copy the mysqldump.sql file from s3 to the new (duplicate) EC2 test instance file system.
    The command to copy this file will look like this:
    sudo aws s3 cp s3://openemr-v502p5-2021-06-06-migration-data/mysqldump-v502p5-2021-06-06-1615h.sql

Also copy the openemr directory from the production system that was copied to s3:
sudo aws s3 mv s3://openemr-v502p5-2021-06-06-migration-data/openemr-v502p5/ openemr-v502p5/ --recursive

  1. Create a new parameter group for the duplicate RDS instance. Search for and replace every “timeout” containing parameter (except slave related) with maximum allowable so that the large mysqldump import does not time out.

  2. Install mysql client with sudo apt-get install mysql-client if necessary and Restore the mysqldump of the copied clinic OpenEMR mysqldump database backup file to the test RDS system database

The command will look something like this:
mysql -h xxxxxxxrrrrrssssss.ffaass33gg22.us-east-2.rds.amazonaws.com -u openemr -p openemr < mysqldumpfile-2021-06-06-2130h

where xxxxxxxrrrrrssssss.ffaass33gg22.us-east-2.rds.amazonaws.com is the new (duplicate) RDS instance

  1. Log into the duplicate OpenEMR system by opening the IP address of the duplicate Ec2 instance in a web browser.

Next, attempt to upgrade the duplicate system from 5.0.2p5 to 6.0.0:
16. Download the distribution of OpenEMR 6.0.0 Linux onto your personal computer and then transfer it to the duplicate EC2 (or do so directly if you know how using wget, for example).

  1. Backup the sqlconf.php file from the duplicate OpenEMR system:
    sudo docker cp standard_openemr_1:/var/www/localhost/htdocs/openemr/sites/default/sqlconf.php .

  2. Expand the 6.0.0 distribution onto the duplicate EC2 system file system rather than the openemr directory to have a bit more control of what happens in this step.
    tar -pxvzf openemr-6.0.0.tar.gz

  3. Copy the 6.0.0. distribution into the docker, overwriting the 5.0.2p5 files that are there.

  4. Copy the 5.0.2p5 documents into the openemr/sites/default/documents directory:
    sudo docker cp openemr-v502p5/sites/default/documents/
    standard_openemr_1:/var/www/localhost/htdocs/openemr/sites/default/documents
    (see where they went and, if necessary, move them out of a subdirectory)

  5. Make sure all the permissions are correct. Generally, go to the openemr site and execute chwon -R apache:apache * to change the owner. You may also need to change some read, write, and execute permissions.

  6. Open openemr/sql_upgrade.php (http://yourEC2ip/openemr/sql_upgrade.php) in web browser, choose 5.0.2 and click ‘Upgrade Database’

  7. Configure optional settings in openemr/sites/default/config.php files

  8. Log into the duplicate system and make sure all the data migrated properly by checking several charts and appointments. You may want to let your users test it out and make sure it is working.

  9. At some point, install 6.0.0 patch 1.

  10. Delete the sql_patch.php and sql_upgrade.php files … unless you like to leave scripts like that lying around for anyone to execute :wink:

Like I said above. I got this to work once. Still trying to reproduce it a second time. Feel free to correct any errors.

–RBL

Cannot access documents after 6.0 upgrade