MariaDB Cloud Standard Setup Issue - Table contact already exists

Goal


My goal is to run the latest OpenEMR Cloud Standard Edition (running 7.0.2) on AWS (us-east-2) with MariaDB instead of MySQL.

The issue


I am able to download the original CloudFormation template (attached) for the MySQL Cloud Standard Edition setup & successfully deploy through the AWS console.

  • (Success - https://<MY_EC2_PUBLIC_IP_ADDRESS> opens the OpenEMR login page)

Then I modify the template to use MariaDB (attached). I am then able to deploy this (as a separate CloudFormation Stack with all separate resources) but when I try to access https://<MY_EC2_PUBLIC_IP_ADDRESS>, I get Unable to connect

  • My only changes to the template are:
    • Changing "Engine": "MySQL","Engine": "MariaDB",
    • Replacing all occurrences of 8.0.36 (the MySQL version #) with 10.11.6 (what I believe should be the correct MariaDB version number
      • I got 10.11.6 from:
        • Checking for the most up-to-date MariaDB version supported by my OpenEMR Version: OpenEMR_Downloads
          • Compatible with … MariaDB versions 10.3 - 11.1
        • Cross-reference with most up-to-date version of MariaDB supported by AWS RDS: MariaDB on Amazon RDS versions - Amazon Relational Database Service
          • Running aws rds describe-db-engine-versions --default-only --engine mariadb --engine-version 10.11 --region us-east-2 --query "*[].{Engine:Engine,EngineVersion:EngineVersion}" --output text returns: mariadb 10.11.6

original_openemr_with_mysql.template (40.8 KB)

mariadb_updated_openemr.template (40.8 KB)

Digging Deeper

I SSH into my EC2 instance to see whats wrong.

root@ip-...:~# docker ps -a
CONTAINER ID   IMAGE                   COMMAND          CREATED       STATUS       PORTS                                                                      NAMES
0b0d58961d0d   openemr/openemr:7.0.2   "./openemr.sh"   2 hours ago   Up 2 hours   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   standard_openemr_1

Running docker logs standard_openemr_1 shows that this error is being thrown:

Couldn't set up. Any of these reasons could be what's wrong:
 - You didn't spin up a MySQL container or connect your OpenEMR container to a mysql instance
 - MySQL is still starting up and wasn't ready for connection yet
 - The Mysql credentials were incorrect
mkdir: can't create directory '/tmp/php-file-cache': File exists
PHP Fatal error:  Uncaught mysqli_sql_exception: Table 'contact' already exists in /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php:1349
Stack trace:
#0 /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php(1349): mysqli_query()
#1 /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php(369): Installer->execute_sql()
#2 /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php(317): Installer->load_file()
#3 /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php(1278): Installer->load_dumpfiles()
#4 /var/www/localhost/htdocs/openemr/auto_configure.php(41): Installer->quick_install()
#5 {main}
  thrown in /var/www/localhost/htdocs/openemr/library/classes/Installer.class.php on line 1349

Checking the options

You didn’t spin up a MySQL container or connect your OpenEMR container to a mysql instance

root@ip-...:~# docker exec -it 0b0d58961d0d /bin/sh
/var/www/localhost/htdocs/openemr # mysql -h $MYSQL_HOST -u openemr -p"$MYSQL_ROOT_PASS" -e "SELECT 1"
+---+
| 1 |
+---+
| 1 |
+---+

I believe this confirms the OpenEMR container can access the DB running in RDS

While we’re here, lets also check that contacts table:

/var/www/localhost/htdocs/openemr # mysql -h $MYSQL_HOST -u openemr -p"$MYSQL_ROOT_PASS" -e "USE openemr; SELECT * FROM contact;"
/var/www/localhost/htdocs/openemr # 

I believe this shows the contact table exists but contains no records

MySQL is still starting up and wasn’t ready for connection yet

Just to be sure, I re-deployed & waited 2 hours before touching anything & got the same result

root@ip-...:~# docker ps -a
CONTAINER ID   IMAGE                   COMMAND          CREATED       STATUS       PORTS                                                                      NAMES
0b0d58961d0d   openemr/openemr:7.0.2   "./openemr.sh"   2 hours ago   Up 2 hours   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   standard_openemr_1

The Mysql credentials were incorrect

I double-checked echo $MYSQL_HOST & echo $MYSQL_ROOT_PASS, both are correct as far as I can tell

Lets try to find some logs

root@ip-...:~# docker exec -it 0b0d58961d0d /bin/sh
/var/www/localhost/htdocs/openemr # cd /var/www/logs
/var/www/logs # ls -a
.   ..

Hmm, no logs

Alright, lets check RDS in AWS Console…

I see an interesting difference between the database connections graphs between the working MySQL instance and the MariaDB instance:

MySQL instance’s connections:

image

MariaDB instance’s connections:

image

Help Me Obi-Wan Kenobi, You’re My Only Hope


At this point I’m pretty stumped, does anyone have any ideas what the issue might be or more things to try?

Any and all assistance is greatly appreciated!

Hmm.

Okay, let’s talk about openemr-devops/packages/standard at master · openemr/openemr-devops · GitHub a little. In here you’ll find all the magic that goes into Standard, and I want to pay particular attention to the ami folder, which is where we prepare the machine image that Standard runs.

There’s two parts that matter – ami-build.sh, which I run when I set the image up and send it off to Amazon, and you’ll notice it’s actually sorta thin – there isn’t much there but Docker getting installed, the OpenEMR images pulled (but not launched), and some tools that get set up for later.

ami-configure.sh is more interesting. This will get called after the image I made launches in your environment, by the CFT (see line 710 of the JSON in the cfn directory) and it’s really what handles getting the instance ready for the CFT’s exported yaml (747) to fire.

That’s… the end of the magic, honestly. ami-build is my setup, ami-configure is your setup (picking up some answers from your answers in CloudFormation) and then in that created environment CloudFormation generates the final docker-compose and launches it.

So what’s going wrong here? I’m not sure. There are various logs you can check – docker knows its own output log history and the scripts do redirect their own output to places you can go and check – but what we’re seeing here sounds like a configuration that gets part of the way through and then fails and doesn’t understand that it either worked or won’t ever work.

And when I ask myself how that could happen just because you switched to MariaDB, I think the problem might be that my stuff is injecting Amazon’s MySQL signing certificate (ami-configure, line 41), and if the install is inconsistent about pulling this cert in and verifying it, that could lead to this circumstance, where we started the install and got a lot of the way through it before we noticed we had a bad cert and crashed, and then the next time through we… notice that we tried to start an install and now the tables exist and we can’t proceed, so we crash.

How can you solve that? Good question. I’d say that modifying cloud-setup.sh in the CFT might be your best option – let my install run, let my install put the mysql-ca cert where it’s supposed to go, and then either inject Amazon’s MariaDB cert or just delete the erroneous MySQL cert after my setup runs and before Docker cranks up and OpenEMR has a chance to try to use it.

If that’s not the source of the issue, then I’d recommend you launch a regular old Ubuntu instance, launch MariaDB in RDS, handwrite a docker-compose, and verify that that can launch OK – it will (or it’d better), and that’ll help us start to narrow down what’s going wrong.

(Edit: On a second look that might just be a global RDS cert MariaDB shares, so I dunno.)

1 Like

You could also leave the entire environment alone and skip ami-configure’s up until you can log into the instance and run it live yourself.

1 Like

No dice unfortunately @jesdynf

What I did

  • Re-created the stack after removing the "/root/openemr-devops/packages/standard/ami/ami-configure.sh\n", line from the cloudformation template
  • ssh into EC2 instance
  • Ran /root/openemr-devops/packages/standard/ami/ami-configure.sh
  • Checked docker logs
  • Same Couldn't set up. Any of these reasons could be what's wrong: error as before :frowning:

Bonus

I also did a bit more poking around and at least in terms of logs, I found only one difference between the non-working MariaDB instance vs the workingk MySQL instance

The non-working MariaDB instance had a ./var/www/localhost/htdocs/openemr/ccdaservice/node_modules/is-arrayish/yarn-error.log file that ended with:

Trace: 
  Error: Command failed.
  Exit code: 1
  Command: sh
  Arguments: -c mocha --require coffeescript/register
  Directory: /src/qix-/node-is-arrayish
  Output:
  
      at ProcessTermError.MessageError (/Users/junon/.yarn/lib/cli.js:186:110)
      at new ProcessTermError (/Users/junon/.yarn/lib/cli.js:226:113)
      at ChildProcess.<anonymous> (/Users/junon/.yarn/lib/cli.js:30281:17)
      at ChildProcess.emit (events.js:127:13)
      at maybeClose (internal/child_process.js:933:16)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)

The working MySQL instance did not have this file

Any other log sources?

I currently know of the following sources to check for logs:

  • /var/log/openemr-configure.log
  • /tmp/cloud-setup.log
  • /var/www/localhost/htdocs/openemr/ccdaservice/node_modules/is-arrayish/yarn-error.log

(or at least that is all I’ve found using tools like find -type f -name "*.log")

Are there any other log files I should be checking or any other ideas to make progress?

Thanks!