How to scale OpenEMR installations in a data center to meet growing demand?

OpenEMR Development
#1

Hello Everyone

We are facing an exciting challenge that I’m sure many of you can relate to – the need to scale our OpenEMR installations in a data center to meet growing demand. Our healthcare facility has been experiencing a significant increase in patient data, and we’re looking for guidance on how to efficiently scale our OpenEMR infrastructure to ensure optimal performance and data management.

As our patient numbers continue to grow, it’s crucial that our OpenEMR system remains reliable, responsive, and secure. We are reaching out to the OpenEMR community for insights, best practices, and recommendations on how to approach this scalability issue effectively.

Here are some specific questions:

Infrastructure Scaling: What are the best practices for expanding our server infrastructure in a data center to accommodate increased traffic and data storage requirements?

Load Balancing: How can we implement load balancing to distribute the traffic evenly across multiple servers and prevent bottlenecks?

Database Management: Are there recommended strategies for optimizing our database to handle a larger volume of patient records efficiently?

Backup and Disaster Recovery: What are the best approaches to ensure data integrity, backup, and recovery in a scaled OpenEMR setup?

Please share your thoughts, resources, or relevant experiences in this discussion. Let’s work together to ensure that OpenEMR

Thank you in advanced

2 Likes
#2

So you’ve given no indication of how you are running your current OpenEMR setup. What kind of load are you targeting. What are your uptime requirements 2 9’s, 5 9’s, etc. Disaster recovery time windows, etc.

What legal regulations are you under for backup and disaster recovery. How long are your data retention requirements? Are you under US HIPAA requirements?

Your questions are interesting, but without knowing what goals you are targeting, the answer is always going to be ‘it depends’.

OpenEMR’s database layer as it is currently built is going to work best scaling vertically until you have to scale it horizontally from a performance perspective. Any kind of read replica scaling has to happen at the database and proxy layer as there is no application logic support for read proxies.

OpenEMR has a kubernetes module you can use for load balancing for traffic distribution, just be aware that there’s currently a shared drive bottleneck as the database configuration has to be loaded from the shared drive on each request. I have mixed feelings on it all as I think sticky sessions with a something like elastic beanstalk or something comparable and a round robin load balancer is going to perform better than the current kubernetes option due to the shared drive contention issue.

Use CouchDB for your document storage to scale that.

Anyways those are a few thoughts. You may need to hire a consultant to help you with these issues as this is a particular area of expertise that I’ve only seen a few people in the community have and you’re going to get pretty generic answers without revealing your particular target metrics.

1 Like
#3

This is a topic of interest, as we have sites that are growing beyond the scope of what a single pair of application/database servers can support and still maintain an adequate level of performance from the end user perspective. Sooner or later, there’s only so many queries that you can force through before you reach a hardware I/O limit that bottlenecks the entire system.
One thing we’re prototyping right now, is changing the database back end from “a server” , to a Galera cluster of 3 or more MariaDB nodes, and then distributing loads across them, e.g., one node is clinical, the 2nd handles reporting/billing requests, the 3rd does interfacing and data automation for communications with other systems, and so on.
As was pointed out in the above post, there’s no support for multiple DB targets or load balancing baked into the application itself, so it either has to be handled some other way via the underlying infrastructure, or new functionality will have to be added to OpenEMR directly, or maybe plugged in as a module. One of the most difficult things is trying to mock up and test an environment that’s intended to support hundreds of users, hundreds of thousands of patient accounts, millions of stored documents, and also must provide a corresponding patient portal exposed to the public.

#4

There are many, many interesting projects to grow OpenEMR! This one is certainly at the top of a growing list. However what many developers tend to forget what the projects original audience was and still is mainly. That is a small startup clinic getting a start inexpensively or underserved communities in need of a translated EMR. It’s great to here of your success and continued push for supporting your growth using OpenEMR.
Fortunately we have someone like Brady that has been dedicated to the project for at least 15 years who has pushed for certifications along with all us other maintainers, past and present, which truly helps us advance OpenEMR with projects such as FHIR, CCDA and all the other ONC requirements.
We still have other requirement coming up!(Insert ad here)
That being said, we currently have at our disposal many very good engineers willing to take on all these challenges to plan and implement such things that advance our lifecycle, but we just don’t have the funding necessary.
It mystifies me that from the thousands of installations we have and all the vendors that make a living from OpenEMR we have to struggle getting financial support.
This is not pointed at you @Penguin8R but just an opportunity for me to vent a little as a maintainer.
(End ad)

After all that, I think this would be a great topic for our weekly Saturday call that generally all of us maintainers attend and is open for all topics and folks. OpenEMR Weekly Conference Call Details - OpenEMR Project Wiki

Thanks for the great topic. I’m sure it will be useful to others and in the end, maybe us.

BTW: My current specialty is run on sentences!:slight_smile: