OpenEMR 4.1 is very slow

system · September 26, 2011, 2:58pm

tsvas wrote on Monday, September 26, 2011:

OpenEMR 4.1 is very slow. 4.0 used to be very fast. Any suggestions/ideas how can we make it as fast as 4.0?

system · September 26, 2011, 4:16pm

sunsetsystems wrote on Monday, September 26, 2011:

This type of problem can usually be traced to a missing index in the database. The solution normally goes like this:

1. Identify a simple operation that is slow.
2. Check the code to see what SQL query is associated with it.
3. Check the associated table(s) via Administration -> Other -> Database to see what indexes already exist.
4. Add a suitable index, if appropriate.

Obviously if you do not have programming/database experience, you’ll want some help with steps 2-4.

Rod
www.sunsetsystems.com

system · September 26, 2011, 4:29pm

jcahn2 wrote on Monday, September 26, 2011:

You may see something useful in this thread:

https://sourceforge.net/projects/openemr/forums/forum/202504/topic/4676445

Jack Cahn MD
OEMR board

system · September 26, 2011, 5:13pm

tsvas wrote on Monday, September 26, 2011:

Thanks Rod and Jack for clues.

All the front-end operations are slow. When I ran “SHOW PROCESSLIST”, I see lot of hanging threads. I restarted the server 5 mins ago, and these are the hanging threads.

Id User Host db Command Time State Info
Kill 2 openemr localhost:49166 openemr Sleep 185 NULL
Kill 13 openemr localhost:49177 openemr Sleep 355 NULL
Kill 14 openemr localhost:49178 openemr Sleep 305 NULL
Kill 15 openemr localhost:49179 openemr Sleep 306 NULL
Kill 16 openemr localhost:49180 openemr Sleep 305 NULL
Kill 17 openemr localhost:49181 openemr Sleep 0 NULL
Kill 49 openemr localhost:49214 openemr Query 0 NULL SHOW PROCESSLIST

I had similar problem with one of my public high traffic website which has all MyISAM tables (converted from InnoDB). Then I removed all indexes!!! It retained the fastness. Now that high traffic website (more than 10 million hits per month) runs without indexes. I am lazy enough to read difference between MyISAM and InnoDB, and indexes structure.

If something is with database indexes and performance, let me do further research and update you.

system · September 26, 2011, 5:26pm

sunsetsystems wrote on Monday, September 26, 2011:

It’s normal for Apache to run multiple processes so as to respond more quickly to requests, so I wouldn’t worry about that.

Rod
www.sunsetsystems.com

system · September 26, 2011, 6:56pm

bradymiller wrote on Monday, September 26, 2011:

Hi tsvas,

For us to get a better grasp:
Can you post your server specs and also what browser the client is using?
Can you post the number of rows in your patient_data table (ie. the number of patients in the database)?
Also, as discussed in the thread posted by Jack, consider installing Firebird, which is a nice tool to see response times of different scripts (including ajax calls) to help localize the problem/slowdown.

-brady

system · September 26, 2011, 6:58pm

bradymiller wrote on Monday, September 26, 2011:

Typo above:
I meant FIREBUG, not firebird : http://getfirebug.com/

-brady

system · September 26, 2011, 7:32pm

tsvas wrote on Monday, September 26, 2011:

Brady, these are server specs:

OS: Windows Home Premium Edition
H/W: Pentium Quad Core
XAMPP: 1.7
RAM: 8GB
HDD: 1TB

Number of patients: 3500

Client browser: Firefox 6.x

system · September 27, 2011, 1:30am

bradymiller wrote on Tuesday, September 27, 2011:

Hi tsvas,

This is a nice size database. The first thing that will be useful is downloading/installing FIREBUG: http://getfirebug.com/
Note it’s a firefox extension so is very easy to install with firefox.

Then open up firebug and go to OpenEMR and to a patient summary screen and you should see something like the following after viewing the patient summary screen:
Firebug is cool. Here’s what I get for the patient summary screen:
POST http://192.168.1.143/openemr/interface/patient_file/summary/pnotes_fragment.php 200 OK 165ms
GET http://192.168.1.143/openemr/interface/patient_file/summary/pnotes_fragment.php 200 OK 598ms
GET http://192.168.1.143/openemr/interface/patient_file/summary/disc_fragment.php 200 OK 630ms
POST http://192.168.1.143/openemr/interface/patient_file/summary/clinical_reminders_fragment.php 200 OK 611ms
GET http://192.168.1.143/openemr/interface/patient_file/summary/patient_reminders_fragment.php 200 OK 677ms
GET http://192.168.1.143/openemr/interface/patient_file/summary/vitals_fragment.php 200 OK 701ms

The numbers to the right are time it took to get the scripts, so am really interested to see what you’re getting there.

Also, Jeremy Wallace has suggested placing indexes in pid and pubpid in the patient_data table speeds things up.

The main goal here is figuring out the performance bottlenecks and going after them; just not clear where the bottlenecks are yet.

thanks,
-brady

system · September 27, 2011, 2:21am

tsvas wrote on Tuesday, September 27, 2011:

Brady, updated one of remote client machine’s Firefox 6.0.2 plugins to:

Adobe Acrobat Adobe PDF Plug-In For Firefox and Netscape 10.1.1 10.1.1.33
Silverlight Plug-In 4.0.60531.0
Java Deployment Toolkit 6.0.270.7 NPRuntime Script Plug-in Library for Java(TM) Deploy 1.6.0.27
Java(TM) Platform SE 6 U27 Next Generation Java Plug-in 1.6.0_27 for Mozilla browsers 1.6.0.27
Shockwave Flash 10.3 r183 10.3.183.10

After plugins update, speed increased little. Not sure if this is due to plugins upgrade or offpeak time. Tomorrow I will test it more closely with Firebug on different user machines.

Also will monitor with MySQL Workbench and check which query is creating bottleneck… will update you by the end of tomorrow.

system · September 27, 2011, 3:42am

rpl121 wrote on Tuesday, September 27, 2011:

>
>Also, Jeremy Wallace has suggested placing indexes in pid and pubpid in the patient_data table speeds things up.
>

I did this on my 4.1 installation and it did not make any subjective difference in the slowness of response. During the delay, I see a small notation in the lower left corner of my broser indicating javascript() errors. The error notice (which does not contain specifics) goes away when the operation finally finishes. This is when I click on the patient name to view demographics and other information in that tree. Looks more like errors than just slow operations.

Ronald Leemhuis

system · September 27, 2011, 3:59am

rpl121 wrote on Tuesday, September 27, 2011:

>Wow, there are some pretty impressive users out there. I also am 1 day into
>using 4.1 and so far would be interested in #1 and #4 above. My Immunizations
>seemed to work better by making a custom list under admin/lists with an attached
>CVS code, turned on in globals, but I did run into the August flu shot problem

When I get some time to figure out git, I’ll contribute my fixes, but in the meantime here are the basics for the August flu shot problem:

edit file /var/www/openemr/library/clinical_rules.php

search for Flu season in the file.

Below the place where you find Flu season you will see a few lines of code with 9s in four places. Just make these into 7s or 8s depending on when you want to start receiving notices. Actually, I’m thinking 8s (i.e. August 1) is a good choice.

I like to keep a copy of the unedited file in the same directory in case I mess up - perhaps you could call it clinical_rules.php.original

Let me know how this works for you.

Ronald Leemhuis

system · September 27, 2011, 6:36pm

tsvas wrote on Tuesday, September 27, 2011:

Wanted to post update…

1. Changed database and all tables to utf8_general_ci (to rectify immunization display bug).
2. Changed all tables to MyISAM and did repair and compact of tables - little better speed.
3. Updated Firefox plugins and browser to V7.0 - little more better.
4. Slow at 2 events - (1) Search for a patient (2) Open encounter first time. For the same patient, if we try to open another encounter, it is fast. Also few errors.

The Web Console logging API (console.log, console.info, console.warn, console.error) has been disabled by a script on this page.

GET http://EMRProdServer2/openemr/interface/patient_file/summary/demographics.php?set_pid=317
Error in parsing value for ‘filter’. Declaration dropped. @ http://EMRProdServer2/openemr/interface/themes/style_oemr.css:884
Error in parsing value for ‘background’. Declaration dropped. @ http://EMRProdServer2/openemr/interface/themes/style_oemr.css:885
Unknown pseudo-class or pseudo-element ‘Subjective’. Ruleset ignored due to bad selector. @ http://EMRProdServer2/openemr/interface/themes/style_oemr.css:1032
Error in parsing value for ‘visibility’. Declaration dropped. @ http://EMRProdServer2/openemr/interface/patient_file/summary/demographics.php?set_pid=2317
POST http://EMRProdServer2/openemr/interface/patient_file/summary/stats.php
POST http://EMRProdServer2/openemr/interface/patient_file/summary/clinical_reminders_fragment.php
GET http://EMRProdServer2/openemr/interface/patient_file/summary/pnotes_fragment.php
GET http://EMRProdServer2/openemr/interface/patient_file/summary/disc_fragment.php
GET http://EMRProdServer2/openemr/interface/patient_file/summary/patient_reminders_fragment.php
GET http://EMRProdServer2/openemr/interface/patient_file/summary/vitals_fragment.php

GET http://EMRProdServer2/openemr/interface/main/daemon_frame.php

system · September 27, 2011, 9:29pm

rpl121 wrote on Tuesday, September 27, 2011:

I checked the error log for apache2 in /var/log/apache2/access.log and found a lot of errors during the use of openemr - here are some excerpts from today. Could these be causing slowing of response?

“http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.37 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site= HTTP/1.1” 200 619 “http://localhost/openemr/interface/login/login.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.37 - - “GET /openemr/interface/login_screen.php?error=1&site= HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/login/login.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)”
192.168.1.27 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.27 - - “GET /openemr/interface/login/login_frame.php HTTP/1.1” 200 763 “http://192.168.1.5/openemr/interface/login_screen.php?error=1&site=default” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.27 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.27 - - “GET /openemr/interface/login/login_frame.php HTTP/1.1” 200 763 “http://192.168.1.5/openemr/interface/login_screen.php?error=1&site=default” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.27 - - “GET /openemr/interface/login_screen.php?error=1&site= HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/login/login.php” “Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; IPH 1.1.21.4019; BRI/1)”
192.168.1.37 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)”
192.168.1.42 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “-” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; AskTbX-SD/5.11.3.15590; BO1IE8_v1;ENUS)”
192.168.1.42 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; AskTbX-SD/5.11.3.15590; BO1IE8_v1;ENUS)”
192.168.1.5 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://localhost/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)”
192.168.1.37 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)”
192.168.1.27 - - “GET /openemr/interface/login_screen.php?error=1&site=default HTTP/1.1” 200 619 “http://192.168.1.5/openemr/interface/main/main_title.php” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.27 - - “GET /openemr/interface/login/login_frame.php HTTP/1.1” 200 763 “http://192.168.1.5/openemr/interface/login_screen.php?error=1&site=default” “Mozilla/5.0 (Windows NT 5.1; rv:6.0.2) Gecko/20100101 Firefox/6.0.2”
192.168.1.42 - - “GET /openemr/interface/login_screen.php?error=1&site= HTTP/1.1” 200 619

Ronald Leemhuis

system · September 27, 2011, 10:45pm

rpl121 wrote on Tuesday, September 27, 2011:

In other applications, I’ve seen reference to the java.thread:619 error implied by the codes from the apache access.log above. The results seem to indicate that the connection was ultimately successful (the 200), but that a java.thread:619 issue came up. Googling various other resources suggested to me that this may have a problem with “JVM” whatever that is. In any case, they were finding that computer memory was being used up by JVM. In any case, if we are swapping to hard disk, that could explain why things worked, but slowly. They say the solution is to use -Xss or -Xmx options to reduce memory requirements. I have no idea what this means, but I wonder whether it is relevant to the OpenEMR 4.1 speed problem.

Ronald Leemhuis

system · September 28, 2011, 1:38am

rpl121 wrote on Wednesday, September 28, 2011:

Another possible clue to slow responses in OpenEMR - When I look at /var/log/apache2/error.log I see very frequent and seemingly endless notice level errors about undefined indexes and undefined variables in openemr. Here is an example from the tail end of the error.log file:

   PHP Notice: Undefined index: relogin in /var/www/openemr/interface/login/login.php on line 152, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authUser in /var/www/openemr/library/log.inc on line 483, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authGroup in /var/www/openemr/library/log.inc on line 517, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authUser in /var/www/openemr/library/log.inc on line 483, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authGroup in /var/www/openemr/library/log.inc on line 517, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: use_adldap_auth in /var/www/openemr/interface/login/login.php on line 199, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authUser in /var/www/openemr/library/log.inc on line 483, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authGroup in /var/www/openemr/library/log.inc on line 517, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authUser in /var/www/openemr/library/log.inc on line 483, referer: http://192.168.1.5/openemr/interface/login/login_frame.php
   PHP Notice: Undefined index: authGroup in /var/www/openemr/library/log.inc on line 517, referer: http://192.168.1.5/openemr/interface/login/login_frame.php

Are these errors a problem?

Ronald Leemhuis

system · September 28, 2011, 10:13am

bradymiller wrote on Wednesday, September 28, 2011:

Hi,

Looks like several things going on (ie. multifactorial).

1. Ajax requests on the patient summary seem to all take the same amount of time, which is odd.
-First way to attack this is to try out the most recent jquery library on the patient summary page to see if it handles ajax calls better (the current jquery library used there is rather outdated)
-Then can consider re-ordering them (fastest ones first), however this is not a very sound plan since just because it was sent first (by a millisec) doesn’t mean it will get to the server first)
-If still issues, can consider migrating some of the ajax calls back to the main script.

2. Mysql indexes. Guessing the slow patient search and the opening of encounters are related to this. May want to figure out what mysql queries in these situations are slowing things down (ie. mysql workbench).

3. Processing issues. The Clinical decision engine (clinical reminders and patient reminders) does some serious processing and multiple mysql queries(so potentially related to mysql index issues as 2 above). One option is limiting the number of rules that are active (some rules will be more intensive than other, for example, I’m guessing the diabetes rules (lots of diagnosis code checking) will use more processing time). For the future, I think I have a way worked out (in my head only at this time) to be much more efficient (create a couple cache_rule table(s) that are processed in the background only once daily (unless force a re-process) when open up the patient file); this feature is likely months away though.

Also, I don’t think undefined index’s/variables is a problem (this was also reported in previous versions) and this sensitivity of logging can be turned off in php.ini. Also, the 200 code stuff with the error=1 thing seems pretty infrequent.

So, first good steps may be trying out most recent jquery library in the patient summary page and watching sql queries in the new encoutner and patient search pages.

-brady

system · September 28, 2011, 11:55am

tsvas wrote on Wednesday, September 28, 2011:

Brady, all day yesterday we played with indexes to tables like patient_data. Didn’t improve any speed. Next step is - tuning of programming libraries and programming.

system · September 28, 2011, 12:08pm

yehster wrote on Wednesday, September 28, 2011:

@tsvas
Just a shot in the dark, but do you have a PHP Accelerator installed?
http://en.wikipedia.org/wiki/PHP_accelerator

-Kevin Yeh

system · September 28, 2011, 8:49pm

yehster wrote on Wednesday, September 28, 2011:

Also, I’m noticing that there are times when the ajax calls succeed, but the browser doesn’t actually get updated. (I can view the responses in firebug, but the loading icons stay up).