Upgrading from v4.2.2 to v6.0.0 10% per hours

Y’all need to remember that php has to create the UUID so I image this is not a server issue as much as a php issue.

https://www.open-emr.org/wiki/index.php/FAQ#What_are_the_correct_PHP_settings_.28can_be_found_in_the_php.ini_file.29.3F

hi,
I did some testing on this a couple days ago. Created about 400,000 database entries to see how long uuid creation took. On my laptop was creating 10,000 uuids per minute. Plan to try it out again soon with several million to see if things get slower as the number increases (it shouldn’t, but figured good to check).

btw, strategy to avoid these big hits in future are:

  1. to upgrade to 6.1.0 in future, update uuids again on that upgrade (openemr/6_0_0-to-6_1_0_upgrade.sql at master · openemr/openemr · GitHub)
  2. at and after 6.1.0 is, a service will update them all every 6 hours:
    automated uuid updating service by bradymiller · Pull Request #4273 · openemr/openemr · GitHub

@stephenwaite I am slightly offended by your suggestion that I would forget or now know the php setting. But anybody new to OEMR may not be.
:expressionless:

Yes, that was the second thing I did in standing up this server.

1 Like

What do you have in your laptop? Mine can only produce 150/min.
If you are getting 10,000 records per minute, I need to give you the data to convert. :grinning:

If I did this then I would have to run the build on the rel-600 to get it to work. I was trying to avoid that.

build takes maybe a minute

what’s your innodb buffer pool size?

and the build is fun :), composer’s cool and whooshes by
npm install is maybe 15 secs
npm run build, ok takes like 5 mins

I set it to the value you have given above.

My CPU is an i5 - 4200U 1.60GHz

do you have an ssd in that laptop?

No SSD.

edit:
Seagate Momentus 1 TB 2.5-Inch SATA III 5400 RPM 8MB Cache Hard Drive Bare (ST1000LM024)

My world is not a pretty as yours.

composer 15 secs my aunt fanny! :laughing:

1 Like

2 per second… you might as well manually write them and transcribe them to your computer :slight_smile:

2 Likes

@brady.miller “What’s in your laptop?”
Are you using an old 4.2.2 database in your testing?

@juggernautsei ,
Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8
32 GB
1 TB SSD

Rate on upgrade script (using a timer) appeared to be about 10000 per minuted. To hone this down am using the following script (note this should not be used on a production instance; just meant for testing to evaluate time). I have tried 400000 so far (so does 400000, 200000, 100000, 50000 runs and ending up about 6500 per minute). Am gonna try 1000000 overnight. I’d rec trying just 10000 to see where things stand on your environment:

<?php
// halved on each subsequent run (there are 4 runs)
$records_run_start = 10000;

set_time_limit(0);
$ignoreAuth = true; // no login required
require_once('interface/globals.php');

use OpenEMR\Common\Uuid\UuidRegistry;

foreach (['one', 'two', 'three', 'four'] as $run) {
    if ($run == 'one') {
        $records_this_run = $records_run_start;
    } elseif ($run == 'two') {
        $records_this_run = $records_run_start / 2;
    } elseif ($run == 'three') {
        $records_this_run = $records_run_start / 4;
    } else { // $run == 'four'
        $records_this_run = $records_run_start / 8;
    }
    echo "number of records on this run: " . $records_this_run . "<br>";
    $start = microtime(true);
    for ($i=0; $i<$records_this_run; $i++) {
        sqlStatement("INSERT INTO `procedure_result` (`date`) VALUES (NOW())");
    }
    echo "time (minutes) to complete data insert for run " . $run . ": " . round((microtime(true) - $start) / 60) . "<br>";
    $start = microtime(true);
    (new UuidRegistry(['table_name' => 'procedure_result', 'table_id' => 'procedure_result_id']))->createMissingUuids();
    echo "time (minutes) to complete uuid insert for run " . $run . ": " . round((microtime(true) - $start) / 60) . "<br>";
    echo "uuids per minute for run " . $run . ": " . round($records_this_run / ((microtime(true) - $start) / 60)) . "<br><br>";
}
?>

btw, trying some things out here:

I don’t have anything that powerful. :disappointed_relieved: :open_mouth:

I think, starting with the hardware and work our way back to the program. Because there is no way to achieve the results you are quoting without being on an even hardware field.

Also, I am going to take the advice of @APerez to do incremental updates. So the next steps after buying a newer machine to crunch the numbers are to upgrade from 4.2.2 to 5.0.0 to see if that works. I will report back when I have achieved some of these goals.

hi @juggernautsei, no need to incrementally proceed

@stephenwaite I do understand the upgrade script does that. It applies each update start from the installation number. I get that. But what does not do is tell where it is failing. I need to eliminate and do some stop-gap checking. I may have corrupt data. There are a lot of views in this database that could be interfering with the upgrade process. So, that I why I am taking this approach. If the data can’t be updated from 4.2.2 to 5.0.0. Then I have a far bigger problem than what we are talking about. :no_mouth:

the script will tell you where it fails in the bottom pane

MySQL views, if at all, would affect database version conversion of system tables - nothing to do with upgrade/data conversion unless standard emr tables have been replaced with the views.

1 Like