Backup Error

bradymiller wrote on Wednesday, January 13, 2010:

Joe,

Let me know how it goes. As I recall this tested fine in the past. Am wondering if this will have to do with the xampp-openemr environment (perhaps a permission issue in the temp folder or soemthing).

-brady

ideaman911 wrote on Thursday, January 14, 2010:

Brady, Tony & Bill (plus Houston);

I just completed the internal backup per the expected process using an XP Home desktop where I am using the 3.2 pre-release version, after loading the latest (1.7.3) Xampp.  It indicated as per expected for the dumps.  But I found some problems whose causes may be evident as well:

In the globals.php file, the Windows temp directory is only for below a certain value for PHP version.  Otherwise, it says it will use the location defined in PHP, (which is NOT a good explanation for newbies especially, since they will have no idea what that means).  I am pretty certain the latest Xampp (1.7.3) exceeds that.  So, as expected, the files were not in C:/windows/temp  (please note the slash direction convention I have been following here which differs from the windows standard, while the php.ini file uses the Windows left lean convention, which may contribute to the issue.  It says to use the C:\xampp\tmp location.

There I found five files with timestamps which could be correct, though they are all within the same minute, even though I purposely used my real data in the 3.2 testing, so would have expected the first backup to take longer, which it seemed to do.  Anyway, all five files are named Ib7x.tmp where x seems to sequence as 7, 8, 9, A and B.  But ALL have ZERO KB file size.  I do not believe that is likely to contain the same data which takes 97 MB total for all the backups by my batch method.

So I would conclude that Houston is not smoking whacky, and the internal backup is probably not functioning correctly.  My starting point would be the php.ini file, but the design and usability guy in me says make the globals.php not care what version, and instruct users to make that directory what THEY want, as my personal suggestion would be to write to another drive at the very least.  Backing to the drive which is at risk seems pointless, no?

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

bradymiller wrote on Friday, January 15, 2010:

Joe,
The backup script deletes the backup files from tmp after it’s completed, so don’t expect you to find anything in tmp directory (we felt it was unsecure to leave these in the tmp directory, so they get deleted) The only goal of this script is on-demand backup and download (from what you said, I think it worked? right?).
thanks,
Brady

ideaman911 wrote on Friday, January 15, 2010:

Brady;

Wow.  THAT is convoluted logic.  I guess my only reply would be that I have no idea where the file, or whatever it is supposed to look like, was sent.  If that is the case, my point is that it is by no means clear what I am supposed to go use to supply a re-install if my hard drive suddenly goes tango uniform.  And, of course, that also means I have no idea STILL how I would use whatever that file or files is or are, once I find out where they are supposed to be.

So, assuming the two “Dump” statements Houston noted are supposed to look like that, and they are supposed to leave a blank set of five files once they are finished with whatever magic they perform, then yes, the process occurred.

But if my drive crashed tomorrow, I would still be dead meat, and still have no idea about:

1)  Where is whatever I am supposed to have created as a “backup” dataset?

2)  If I don’t know it, how should any other newbie know it?

3)  Even when you tell me where it is, the “script” noted above to reload is ONLY a Linux tool.  How do Windows users reload their data in a new install?  PLEASE remember, the purpose of the backup is to recover from disaster, not necessarily as an upgrade, so we cannot assume they will recall the password they assigned originally.

4)  When that is all sorted out, if it is supposed to delete from whatever temp folder it uses anyway, the temp location only matters to a user if they are somehow trying to watch process.  I suspect most have better things to do with their time.  What I would bet they would prefer to have would be a concrete location they could go to which would allow them to have a batch file, totally in the background and without further need for human action, to make copies of that dataset on multiple remote storage devices.  And those anal ones like me would prefer I didn’t even have to select Admin - Backup to get there.  MY preference would be a place in the Globals.phpo file which has a default location, but which can be made anything the user wants, including any drive they can access.

So, in summary, I still do not see how that works for Windows users yet.  I can, and have, described my alternative process for securely having recovery ability using the Xcopy procedure.  I have yet to see a complete instruction set describing the process for using the Backup button and then recovering using the result therefrom.

Like the CEO at Black & Decker famously said; “Customers don’t want quarter inch drills.  They want quarter inch HOLES”.

Let’s keep that in mind.  The “goal of the script” is to provide users protection from disaster, NOT to show off our programmer skills.  So, NO, it does not fulfill my expectation any more than it did for Houston.  Sorry.   But let’s FIX that.

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

bradymiller wrote on Friday, January 15, 2010:

hey,

this is Houston’s bug from above:
“tried the backup. However, a file never starts downloading like it does on the online demo.”

his bug is not:
“provide me with a comprehensive OpenEMR backup solution”

To be more clear, the adminsitraiton->backup script:
—should send a backup file for the user to download. (did this work for you?)

The script currently does not:
—backup to disk (the tmp directory is only used for backup file processing, and is not where any backup files are stored)

The scope of this script  is not meant to be the “backup mechanism” that all responsible users should have in place like your xcopy method, for example. The goal is to just provide a multi-platform way within OpenEMR to create a manual backup that an admin user can download. (please don’t make me repeat this again… :slight_smile: )

The backup file is currently packaged via a standard mechanism, that can be unpackaged into a mysql file and the entire openemr web directory. To manual restore it is straighforward; would be nice to have docs on how to do this(you volunteering?); would be even nicer to have a windows compatible restore script (you volunteering?).

-brady

anonymous wrote on Friday, January 15, 2010:

All,

For Windows we have our clients run a .bat file on a scheduled basis that stops the services, copies the directory, and then starts the services. In a nut-shell here is how to do it:

1) Create a notepad file in the c:\xampp folder called “openemr_backup.bat” - (make sure you named it .bat and not .bat.txt)
2) Edit the file and place in the following (This tells the computer to stop the services then copy only the files that are newer, then start the services back up):

NET STOP “mysql”
NET STOP “Apache2.2”
XCOPY c:\xampp s:\openemr_backup /D /E /C /H /K /Y /R /O /I
NET START “mysql”
NET START “Apache2.2”

3) Run the following command in a dos prompt (this creates a windows scheduled task to run the bat file everyday at 2:00am):

schtasks /create /tn “OpenEMR Backup” /tr c:\xampp\openemr_backup.bat /sc daily /st 02:00:00

NOTE: Please edit the above as need. For instance the destination of where to copy the files to is set to s:\openemr_backup and that may not be the case in your situation.

ideaman911 wrote on Friday, January 15, 2010:

Brady;

First, my apologies, as I have been mis-understanding what was supposed to be happening.  I thought there would be a file placed somewhere which would be the tar package of my dataset.  But I had not actually used the online demo, since it works in Linux.  So I ran the two processes side by side - the online vs the 3.2 pre-release, as well as the 3.1.0, in Windows XP.

But there is our disconnect.  I had not realized that it was expected to send the file to my browser, for ME to redirect to a folder.  Now that I have my head out of the nether regions, I can safely tell you that in the case of both of the Windows installs, Backup does not send anything to the browser.  So it is NOT working.

And I apologize for seeming to ask you to spoon feed me.  That was not my intent.  And if you understand my expectation from above, then you can see why I was frustrated, having spent all that time looking for a file which did not exist.

If I can get someone who understands how the backup is supposed to work within Windows to fix it, and then I can learn what is needed to restore that in Windows, I will happily make that restore process as automated as possible and publish the batch file details to achieve that as well as the HowTo to do the entire process.  But I am dead in the water until I can see the process work.  FYI; I did try it as well in my 3.1.0 “real” which has the Patch#5 (I think that actually makes it 3.1.5, but what do I know ;-), and had exactly the same results as Houston had - only the two “Dumping” indicators, but NOT the “Will now send download” indicator.  But since I did get and save the file from the Online Demo called “emr-backup.tar” (replace the dash with an underscore, which will not display here), I will play with it to see what I can learn about the restore in Windows.

Tony & Bill;  Any idea what has changed since you had it working as i understood it (though not actually tried it myself)?  It does not appear to be working in Windows in either the xampp-windows 3.1.0 distribution (I believe it is Xampp 1.7.1) nor in 3.2.0 pre-release with the manual Xampp 1.7.3 install.

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

bradymiller wrote on Friday, January 15, 2010:

Joe (and Clucena),
Thanks for testing it out. Sounds like there is a real bug looming in the windows backup script.  Don’t worry about trying to fix the script (I’ll look into it, since I’ve already spent the time deciphering this script it in the past; bug is likely my doing anyways, although it tested well in my windows environment in the past). Would be great if both you and clucena would post your backup methods/scripts here on the wiki (and restore docs would be great too):
http://www.openmedsoftware.org/wiki/OpenEMR_Backup_Tools
-brady

bradymiller wrote on Saturday, January 16, 2010:

hey,

here’s my initial test results:

1)Installed OpenEMR-XAMPP-3.1.0 package
2)Installed Patch #5
3)Tested the Administration->Backup script. It worked (tested both IE and Firefox).

Very odd. Any ideas?

-brady

bradymiller wrote on Saturday, January 16, 2010:

Joe,

I think I figured it out. In php.ini need to increase the max upload size:
file: C:\xampp\apache\bin\php.ini
setting: upload_max_filesize = 64M  (since only set at 64MB, why Houston and yours not working. Need to make this very large, as you could imagine)

Try increasing this and see if yours now works. (note that you need to restart the apache service/application for the new settings to take effect)

-brady

bradymiller wrote on Saturday, January 16, 2010:

Joe,

I’m wrong. It’s not the upload cap. It’s considered a download. I’m still at a loss for why it works on my 3.1.0 with patch5 and not yours.

-brady

bradymiller wrote on Saturday, January 16, 2010:

Joe,
Instead of the upload parameter php.ini, increase the max_execution_time parameter to see if that works.
-brady

ideaman911 wrote on Saturday, January 16, 2010:

Brady;

No Go.  I increased in both php.ini and php5.ini to 360 each for max_execution_time with no change - still only the two “Dumping” indicators, just about a minute apart.  How much data do you have?  I read there is a limit, not defined at all, for the phpadmin to handle “large databases”.  Not sure what Houston’s looks like, but I am purposely working with “real” data set, which takes 600 MB for the Xampp directory and subfolders.  If you are testing without that much data, you might be squeeking thru?

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

ideaman911 wrote on Saturday, January 16, 2010:

Brady et al;

While we continue to test, I will prepare a HowTo using the Xcopy, and post all the instructions and batch file to the Wiki so Windows users are not without protection.  I will also test the suggestion by Chris Lucena, though I note that can only work if the Apache and MySQL are services (which conforms to his Xampp-openemr 3.1.0 one-click distribution), and not if they are manually started and stopped which is how I have done it from the start as a means to save resources.  That said, I do recognize that most users are NOT doing development, and mostly dedicate that one hardware piece anyway, so it’s not likely to be a problem, and I’ll writeup the process to start the services for all users.  I’ll let all know what I find, but I intend to PROVE whether it will prevent a write while the backup occurs, and what happens to those who try.  That is supposed to be the MySQL standard for the mysqldump.  So if that works per Chris’ suggestion, I think I would suggest we make the internal Backup NOT EVEN AVAILABLE in a Windows install (just display “Feature Not Available - Use Batch XCOPY Process”) or simply do nothing as a worst case.  There is an old saying - “Never try to teach a pig to sing; it wastes your time and it merely annoys the pig”.  Maybe OpenEMR in Windows ain’t Pavirotti.

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

jlegowik wrote on Tuesday, January 19, 2010:

As a dumb user I appreciate the work you folks are doing.  Does anyone have a relatively short (under 200 page) description as to how to combine information from more than one site.  I am going to try to run this on computers from five offices and will need to combine the information from all of them into one (massive?) file.  I am a novice at Linux and use the Windows version (in part because the office staff changes so often that training in Linux would be impossible) and am trying desperately to learn MySQL…  If you think the answer is beneath you, don’t be afraid to tell me I need to play in another sand box.  Thanks… John Legowik M.D.  407-247-011  legowik@comcast.net   (by the way I think you folks doing this are real heros.  Being able to afford this type of software for the small struggling doctor is horrendous.  Not only that but the availability in the developing countries is really special… you guys need halos…

ideaman911 wrote on Wednesday, January 20, 2010:

Dr Legowik;

I am a little confused about your intent.  If you have multiple places, perhaps you should consider running virtual servers for each as needed.  Doing so has the added advantage of enabling you to make them virtual servers which can readily, and quickly, move between physical servers.  VMWare is one such vendor, and they make them for most OS’.

But your note says “one file”.  Do you really want to try to combine a backup from five different physical sites?  It would seem so much smarter to keep them separated.  Or, if they are into the idea, maybe combining them as a HIE system from a single server for all might be an alternative?  To do that, you would simply use a single OpenEMR install for all, with all the data available to all the providers.  Make certain you really want to do that first.

I am not sure you need to learn MySQL so much as you need to learn how and what OpenEMR does, then figure how best to use that for your practice(s).  But I encourage you, and I am pretty certain those of us on whom you bestow halos (probably NOT on me, BTW :wink: appreciate it.

Joe Holzer    Idea Man    315-622-9241     im@holzerent.com  or  joe.im0602x@gmail.com
http://www.holzerent.com  or  http://www.EMRofCNY.com

anonymous wrote on Saturday, February 06, 2010:

OK, so I’ve copied the xxamp folder to a remote location - I at least have one recent backup.

I tried the one-click install of OpenEMR on a different windows machine and the backup works just fine.  So, here is what I want to do -  I want to re-install OpenEMR on the computer in which the backup does not work.  However, I will need to restore all of the old database files. 

Can anyone give me instruction on how to  do this?  Again, my backup feature doesn’t work, so I won’t have a .tar file to use.  I guess I’ll just have to copy and paste some folders?

First, to practice and to ensure I don’t ruin the master database, I will take the backedup database and restore it on my other computer with a fresh install and patch of the latest OpenEMR.

Thanks again for all the help!

tmccormi wrote on Saturday, February 06, 2010:

The restore process is easy.  Once you have saved a backup file to somewhere besides the computer you are running on …
0) Make sure you have XAMMP installed and running with MYSQLD
1) Untar the file openemr.tar.gz into your installed XAMPP openemr/ folder, this put the programs and the documents and the config changes back in.
2) Untar the file openemr.sql.tar.gz to some temp location
3) from the command line start mysql command and open your openemr database
typically, mysql -user=openemr -host=localhost openemr
4) enter your database password
5) at the mysql prompt type
> source openemr.sql;  (from the temp location)
That will load all the data back in.
quit mysql

That’s it.

-Tony

anonymous wrote on Sunday, February 07, 2010:

Thanks Tony.  That will certainly come in handy after I do the fresh install.  However, right now, my backup is currently just a copy of the xxamp folder.  

Basically, what I need to know how to do is merge two databases into one -  need to do a fresh install and add patients from an old database.

Thanks!

tmccormi wrote on Monday, February 08, 2010:

Merging two databases is beyond the scope and quite difficult.   If you want to contract for some professional help on that contact me or one of the other commercial support developers of this list.
-Tony