Backup.php that works in windows posted

tmccormi wrote on Thursday, November 12, 2009:

My gut feeling is that if an install does not support the Archive php module then the user should update the php installation or go back to using the old Backup script.  The implementation is intended to provide flexibility but work the “old” way if left alone.

I totally disagree about the pathing, you should never rely of the environment for pathing to binaries.  It a very big security hole.  It is very simple to hack the path to capture anything you want.

If you or Rod want to tweak it some more that’s cool, but I’m out of money to spend on this particular enhancement.

-Tony

bradymiller wrote on Thursday, November 12, 2009:

hi,

But you’re so close. :slight_smile:

I don’t know much about path security issues, but will take your word there.

So for globals.php, I would then make separate sections for “Linux users only” and “Windows users only” for the OS specific paths and settings(each get their own global). Then can put the () ? : one liners within the backup.php script or at the bottom of the globals.php (where users are not supposed to modify).

Not sure what Rod thinks, but I think the Archive/Tar.php issue obviously needs to be dealt with before placing this in cvs. One of our goals when we make modifications is that is does not hurt current functionality. Quite obviously, a white screen of death on something that previously worked is concerning. There are several ways to deal with this nicely and produce a truly multi-OS backup script (ie. only windows users use the archive/Tar.php method or you check if the archive/Tar.php exist before using method; several well placed if statements should allow support for both the archive/Tar.php and Rod’s old method).

Asking the community to finish this is totally fine, but as you know, there is the real possibililty that it will never be completed.

-brady

tmccormi wrote on Thursday, November 12, 2009:

Thanks Brady,
   I think Bill will finish it up, I think I was just in  a grumpy mood… sorry 'bout that.
-Tony


  

brbill wrote on Thursday, November 12, 2009:

Brady wrote:

*Got white screen death since mandriva(developer appliance) doesn’t have Archive/Tar.php .

The biggest issue seems to be 1; would be a bummer to actually break some of the linux boxes whom don’t have Archine/Tar.php. If this package doesn’t exist I wonder if there is a way top bypass with several ‘if’ commands in linux boxes to use command line instead of Archive/Tar.php.*

Yes, if you don’t have Archive/Tar.php, this won’t work. When I was doing some of the testing of this module prior to using it, I noticed that it was not installed with the older version of PHP that came with OSX 10.4 (3 years old, I have since upgraded to 0SX 10.6). But PEAR is now standard with modern PHP and should be available for usage on any system at this time.

Obviously, it is possible for ANY older PHP to be missing this module. But at some point we have to say, get modern. It’s six of one, half dozen of another. One system doesn’t have tar binary, the other doesn’t have Archive/Tar.php.

Your suggestion for supporting this with different code for Windows and *nix is possible (going back to old tar command for boxen), but doing this will be a much uglier kludge than I can ever be satisfied with.
Rod’s recent updates to backup.php made it quite a bit more complicated. The backup program definitely does not need to be made any more complicated by adding another state model. I urge you to reconsider this objection based on the situation, and possibly peruse how the program executes system commands vs. how it is calling Tar.php.

*1) This is really for Rod. Why are we using the sed stuff to replace things in the dump file. Is this really needed? Because if not, then the Windows users won’t need perl set up.*

I will leave the reason for having this to Rod, because I did not change this functionality. I just needed to make it work in Windows. Since perl comes with the XAMPP stack, I just don’t find this to be a worry point.  Windows installations are going to have perl if they have XAMPP, and they will. Won’t they?

*2) To simplify code and temp directories, why not just have it hard coded always and not check for the php tmp directory? I say this because losing track of these files and placing them in an unsecure area would be bad. so seems more secure and would simplify code to just always direct where it goes from globals.php.*

Earlier in this thread, I asked if this was important, so I guess the answer is yes. It increases the amount of customization that has to be done for each installation.

And I have to ask what is more trouble: reading these conditional lines, whether they be ternary operator or if blocks, or having to decide on their own where they want their temp directories to be?

*3) The globals.php stuff will throw off end users especially the () ? : one liners. Seems like could just have a ‘windows only’ section for mysql/perl pathnames (then backup.php can then decide to use the windows path if windows setting is on and use no path if it’s linux). Then you’ll just be left with one () ? : statement for the tmp directory would could be described with several lines of comments.*

We could have separate windows and *nix sections, as you say, but then we have to decide whether these conditionals, which may be sprinkled throughout globals.php to be associated with other related settings, would seem out of context in such a grouping.

Although I understand that some folks won’t understand the ternary operator, it is supported in C, ruby, perl, PHP, and many other languages, and I found it all over the OEMR source, so it is in common usage. We have users directly modifying actual PHP code in globals.php as part of the installation process, so they have to have some idea of what they are doing.

*4) A minor issue is use of some of your xl() calls in backup.php. Don’t place leading or trailing spaces within the actual string. This gets lost in the translation process and has potential to create redundant english constants for the translators (ie. creating the same string without the trailing space).*

Noted. Will fix.

Bill Cernansky

sunsetsystems wrote on Friday, November 13, 2009:

Thanks Bill.  I have not tested this yet but have taken a quick look at the code.  Much better!

I think it’s OK to introduce this dependency on php-pear.  It should be available to anyone who needs it by using their normal package manager.  However it would be good to detect when it’s not present and display a helpful message instead of the “white screen of death.”

If someone finds that to be a problem, there are a couple of straightforward ways to address it: (1) modify to optionally use tar, or (2) include the needed portions of PEAR into OpenEMR (I think Joomla does that).

Regarding sed, I put in that scan/replace to strip out UTF8 stuff from the dumped tables.  This is so that the restored tables will default to whatever character set and collation are set at the database level.  What prompted me to do this were some “invalid mix of collations” errors with some queries in OpenEMR, after restoring a configuration dump.  This might not be the ideal solution but it solved my problem at the time.

I don’t feel very strongly about the globals.php stuff, but might comment on that later.

Rod 
(http://www.sunsetsystems.com/)

bradymiller wrote on Saturday, November 14, 2009:

hey,

Also got white screen of death in Ubuntu server secondary to missing module.

Hence, this motivated me to make your script compatible with linux boxes that do not have the pear TAR library installed. I also clarified the globals variables to make easier for end users (this should be a configuration file that non-programers can understand). Also fixed the xl() spacing while I did this. I posted the patch in the tracker.

Check it out and critique/revise/modify it as needed. Also, please  put it through your testing flow since I was only able to test it on windows xp and a linux box with the pear tar libraries.

Feel fre to post a revision, and when we’re complete would be nice to get this backwards compatible with the 3.1.0 version patch (will put all the variable from globlas into backup script in the 3.1.0 since can’t include the globals file in the patch)

-brady

bradymiller wrote on Saturday, November 14, 2009:

hey,

As a companion to above posted cvs patch, I also just posted a modified script for 3.1.0 in the tracker. We can’t modify globals.php in our patch releases so put settings within the backup.php(test_backup.txt) file. It works on my initial testing; please (as the cvs patch) put through your testing flows and critique, modify, revise as necessary.

-brady

bradymiller wrote on Tuesday, November 17, 2009:

hey,

Tested Bill’s script (with minor mods by me) in linux and xp, and worked great). Committed it to cvs and to the 3.1.0 branch (it will be included in the next 3.1.0 path likely to be released within the next week). Check out the tracker for more details:

thanks Bill, this was awesome

-brady

  : https://sourceforge.net/tracker/?func=detail&aid=2882124&group_id=60081&atid=493003

brbill wrote on Tuesday, December 01, 2009:

In 3.10’s patch, it looks like there is now an incompatibility between the Windows and non-Windows operation of this program, because it would appear that the .gz compression that non-Windows does with “tar -z” is not enabled in the Windows code (in my code, trying to preserve the status quo, I used a somewhat kludgy $file_to_compress variable to enable this - that variable is mostly expunged, and now in Windows the gzip compression function is never called even when it should be).

That is to say, a backup done with Unix won’t restore in Windows, and vice versa. Not that anyone would necessarily do that, but as it stands, a cross-platform restore would appear to be broken. Right now this is immaterial for restoring in Windows because restore is a bash script. But restoring a Windows backup in Unix would definitely fail with the standard restore procedure.

It also looks to me like some of the 3.2-specific code made its way into the 3.1 patch, and it is confusing at the least. Example: line 104-105 in backup.php v1.5.2.1. That is code from 3.2 that probably shouldn’t be here.

brbill wrote on Tuesday, December 01, 2009:

make that `$file_to_compress` variable in my post earlier. Stupid formatting.

bradymiller wrote on Tuesday, December 01, 2009:

hey,

here’s flow:

1) For sql database backup

Use the funky gz\_compress\_file() function in all situations (i think this is what you are referring to)

2) for directory backup (.tar.gz)

If Archive\_Tar.php exist then use that function (with gz compress method) to tarzip it

if no Archive\_Tar.php then use tar -z

3) for entire package backup (.tar)

If Archive\_Tar.php exist then use that function (with blank compression method) to tar it

if no Archive\_Tar.php then use tar

I’m pretty sure this flow is followed in both the devel tip and patch versions. Let me know if I’m missing something.

Also, by simply hiding the buttons for the new 3.2 specific-code features, this effectively makes it compatible with 3.1 (also for 3.1 did trickery to ensure translations not broken, and put variables in it since can’t put the globals.php file in a patch).

I’d suggest testing it (migrate a windows backup to linux and run restore script). I’m pretty sure i did this on both the cvs version and the 3.1.0 version and it worked… but my memory ain’t so good…

-brady