Data Importer

deschel wrote on Wednesday, January 30, 2013:

We are nearly finished with our Data Importer for OpenEMR and will be posting it in the next 1-2 weeks.  Just need to iron out a couple of bugs and do a couple of tweaks.  This is a feature that people have requested for quite some time.

In the Topic, “OpenEMR and Zend”, Rod wrote:

David, if you want to do the importer with Zend I think it could at least go into the “contrib” directory.  Just make sure it’s separately installable, and I suggest making sure it works with ZF as distributed with Ubuntu to simplify installation in that case.

If the importer becomes truly compelling, we may decide to endorse Zend or someone might convert it to some other model.  There are always options, and we like to encourage contributions.

Rod
www.sunsetsystems.com

I do not understand why Rod suggested placing it in the contrib directory and why he said that it should be separately installable.  I do not necessarily feel that this is appropriate.  Also, because there are different versions of Zend, using a separately/pre-installed version will make debugging a nightmare, and it may not work on a lot of systems.  Additionally, what if someone is using a different OS from Ubuntu.  I’m not using Ubuntu currently to serve OpenEMR.

Additionally, the Data Importer should not be an ancillary feature.  I think that it is importand to have it as a core feature-one that is part of the normal installation, just as PHPMyAdmin is.

The plan is to add a leftnav menu item at “Administration->Other” just under “Database” and label it as “Data Import”.

We used the Zend framework and will be adding Zend to the library folder, to go alongside the numerous libraries that are there.  I have had several people contact me relating to the Zend post that were extremely supportive of using it.  One core OpenEMR developer stated that they are already using Zend for many of the features that they have programmed and are using it in their hosted solution. 

I know that Zend is overkill for this particular project.  However, it is being used more as a proof of concept.  We are planning to utilize Zend in most of my future projects.  So, the installation of Zend as part of this project serves as a foundation for our future ones.

So, can I get a suggestion of a more appropriate directory to place the data importer into?

David Eschelbacher

sunsetsystems wrote on Wednesday, January 30, 2013:

Hi David,

My request to make it separately installable comes from the fact that there is not yet a consensus among the developers to endorse use of Zend for the project.  It’s not just a library, it also adds yet another programming methodology to the mix.

If Zend is accepted for the project, you’ll still need a more compelling and specific argument for adding a particular version of the Zend library to the OpenEMR distribution.  PHP has many different versions, yet we do not include that.  The main reason for adding a library is when it’s difficult for users to find or install it.

My mention of Ubuntu was in the context of a suggestion, recognizing Ubuntu as an example platform that offers Zend as an installation option.  If your code works with Zend as provided with recent versions of Ubuntu, then there’s a pretty good chance it will work with other platforms also.

Best,

Rod
www.sunsetsystems.com

deschel wrote on Wednesday, January 30, 2013:

I don’t think that using Zend Framework works in the same way as using PHP libraries.

By this rationale, why didn’t you keep jQuery external?  Or, why didn’t you keep PHPMyAdmin external?

I think that using ZendFramework externally will make it too difficult to use and configure.  Then, the effort needed to use the data importer will be too great and people won’t be using it.  Or, the technically able people will use it but the non-technically able people won’t.

ZendFramework is a major framework and programming paradigm that won’t be going away.  It is powerful and I think there are other developers interested in using it.

I’m not paying to create the data importer or my other projects so that no one will use them.  I do not want to create a barrier for people to use it.  I want it integrated into OpenEMR without creating new things for people to install. 

Why don’t we try to add data importer and add the Zend library to the developer version (when it is ready to be added)?  Then, if it does not work out, we can always remove the Zend library and rewrite the data importer to act on its own.  If Zend Framework proves to be useful and other people use it in their projects, then we keep it.

What is wrong with trying this?

David

donelewis wrote on Wednesday, January 30, 2013:

I built one in PHP that connects to MS SQL Server or to MS Access if that can help.  The tricky part is inserting rows into patient_data table so that they are counted in the MU calculator.  I found another way to beat that :slight_smile: Don

bradymiller wrote on Wednesday, January 30, 2013:

Hi David,

The best way to attack this issue is to have your developer bring in two commits into github (based on openemr repo):
1. One basically including only the Zend library in the place you want
2. Your code

Then we can begin to really see what makes the most sense and if it’s tough to support Zend when not included(which we can begin to test by not bringing in the Zend package commit). I also think ZH Healthcare is implementing a lab module on top of Zend, so it’s very nice timing.

-brady
OpenEMR

zhhealthcare wrote on Wednesday, January 30, 2013:

David
If I may add my two cents worth here.
• We are actively developing in Zend. 
• We found that people may have objections to adding it directly to OpenEMR.
• We created a module installer so that people can create modules in Zend or PHP.
• Anyone can develop a module and install it via the module installer. 
• It can remain separate and can become part of the core OpenEMR if enough people accept it.
• We plan to build out zend modules for:
Module Installer
Lab Information system
Calendar
Demographics/problems/history et cetera
Encounters
• This we plan to contribute over the next several months.
• People can continue using the old Calendar or the new Calendar.  Older installations with legacy issues will fall in this category.  In other words there will be two parallel calendars and people can enable whichever one they choose.
• So this way we can show the community if ours is worth the trouble rather than forcing it on them.
• The key to all this is the module installer.  So if you are interested I will have my programmers share the module installer with you so that you can create everything you want to in a modular structure.
• Also if you want to we can share separate areas of development we have listed up there.
I think this can be a good compromise and a practical way to go forward.
Shameem

zhhealthcare wrote on Wednesday, January 30, 2013:

Before anyone responds, we are still finishing up the Module Installer and will be submitted sooner than later.

bradymiller wrote on Wednesday, January 30, 2013:

hi,

Also a quick aside on the contrib directory. Lets avoid placing anything there that has been integrated into OpenEMR. One of the security recommendations basically recommends swiping this entire directory (except for icd9/icd10/snomed/rxnorm) after installation since there is legacy code in there:
http://www.open-emr.org/wiki/index.php/Securing_OpenEMR#OpenEMR

-brady
OpenEMR

manxmen wrote on Monday, November 25, 2013:

Hi Guys,

Am currently looking for a way to import xforms/xml forms into OpenEMR. I noted a script in contribs called “import_mi2xml.php” but can not see any example xml files from Medics. Then I came across this thread that references a “Data Importer”, but I can not seem to find any other references to the utility.

Any help with import_mi2xml.php and/or Data Importer would be welcome.

-Terry
South Africa

tmccormi wrote on Monday, November 25, 2013:

The data structure format is at the bottom of the program in what is called XPath format. Each path element is an XML set.

Example:
    patient/familyinformation/father
is
    <patient>
         <familyinformation>
           <father>
              DATA_GOES_HERE
           </father>
              ... (other sets) ...
         </familyinformation>
             ... (other sets) ...
       </patient>