Statistical Analisys Module

aguirrel wrote on Monday, January 18, 2010:

Hi!

I think a Statistical Analisys would be a really nice addon to the application.  The concept is very simple, track the evolution of the patiente in a period of time, join all data (of all patientes) and generate reports.

This sould be very usefull in rehabilitation environment, or to investigate a new method for example.

I think that the first track sould be over patiences problems.  The idea is to generate a “problem tree”, for example “Leg->Left->pain”, “Leg->right->pain”, “leg->left->knee->pain”, “leg->left->strain”, so then you may get statictics about: Leg evolutions, right legs evolutions, leg pain evolutions, leg strains evolutions, etc.

I’m not a medical resercher so, any comment will be very welcome.

I’m php programmer (some projects) so I may code this module, but an global idea and a system mentoring will be very welcome.

Thanks!

Best regards,

Luis

aguirrel wrote on Tuesday, January 19, 2010:

Any comment? Yes? No? I may write some documentation at wiki, but I really would like to know qualified opinions about that. :slight_smile:

bradymiller wrote on Thursday, January 21, 2010:

hey,

There’s a bunch of report stuff being built for “meaningful use”; may be useful for you. Check out here:
http://www.openmedsoftware.org/wiki/CMS_Quality_Reporting

-brady

aguirrel wrote on Thursday, January 21, 2010:

Hi brady!

I have check the CMS Quality Reporting project, but seems different that I need.  Reading my post again, perhaps I was not fully explained.

The final data that I need is for example:
* 30% of the patients that had legs problems resolved this issue within 3 month.
* 70% of the patients that had legs problems reported a positive evolution within 2 weeks.
* 10% of the patients that had some pain, have reported a positive evolution inmediatly.

All of that using charts like http://teethgrinder.co.uk/open-flash-chart-2/ or http://www.aditus.nu/jpgraph/ .

I thinks that this statictics will be really usefull to rehabilitation centers.

May you give me some feedback about the real utility of this?

Thanks a lot,

Best regards,

Luis

bradymiller wrote on Friday, January 22, 2010:

hey,

What do users think of this idea?

check out aguirrel’s feature request here:
http://sourceforge.net/tracker/?func=detail&aid=2931514&group_id=60081&atid=493004

and check out aguirrel’s code to get this function working here:
https://sourceforge.net/tracker/?func=detail&aid=2932030&group_id=60081&atid=1245239

So, the goal is to place entries in the log to follow the outcome of a problem/issue over time. This is a cool concept, and the added code isn’t much. What do others think? Useful? A better way (is any MU stuff gonna do this)?

-brady

aguirrel wrote on Friday, January 22, 2010:

Hi Brad,

The code at https://sourceforge.net/tracker/?func=detail&aid=2932030&group_id=60081&atid=1245239 is a very simple log table to problems, alergies, etc.  This is only a part of the whole idea.  When I did that, i did not have the whole idea completed, so, I have misses the “problem tree” that I describe in first posts.  But before start to code I wanted to discuss it with the community as you seggested at code review.

Thanks a lot,

Best regards,

Luis

bradymiller wrote on Friday, January 22, 2010:

hi,

I was just trying to connect the dots of your feature request, code, and your idea, which I think is cool(I’m also trying to make the point that you are a developer, and have already developed some code towards the project).

I think the idea is nifty, because it seems it would create the option to make fully customized reports/graphs over time of patient data(especially the data that is only represented for one time in our database). My advice is to make it as modular/simple as possible at beginning so we can code review it and get a better idea of what your doing.

-brady

sunsetsystems wrote on Monday, January 25, 2010:

Another way to preserve intermediate issue versions would be to just keep them in the “lists” table.  This already happens in the patient_history table, although nobody has ever created a report to take advantage of that.

Rod
www.sunsetsystems.com

aguirrel wrote on Wednesday, February 03, 2010:

Hi all,

I have created a new code revision at https://sourceforge.net/tracker/?func=detail&aid=2945397&group_id=60081&atid=1245239 .  This is a first aproximation but I think is a good start point.

Thanks a lot!

Best regards,

Luis

Here a copy&paste from the code revision:

Regarding wish http://sourceforge.net/tracker/?func=detail&aid=2931514&group_id=60081&atid=493004 here is the first code.

I have tested it paching a fresh CVS openemr and worked without problems.

Installation:
* Download wish_2931514_all.tar.gz from http://trustgreeninvests.com/cvs/patch/wish_2931514_all.tar.gz (sorry, SF does not to attach let files greater than 256k)
* Uncompress it at root openemr directory
* Uncompress wish_2931514.tar.gz (sorry, I cannot use CVS diff -uN to add new files due I have not write permitions on repository)
* execute patch -p0 < wish_2931514.patch

How it works
* Anytime you save a patient issue, will be saved at list_log too, but only neccesary data (date, patient id, lists id and outcome)

Reports
* Go to Reports->Statistics->New Query
* Select one or more problems
* Select “Days to analize” (This is, from first time that the patient talk about the issue, how long in time you will check for outcome.  For example, a patiente has a leg problem, first month the problem increses and the second month the problems was solved.  If you select 40 days, this patient will count as “incresed”, but if you select 70 days, this patient will count as “solved”). Note: 0 will be “does not care”.
* Select “Days from last patience encounter” (This is to filter patients that does not come to the clinic anymore or have abandoned the medical treatment). Note: 0 will be “does not care”.

bradymiller wrote on Thursday, February 04, 2010:

hey,

Thanks for making it very easy to test.

-Good idea to get a graphical package into openemr; php-ofc looks nice and has compatible license.
-What version of php-ofc library are you using?
-Instead of creating a new table could do as Rod suggest above and save intermediate entries of lists (may require a little work, but should be feasible). Would be good for logging also, and allow you to maintain more information for your analysis. I think this would involve adding a ‘parent’ and ‘delete’ column to table ‘lists’ along with modifications of some code in several places to ensure it works as before while keeping every saved instance in the database (any pitfalls here you can think of Rod??) . Then could avoid creating a new table which wouldn’t benefit many other users, and then your graphing/statistics feature could be used for many more data elements and thus applicable to many more users.
-instead of saving graphing code to ofc, save it in the library directory (keep it in the sub-directories)

statistics_new_query.php
-place all the includes (both php and javascript) near the file at the top
-no need for include_once(“srcdir/sql.inc”); (already included in globals.php)
-use word ‘Statistics’ instead of ‘Statisticals’
-split up line 52 into xl() strings like line 61
-use word ‘Analyze’ instead of ‘Analize’
-consider number validation of your input via javascript functions
-line 93 and 97, consider ‘ternary if statement’ to shorten ( detailed here: http://www.scottklarr.com/topic/76/ternary-php-short-hand-ifelse-statement/ )
-line 110 and 117, place the onclick calls into a javascript function call instead, and remember to include a call to top.restoreSession() (see this link for details : http://www.openmedsoftware.org/wiki/OpenEMR_System_Architecture#PHP_Sessions_and_Browser_Windows)

overall, a very nice starting point,
-brady

aguirrel wrote on Thursday, February 04, 2010:

Hi brady!

Many thanks for your revision! :slight_smile:

The OFC version is  “Version 2 Lug Wyrm Charmer (28th, July 2009)”.

About database, IMHO, I think that we may do the following:
* Create another column, named lid (list id)
* When new issue is created a new row is created as always, taking care to save with lid = NULL
* When an issue is updated, update the issue row and add a new one, with this new information and use:
  date column to save the timestamp (the updated row did not update date column) AND use lid to save parent id.
* So, if you need to see the original table, you may filter adding lid=NULL to the where clausure.

Sorry, I did not understand “-instead of saving graphing code to ofc, save it in the library directory (keep it in the sub-directories)” may you explain it better?

I will change the code! Thanks a lot for your advices! :slight_smile:

Best regards,

Luis

bradymiller wrote on Friday, February 05, 2010:

hi,

Sorry for confusion. Simply meant to put the ofc directory within the library directory instead of the openemr directory.

We’re not gonna support php4 anymore, so might as well use the php5 specific library php5-ofc from the ofc package.

Your database mechanism sounds good fo now, can always be optimized/changed around as get it to work with the current algorithms that query the lists table (for now, I’d just focus getting your new lists table scheme to work on the add_edit_issue.php page as you’re doing, then can expand it to rest of code).

As this project matures, can import the ofc library into the cvs here on sourceforge in a way to allow simple ofc library upgrading in the future (unless somebody objects to adding a graphical package or has another better graphics library to use).

-brady

aguirrel wrote on Monday, February 08, 2010:

Hi brady,

I have tested modifing the table as I have described before and it seems that I have to touch (as we know) some application code.

I have been talking about how to solve this transactional information with some database-designers (PeopleSoft and Point of sales) that works with me in the company and how they solve it at similar situations.

Both recomended me to do a new table to add all the transactional and historical data.  The new table sould be the same as the original including a reference to the original table id.

The principal advantaje is that you do not mix and overload with historical data with the day-by-day data.  The second advantaje in this case is the there is not necessary to change any issue-related code and this particular feature is absolutly issolated.  IMHO (In my Humble Opinion) we sould take this path.

I have moved the ofc directory to library and I have migrated all php4 code to php5-ofc and is working well. :slight_smile:

Please, let me know your how would you like to solve the table problem (same table o a different one) so I may continue improving the reports :slight_smile:

Thanks in advance,

Best regards,

Luis

bradymiller wrote on Tuesday, February 09, 2010:

hey,

Agreed that it would be some work for the simple effect your trying to get. Just hard for me to rationalize placing a new table in the codebase that is so user specific. A possible compromise is this:

1) Build your code to instead to show the report from the smoking status column in the history_data table; note this table stores a new entry whenever it’s modified (would be ideal for lists to do this also, but agree it would be some work). Then your graphing library and a simple mechanism will be introduced into the codebase that is usable for the general userbase.
2) In your local openemr, then simply need to add your custom table and insert your sql query on your custom code instead of the smoking status code.
3) The cool thing about this is the smoking status mechanism in the future could potentially be modified to be more generic and cover any element in any form
4) And in the future, somebody could build into the lists table a logging feature (likely needed for logging purposed anyways, so hopefully will get done at some point).

let me know what you think,
brady

aguirrel wrote on Tuesday, February 09, 2010:

Hi brady!

First of all, thanks for be so patient, I really appreciate it. :slight_smile:

I really like your idea about to have a generic way to implement this and be as extensive as we can and I understand that the table I propose is very user specific.

The global idea about this module, is to trac all time-dependant information in the system, starting with problems outcome and extend it to all interest statistics.

I have been investigating how another open source projects do that, I have checked “Trac” application (http://trac.edgewall.org/) and gives me a similar idea that you propose if you are agree.

They use the following table to track ticket changes, I use it to generate a gantt:

(SQLITE3 Code)
CREATE TABLE ticket_change (
    ticket integer,
    time integer,
    author text,
    field text,
    oldvalue text,
    newvalue text,
    UNIQUE (ticket,time,field)
);

So, I think a generic solution may be create a new table but this will take in consider all logging features.  The table is very simple:

create table history_log {
date timestamp,
source_name varchar(255), (should be the table name at this case, but is generic)
field_name varchar(255), (should be row name at this case, but is generic)
source_id integer, (should be the row id)
old_value text, (if available at insert moment)
new_value text (new value, obviusly :slight_smile: )
);

please, let me know your opinion,

Thanks in advance,

Best regards,

Luis

bradymiller wrote on Wednesday, February 10, 2010:

hey,

The question of the day is:
Would you be creating a new table if a lists mechanism already existed that saved a row of data every time the save button was hit (at some point this will exist, since needed for logging)?
Not sure why you’d need to create a new table of data, if it already exists in another table, maybe I’m missing something?
For example, if you chose to graph the vitals temperature over date/time form the vitals form, all the information is already there in the database, no need to create a new table, right?
At some point this mechanism can also be plugged into the LBF (layout based forms), which are the main new feature of version 3.2 (releasing next week) : http://www.openmedsoftware.org/wiki/LBV_Forms
Let me know what you think?

Rod, Tony, or anybody else,
Whom is doing the lab module? Would this graphics package be useful for graphing of the data?

-brady

sunsetsystems wrote on Sunday, February 21, 2010:

Hey, just a couple of surface-level comments as I’m busy with other tasks right now.

In general I do think you can take advantage of most table structures containing patient data for the saving of historical versions of that data.  Don’t see why new tables are needed for that.

Yes, lab results will be a gold mine of discrete data and surely a prime candidate for graphing.

Rod
www.sunsetsystems.com