Crowd Source Funding for CQM/AMC

Day-to-Day Users
1

yehster wrote on Wednesday, October 31, 2012:

http://www.indiegogo.com/OpenEMR-CQM-AMC
I am kicking off a campaign on Indiegogo to try and raise funds to support a revamp of the CQM and AMC reporting in OpenEMR.
The link describes the scope of what I hope to accomplish if funded.  And while I appreciate that there are other features/enhancements people would like, what I’ve proposed is what I believe I can reasonably accomplish and have well tested.
Questions/comments/concerns are best discussed openly, so please try to post them here, but if needed, you can contact me via email.
-Kevin Yeh
kevin.y@integralemr.com

2

bradymiller wrote on Thursday, November 01, 2012:

Hi Kevin,

Have you tried the most recent CQM/AMC reporting in the development code (or most recent 4.1.1 patch)? See this commit comment (section below ‘Some Real Examples…’) for details and examples of use. I suggest trying out the example instructions on a sample database of about 10000 or so patients to get a feel for it: http://github.com/openemr/openemr/commit/967af3f100af84f2edd5670faa2bbb88a71fc4b3

-brady
OpenEMR

3

yehster wrote on Thursday, November 01, 2012:

Brady,
I recognize that your most recent changes improve the performance of CQM and AMC. That does make my proposal a little less urgent at first glance.  I suspect that most of those gains are from bypassing the audit engine, and not from the batching process.  A formal profiler analysis would be an interesting academic exercise.  Thanks for your efforts thus far, but I still believe more work would be beneficial to the community as a whole.

However, Tony is still reporting out of memory failure and system slow down with his real datasets of 40K patients.   Other users are reporting incorrect or at least unexpected results, and there is some interest in new reports. If more than one site tried to generate these reports concurrently on a multi-site install, I suspect there would be issues.

The current implementation may be “good enough” for achieving MU Stage 1, but only two groups of Physicians (one in Erie, PA, and one in Colorado) have reported on the forums that they have successfully attested, and they both reported needing to do some “workarounds” of the OpenEMR reporting. There may be others who have succeeded but haven’t told us. However, it is also possible that  the current AMC/CQM is a significant barrier.

The formal testing I’m proposing as part of my project would serve to boost confidence in the correctness results.  Rather than just the qualitative “this is testing well,” I would like to be able to say. Test data of 10,000 patients, complete entries for BP, Height and Weight on 5,000, BP only for 1,000 in 90 day time period 8/1/2012 through 10/29/2012  correctly reports numerator 5,000, denominator 10,000 for rule 302f. All of the other rules would get similar treatment in my testing procedures.

I suspect that the sample database you have used for your testing does not contain encounters and/or clinical data for a large percentage of the 10,000 patients in your runs.  While I can’t guarantee perfection, I will be able to systematically analyze issues and repeatably confirm bug fixes.

The document I am proposing to provide as part of this project may also be useful to folks.

An informal goal of my project is to be able to run the AMC/CQM reports on a daily (or maybe weekly for larger installs) basis without hassle so they can accurately monitor their progress towards meaningful use.

I am trying to make the case that my proposed project would be useful and if the community agrees, then they will respond with their monetary support and comments.  If not, then c’est la vie. The funding I’m requesting would let me focus on these issues and devote the time and energy needed to accomplish the goals I’ve laid out here and on the Indiegogo Campaign page.

-Kevin Yeh, MD
kevin.y@integralemr.com

4

tmccormi wrote on Thursday, November 01, 2012:

All,
  I agree with kevin in that this is an important next step for Stage 2 MU and for our general improvements to OpenEMR which will derive from that (much Iike the report queuing things Brady did can be pulled generally into all reports).

As to the 40000 patent DB, I am not having timeouts or memory issues on that DB any more.  I have been unable to get definitive timing on the run as yet, it’s runs longer that I care to watch and the times being reported by the report queue seen to be CPU time not elapsed times (they are about 1/3 of actual) on my 17000 patient DB.

Tony
www.mi-squared.com / @tonymi2
oemr.org / @OEMR_org

5

bradymiller wrote on Friday, November 02, 2012:

Hi Kevin,

If your goal was to improve the AMC/CQM engine and help out users, I am all in. However, after reading your proposal on indiegogo.com, my main issue here is tunnel vision. The developer seems stuck on the fact that the engine queries patient by patient as “The Issue”, bases this conclusion on many issues that are resolved and thinks a “new” engine needs to be built because of this(its even more odd from my perspective, because the current engine could be optimized to support patient aggregated sql queries). Here are some things to consider:
1. Tony is no longer reporting out of memory failures (that bug has been fixed).
2. Tony has not reported system slow downs with the current code (new code allows setting of NICE levels to avoid this)
3. On my OpenEMR Appliance, I have run 10 reports concurrently with no obvious slow down in OpenEMR (again, via ability to set NICE levels)
4. Since the latest code release in a patch a month or so ago, no users have reported issues.
5. Stating the fact that no users are saying it is working is a false premise in the open source world. If things work, you generally never hear about it.
6. I am guessing Tony’s 40K database will take about an hour or so and will not hold up the server at all (need to await Tony’s confirmation for this). To improve this, without any testing, I can tell you pretty confidently that optimizing the diagnosis/exclusion filtering steps will drop the time from about an hour to just 5 or so minutes(If curious, I’d be glad to point a developer on where to work on this). Then could move onto an optimization that actually aggregates the patients that would likely get the report time to less than a minute.

With the above being said, a project with the theme to improve the AMC/CQM reporting and supporting this for users and provide more detailed documentation is definitely worth doing (do you plan to only show users that pay over $100 this documentation?). What gives me pause are the apparent conclusions drawn on outdated code. Why not just state the goals are to improve performance (goal of a report in less than a minute etc.), provide quality testing and provide support so docs/clinics can better work towards meaningful use? These are things the community can get behind.

-brady
OpenEMR

6

bradymiller wrote on Friday, November 02, 2012:

Hi Tony,

The times are not CPU times, but are the actual time. It is calculated from a timestamp that is placed (in database) when the report starts (date_report token) and a timestamp that is set when the report finishes (date_report_complete token). I see no plausible way that the date_report_complete token will be set early (it is only set after the report is complete). So I think the time you are seeing is the actual time of the report (a manual way to test this is to look in the database entry of the report in report_results and compare the value of the report_results token to the date_report_complete token). If need to further discuss this, lets make another thread on the forum since we are going off topic.

-brady
OpenEMR

7

yehster wrote on Friday, November 02, 2012:

My fundamental goals are as you state.
1. Improve performance
2. QA
3. Documentation

Some of our “conflict” on this issue I think stems from limitations of communication on the internet as well as my own lack of experience with crowd sourced funding and the somewhat political nature of this exercise.

Why not just state the goals are to improve performance (goal of a report in less than a minute etc.), provide quality testing and provide support so docs/clinics can better work towards meaningful use?

Those are the goals, but I did not want to make empty campaign promises without a plan outlining a specific approach.  Plans are of course adaptable.  If we already knew what was going to work then presumably we’d already have a solution.  Although I have listed the performance goal first, the QA/testing piece needs to happen early in the project to provide a baseline for future comparison. The QA piece will also serve as learning exercise for myself so that I can author the promised documentation.

Borrowing from the model of paid journals, my plan with the documentation is to distribute it directly to those who are kind enough to provide funding at higher levels as soon as it is ready.  I will release it generally after an embargo period expires.  I am hoping this approach will provide sufficient incentive for users to contribute at higher level while still maintaining the spirit of openness for the project.

People who cannot/do not want to contribute at that level will be able to get the documentation eventually.  However, keep in mind that this is a fixed funding campaign, so if the overall funding goal isn’t met, then the project isn’t going to happen at all.

I’d rather spend my time implementing instead of campaigning for funds, but I cannot afford to be as altruistic as Brady when it comes to OpenEMR.

8

bradymiller wrote on Friday, November 02, 2012:

Hi Kevin,

So it sounds like we actually are in agreement here (and you were providing meat on the bone for your proposal), since the initial part of your funded project is to test out and analyze the current engine and then to design a plan from there (which may be somewhere on a spectrum from keeping the current engine and improving it to completely throwing it out the engine and redesign it). The fundamental goal to improve, QA and document cqm/AMC reporting for users with 3 months of effort would be very cool, especially since MU really relies on being able to do these things. Just one more question; How long will the embargo period be for the documentation?
(btw, you and others should be paid for your services; in fact, I think this is vital to the project’s success).

-brady
OpenEMR