mdsupport wrote on Friday, June 17, 2016:
Are you rationalizing directory structure? Current zend location is configured as an “interface” requiring an override of all zend defaults. Catch-22 now is that you cannot start new root without reworking existing zend modules. Same goes for composer defaults as well.
robertdown wrote on Friday, June 17, 2016:
Modules will do in the “library” folder. library/Patient.
We can configure zend to play nice for now, although Zend was not configured optimally when initially put in
matthewvita wrote on Wednesday, June 22, 2016:
Robert,
Once I’m done with the website rework project, I will volunteer to help with this.
I appreciate the fact that you started listing out best engineering practices we should introduce to the codebase.
Rod brings up a great point that implementing all of these patterns/changes at once may be too ambitious (huge cross-cutting pull requests, lots of things we can/will break without having existing unit tests will not be fun). Therefore, we should create a roadmap and decompose these work items as much as possible. This will be more approachable and the codebase can evolve with regular pull requests instead of doing everything all at once.
I’m thinking as a first step, we can identify how we plan on separating the existing logic that shouldn’t be in controller into plain old PHP services and deal with overall code organization. What are your thoughts here?
Random thoughts:
Thanks,
Matthew
robertdown wrote on Wednesday, June 22, 2016:
Hi Matthew,
Thanks for reaching out. I am putting a road map together and will post it to the wiki as well as the SF forums. I hope once other developers sees the road map they will warm up to the idea. There is certainly no way this happens in one PR, it will take months of close collaboration to get this done.
Are you familiar with Zend? If so my goal is to create Zend modules for out code. This means mostly plain old PHP services as you said, an ORM object to represent DB results and then a controller (which will likely be the last thing to be implemented. I foresee a lot of model to view direct usage in the initial stages.
Would be great to start jotting this work down in the active projects page
I’ll get this up later today
We need to perform regression tests with each pull request. Don’t know
how to orchestrate that but it seems like a good idea
Unit tests will be key!
I agree that using more of Zend would be nice, but I feel that we may be better off with libraries as the codebase evolves… for example, when we start to talk about separating out view logic from the views so JavaScript can deal with it… Knockout.js would be a better choice than Angular.js since it can be embedded with PHP and isn’t a huge framework. Same thing with picking out an ORM. If we do a good enough job keeping things generic, we can eventually merge everything into a greater framework.
I think Zend is a great framework and we already have it installed. I also prefer Angular.js (especially Angular2 as I’m a GIANT fan of TypeScript). The ORM will be interesting as we are so heavily invested in adodb and don’t see that changing anytime soon.
I like the PHP documentor you mentioned. Looks great.
Thumbs up!
We should look into PHPCS and JSCS for enforcing style that you mentioned (definately a long-term goal)
I agree, PHPCS is great. This is definitely long term because changing code just to fix styling seems counter intuitive and if we automate it we run the risk of breaking things (because of the way the code base is already setup. I suggest moving forward we adhere strictly to a code standard, which I will propose on the forums soon. We already have some general guidelines, but it’d be good to enforce it. a git pre-commit hook would be a good place to automate the process.
mdsupport wrote on Wednesday, June 22, 2016:
We are currently working on Patient Portal. This component was probably added to meet minimum requirements of MU1 certification - display of few tables using interface that now looks dated. Because current application is not flexible, there have been attempts to provide this mandatory functionality with external applications. However exposing full features of an rapidly evolving application through SOAP or any other external interface is difficult. It also does not leverage the core strength of this application - it’s native web interface. As long as network access is secured, we should be able to deploy it for multiple user classes.
Portal also has requirements that could test MVC framework and possible new components. When designing base classes we should build in models that implement security. So current EMR user has access to all patient records, a “patient” user has access to a single patient record and a “group” user can have access to all patient records that belong to a group.
Most frameworks will play nice with components that manage multiple form factors of new display devices. This becomes even more critical for patient portals as majority of use by patients will be mobile.
Zend is nice if we take advantage of its pre-built modules. Some of the examples are logging and authentication. Portal can be a good showcase for the project to see if current methods can be migrated or potentially replaced by builtin alternatives.
Finally, Zend or any other framework will have steep learning curve for developers. So it is important to have good cookie-cutters that can be copied and few lines of logic can be added at a specific place by a novice without requiring much knowledge of underlying framework.
matthewvita wrote on Thursday, June 23, 2016:
Awesome!
I’m sure I can figure out Zend. Can’t be too much different (at a high level) from other MVC web frameworks.
I still think that Knockout.js would make more sense than Angular.js at this time simply because it is much easier to embed in with existing code. Angular is a total rewrite.
bradymiller wrote on Thursday, June 23, 2016:
Hi,
This thread sounds very promising.
Also important to consider future proofing and avoiding potential dead ends. Since the project barely keeps head above water from a resource perspective, important to ensure don’t place the project in a position down the road where a huge refactor is needed (ie. when PHP, Zend, or other frameworks brought in require a version upgrade and/or dubious licensing like Sencha).
-brady
OpenEMR
wakie87 wrote on Thursday, June 23, 2016:
Just to keep this conversation going, I have done a lot of reading and research on the best way to go about dealing with all of this.
One of the best things I found was an opensource cms called Concrete5 which has undergone a huge refactor to meet the PSR standards and go from a legacy php application to a modern one. Some of the importants points I pulled from reading through their process:
I think we need to do a bit more research looking at other php projects and how they accomplished refactors and the processes they used and adapt them to our project.
Also Matt & Robert, would love to maybe organise a chat online? maybe skype (or something) sometime to hash out some more ideas. Being in Australia puts me on the opposit timezone to you guys 
drop me an email scott@npclinics dot com dot au
Scott
robertdown wrote on Thursday, June 23, 2016:
I could not disagree more with the idea of making OpenEMR our own framework. We should absolutely not be recreating the wheel. A framework like Zend handles all the low-level stuff that has nothing to do with OpenEMR (Routing, Configuration, HTTP, DB Adapters) – let the people who are great at making frameworks make the frameworks and the people who are great at making EMR software make EMR software.
I think Doctrine is better than adodb but we have to handle our resources carefully. I don’t see this change happening anytime soon.
Backwards compatability should end with OpenEMR 5. Semantic versioning suggests that a major release breaks compatability (so OpenEMR 4 to OpenEMR5). See semantic versioning. Unfortunately the MU certification process gets weird with regards to version numbers. I think we are approaching the need to start planning version 5.
Scott, I saw your message, I’ll reach out soon–it’s been a very busy 2 weeks for me in Orlando
sunsetsystems wrote on Thursday, June 23, 2016:
Not seeing much agreement yet about frameworks and third party tools. Can we at least agree that refactoring the data access into an object model is a good first step regardless of what framework is used? Among other things this means all SQL statements will be built and executed in a class somewhere. Again, this can be done with just PHP and contributed piecemeal to the code base. Modular code is a huge plus in moving forward, no matter the direction.
Note there’s about half a million lines of PHP code in the project. This will not happen quickly and I don’t want to see us overcommitting to something that holds up other work. Sadly, experience shows that developers often lose interest when work gets repetitive with the finish line on the distant horizon.
osverdlov wrote on Thursday, June 23, 2016:
Before starting working on porting everything to Concrete5 or Zend or Flow or whatever framework we like… Won’t you want think about some architecture first? What are the main parts of OpenEMR and how they are connected?
MVC is not an universal recipe. For example, traditional ORMs available in frameworks need some work before they will handle LBF forms. Time-based series data needs some thinking.
I suggest to look at the architecture before we even run a “New Project”
CRM aspect can be improved. Data storage layer can be improved. Calendar and scheduling.
mdsupport wrote on Friday, June 24, 2016:
From past project management experiences I know refactoring is problematic for anyone looking at new code. There is no satisfaction / pride of ownership in making change to something that the new developer thinks is wrong in its entirety and the target is not to change any output. In other words spend 2 hours of analysis and developement to get 1+1=2 when the current code is doing it anyway!
sunsetsystems wrote on Friday, June 24, 2016:
Yes there’s that. It’s a special kind of person who will take pride doing well what most of the world won’t notice. Money would help too.
matthewvita wrote on Friday, June 24, 2016:
I definately can appreciate your 1+1=2 example having worked with old code that was seemingly written without care. However, attitude is everything with projects like this.
matthewvita wrote on Friday, June 24, 2016:
I also want to leave a note that we may find that refactoring without sufficient unit test coverage is going to be very rough.
I fear that adding this coverage after the fact (but before the refactor) will be too painful especially since the tests will effectively need to be rewritten after the refactor. Perhaps the best ROI would be to build up the existing Selenium tests. Here are the benefits in my mind:
Thoughts? Would Selenium tests be #1 item on the refactor roadmap?
NOTE: It is worth mentioning that I do think unit tests and automated tests are the way to go after the clean up… just probably not very practical to have decent unit test coverage after the fact with the current codebase.
robertdown wrote on Friday, June 24, 2016:
Zend is a framework. Zend is already in the codebase. Zend should be used as the framework.
Step 1 - Create a PHP class that represents a model in the database. This can be a pure PHP class (It actually should be a pure PHP class, completely agnostic of implementation.
Step 2 - Create a PHP class (aka a “service”) that handles all the interactions with a model. All the DB queries, transformations, etc go here.
I’m doing this now for the Patient model, I’ll have a commit soon.
bradymiller wrote on Saturday, June 25, 2016:
Hi Matthew,
Selenium testing would be great. Matrix just committed a starting point for this in the main codebase and they were hoping the community would expand on it:
https://github.com/openemr/openemr/tree/master/Tests/selenium
Note it is a very basic test at this point and just tests logging in and adding a new patient. I tested it and it’s pretty cool (note the tests fails about 50% on google chrome, guessing for a timeout issue, and always passed on firefox).
I just bookmarked the project here:
http://www.open-emr.org/wiki/index.php/Active_Projects#Testing
Having extensive selenium testing would be very helpful no matter what happens regarding MVC.
-brady
OpenEMR
bradymiller wrote on Saturday, June 25, 2016:
Hi,
The Patient model will likely not be very straightforward since there are only several mandatory hard-coded database mappings due to the ability to heavily customize it (for example, some places may create 3 middle names, rename fields, add other fields, remove fields, etc.). I am not saying a Patient model is a bad idea, but it will likely get a bit complicated
I like the idea of producing a roadmap. Perhaps if there was a roadmap, then the steps to get there would be more clear(directory structure, etc.) and then we could all work towards that direction. Feel free to place the roadmap on the current roadmap wiki page listed here(or another page if wish):
http://www.open-emr.org/wiki/index.php/OpenEMR_Wiki_Home_Page#Roadmaps
-brady
OpenEMR
matthewvita wrote on Saturday, June 25, 2016:
@Brady: Awesome, thanks for the link. I think having the automated test coverage will help with the refactor. As I mentioned before, we’ll be able to use them during the refactor and continue to maintain them long after the refactor. Good stuff.
@Robert: Once I’m done with the website rework project, I can work on the Selenium tests in tandem with the refactoring efforts. Is this okay?