PHP Async Library

OpenEMR Development
,
#1

Calling all project leads:
@sjpadgett
@stephenwaite
@brady.miller
@robert.down
@adunsulag

This message concerns the Telehealth module operation. @adunsulag is included because he is the architect of the module along with @sjpadgett .

Currently in the module if the email is not configured properly meaning that they have credentials in the config for mail service but the mail service does not actually send mail. The send mail silently fails in the background. But in the UI the user sees nothing but the prompt that the telehealth can’t start right now. Nothing indicates that the email failed.
I thought it would be great if the mail attempt ran separately from the telehealth UI presentation. It would be nice to run the TelehealthParticipantInvitationMailerService after the telehealth UI came up. I started looking into if PHP had an Async feature like JavaScript. It does! GitHub - spatie/async: Easily run code asynchronously

The second reason that I am writing this is that the module is not built to have or manage its own dependencies. The module depends on the dependencies from the host codebase. That means that the async needs to go into the core and not just the module. I wanted to ask the opinion of the lead project lead of what are the chances of getting this package into the core.

The package has been around for 6 years and seems to have active support. 2k+ stars. This could help with other situations where one operation can be executed first and completed before launching another.
@mdsupport I like your thoughts also. Are you already using this library?
Best!

#2

Wouldn’t it make sense to move mail operations to be stored off in a queue and use a background processor to process mail notifications in the queue? I’d use the database a queue storage. Technically another option would be to use the linux server’s sendmail which can be configured this way, but not every OpenEMR will have sendmail installed.

We could mitigate this in the Telehealth module by just ignoring failures and not trickling it up, but most users need the launch email. Another option would be to trickle up the email failure message to the UX.

My concern about a php async process is handling retries if the SMTP operation is only temporarily unavailable, or handling any kind of error process.

Also this particular library says that it is a wrapper around the php extension pcntl which specifically discourages its use in a webserver environment in the extension instructions:
https://www.php.net/manual/en/intro.pcntl.php

Process Control support in PHP implements the Unix style of process creation, program execution, signal handling and process termination. Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.

#3

I like the background email idea :slight_smile:

#4

Also interestingly, I also looked into the pcntl to solve the email issue i was trying to solve (didn’t want bad actors to figure out if a account name actually existed by measuring the time since sending the emails was lagging when to many logins attempts for a user). Turns out pcntl will not work on windows so ended going with the email queue instead.

#5

@juggernautsei, usually there should be ability to send immediate confirmation and also deferred/scheduled notifications. The later requires unattended process. The standard codebase tries to emulate cronjob but that requires at least one active session to exist. So suggest the following -

  1. For immediate notification use javascript to initiate the process as a promise.
  2. For scheduled notification(s) like reminders, use a cli script and make sure it is ready tor running as a cronjob.

You may even be able to leverage email reminder script for both.

#6

There may be an issue here. Anything that establishes a background process that services a queue works great at one worker node but may fall apart in a cluster.

#7

I’m assuming whatever is done for the queuing solution would need to lock the table record before processing any record in an idempotent way to avoid contention issues on other cluster nodes. I prefer just using something like SQS if something will be run in a cluster but not everyone will be running on AWS.

#8

I want to thank you all for joining in this conversation.

#9

Please avoid custom implementation, use a dedicate library like ecotone.
It solve all th above problems (deduplication, error handling, retrys, guaranteed delivery, concurrency and more)

#10

I really appreciate your work, Openemr is a great system that is constantly developing.
I just wanted to ask (I had already seen it in the previous patch). What it is and how it works the New Background Service to Manage Outgoing Emails.
Thank you so much.
Luis

OpenEMR 7.0.2 has just been released!
#11

hi @luisuriarte ,

Look in codebase for where MyMailer::emailServiceQueue is used (just in 2 places at this time and there is a even cooler function MyMailer::emailServiceQueueTemplatedEmail that can incorporate custom templates). When it is used, it places the email in a queue rather than sending it right out. And the background service (Reports->Services->Background Services-Email Service) is the service that emails out the queue every 2 minutes. So, it’s more of a developer feature (the hope is that we migrate all other email sending in codebase to this and that new features that need email sending also use this). The benefit of this is that it will not hold up the main script until the email is sent out (there can be significant delays sometimes), and was actually added by necessity for security reasons since these time delays could be telling on login failures etc.

3 Likes