Analytics and Telemetry data collection module for OpenEMR

OpenEMR Development
, ,
1

Summary

This module would integrate a lightweight JavaScript-based tracking script into the OpenEMR front-end, sending asynchronous event data (like menu click totals and locale details) to a secure API endpoint. The backend stores and processes this data in custom tables, while an admin dashboard provides visualizations and reports for ongoing monitoring. Throughout the design, focus on performance, security, and regulatory compliance to ensure the system is robust and user-friendly. Hopefully this will keep us somewhat sane moving forward.

I truly believe that this feature will greatly help potential implementors decision process, future marketing decisions, future development focus and help our vendors develop successful business plans.

I will be submitting a project pull request soon that initially includes the general module framework with the request for opt in and out module enable logic.

I hope many in the community, including vendors, will help support this effort either by coding contributions and/or financial sponsorship. Please don’t leave me alone in this endeavor!:slight_smile:
Thank you.

Example ask:


<script>
  const trackingUsageHTML = `
<ul>
  <li><strong>${xl("Purpose of Data Collection:")}</strong>
    <ul>
      <li>${xl("To improve the overall user experience within OpenEMR.")}</li>
      <li>${xl("To understand how users interact with the system, helping us optimize and enhance navigation and features.")}</li>
    </ul>
  </li>
  <li><strong>${xl("Data Being Collected:")}</strong>
    <ul>
      <li><strong>${xl("Menu Click Tracking:")}</strong>
        <ul>
          <li>${xl("Counts of how often menu items are clicked.")}</li>
          <li>${xl("Insights into which parts of OpenEMR are most frequently used.")}</li>
        </ul>
      </li>
      <li><strong>${xl("Locale Information:")}</strong>
        <ul>
          <li>${xl("General geographic data including:")}</li>
          <li><strong>${xl("Country")}</strong></li>
          <li><strong>${xl("Region/State")}</strong></li>
          <li><strong>${xl("City")}</strong></li>
        </ul>
      </li>
    </ul>
  </li>
  <li><strong>${xl("Privacy Assurance:")}</strong>
    <ul>
      <li>${xl("Only the minimum necessary data is collected to provide useful insights.")}</li>
      <li>${xl("No personal or sensitive information is recorded beyond general location and usage statistics.")}</li>
      <li>${xl("Data is used solely for analytics and to drive improvements in the application.")}</li>
    </ul>
  </li>
  <li><strong>${xl("User Consent:")}</strong>
    <ul>
      <li>${xl("By enabling this feature, you agree to the collection and analysis of the above usage data.")}</li>
      <li>${xl("You can opt out at any time if you prefer not to share this data.")}</li>
    </ul>
  </li>
</ul>
`;
</scr

Justification:

  • Enhanced User Insights:
    • Telemetry enables us to understand how users interact with OpenEMR by tracking menu click frequencies and other UI interactions.
    • It supports data-driven decisions to improve navigation and feature design.
  • Minimal & Responsible Data Collection:
    • Only non-personal, aggregated data is captured—specifically, usage metrics and basic locale information (country, region/state, city).
    • Data collection is strictly opt-in, ensuring that user privacy is respected.
  • Community-Driven Improvements:
    • By implementing telemetry, we can quickly identify pain points and optimize the system based on real-world usage patterns.
    • It fosters collaboration and continuous improvement, aligning with the community’s needs.
      Below is an outline for designing a module that tracks OpenEMR usage data via an embedded script. This design balances data collection, system performance, and compliance:

1. Define Data Points & Requirements

  • User Interactions:

    • Track menu click totals (which menus are most used, frequency, etc.).
    • Capture other UI interactions if needed.

  • Locale Information:

    • Record user locale (language settings, geographic region, etc.)
    • Optionally include browser and device info for deeper analytics.

  • Granularity & Frequency:

    • Decide whether to capture events in real time or aggregate them over intervals.
    • Determine if the data should be logged per session or in aggregate.

  • Privacy & Compliance:

    • Ensure data collection complies with privacy laws (like HIPAA, GDPR, etc.).
    • Inform users (or administrators) about what data is being tracked and provide opt-out mechanisms if necessary.

2. Embedded Script Design

  • JavaScript Event Listeners:

    • Embed a script in the OpenEMR front-end that attaches event listeners to key UI elements (e.g., menu items).
    • Each click (or interaction) should trigger a lightweight function that logs the event along with the timestamp and locale data.

  • Asynchronous Data Submission:

    • Use AJAX or the Fetch API to send event data to a designated back-end endpoint without interrupting the user experience.
    • Consider batching events (e.g., send every 10 events or after a fixed time interval) to reduce network overhead.

  • Data Sanitization:

    • Sanitize all collected data to avoid injection issues.
    • Limit data size to only what is necessary for analytics.

3. Backend Integration

  • Custom API Endpoint:

    • Develop a RESTful endpoint within OpenEMR that receives usage data.
    • Validate incoming requests and ensure authentication/authorization as needed.

  • Database Design:

    • Create custom tables (or extend existing ones) to store the collected metrics. For example:
      • UsageEvents: with fields for event type, timestamp, user ID, locale, and any additional context.
      • AggregatedMetrics: for summarizing data (e.g., total clicks per menu item per day).

  • Data Processing:

    • Consider background processing (e.g., cron jobs or queued jobs) for heavy analytics or aggregation tasks to avoid performance issues during data entry.
    • Use indexing on key fields (like event timestamps and types) for efficient querying.

4. Reporting & Dashboard

  • Admin Dashboard Module:

    • Build a reporting interface within OpenEMR for administrators to view usage data.
    • Provide filters (e.g., date range, menu items, locale) and visualizations (charts or tables) to aid analysis.

  • Visualization Tools:

    • Use libraries like Chart.js or Highcharts to create dynamic graphs and usage reports.
    • Ensure that the dashboard is responsive and does not compromise the core functionality of OpenEMR.

5. Security and Performance Considerations

  • Performance:

    • Keep the embedded script lightweight to avoid impacting the user interface responsiveness.
    • Optimize backend processing to handle bursts of event data without overloading the server.

  • Security:

    • Validate and sanitize all data both on the client and server sides.
    • Implement proper logging and monitoring to detect any misuse or anomalies in data submission.
    • Ensure that only authorized systems or users can access the usage data dashboard.

  • Scalability:

    • Design the module so that it can handle increasing amounts of data over time.
    • Consider archiving older data if real-time analytics are only necessary for recent periods.
  • Note that I had ChatGPT help creating this outline. I am thankful that I did not have this tool available to me as I was pursuing my BSEE degree or learning programming. Personally I believed it would have harmed building my skillset. I hope I’m wrong and young engineers will adapt.
1 Like