Number of patient records stored doesn’t really have a direct effect on live system performance, that’s just rows in a table. Unless, there is an issue with the SQL back end.
I’d start with checking basic things and work up from there. What are the available database resources? Are the application and database pulling from the same resources ? If the web server has 16vCPU and 128GB RAM, but the database server has minimal specifications, then your system is going to run like crap. I’ve never actually pushed a half a million patients into OpenEMR, but I know of sites with over 200,000 patients, and the minimum SQL host hardware is at least 64GB of RAM with 16+ CPU cores to keep up with dozens of providers and hundreds of users.
Check your database status, are the tables healthy, any errors in the SQL server logs? If your back end is only reporting 10% utilization, something else is wrong.