Upgrade to 6.0.0 - Docker volume runs out of space (problem on production not seen on test)

Support
#1

Situation
One more thing … when I try to upgrade my production sever, I’m running out of space when I try to copy my documents onto the docker volume. The test system did not have this problem. I tried deleting the error logs to free up room. Still failing. I read somewhere that this can sometimes be a permissions issue. Anyway, I’m trying to figure out to how make space (I have plenty of disk space but my docker space is apparently full? 43G?). Do I need to resize my Docker - sounds a little scary - or can I make room some other way?

ubuntu@ip-10-0-2-213:~$ sudo docker cp documents/ standard_openemr_1:/var/www/localhost/htdocs/openemr/sites/default/documents/
Error response from daemon: Error processing tar file(exit status 1): Error setting up pivot dir: mkdir /mnt/docker/aufs/mnt/3dccbf1bdee1332497bb348ba67b7a73b5c3209df4b406386b4b46xxxxxxxx/var/www/localhost/htdocs/openemr/sites/default/documents/.pivot_root075757591: no space left on device
ubuntu@ip-10-0-2-213:~$ df -lH
Filesystem      Size  Used Avail Use% Mounted on
udev            4.2G     0  4.2G   0% /dev
tmpfs           837M  9.4M  827M   2% /run
/dev/xvda1      209G   20G  189G  10% /
tmpfs           4.2G  1.1M  4.2G   1% /dev/shm
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs           4.2G     0  4.2G   0% /sys/fs/cgroup
/dev/loop2       59M   59M     0 100% /snap/core18/2066
/dev/loop1       34M   34M     0 100% /snap/snapd/11841
/dev/loop0       35M   35M     0 100% /snap/amazon-ssm-agent/3552
/dev/loop4       34M   34M     0 100% /snap/amazon-ssm-agent/2996
/dev/loop3       59M   59M     0 100% /snap/core18/1997
/dev/loop5       34M   34M     0 100% /snap/snapd/11588
/dev/xvdd        43G   43G     0 100% /mnt/docker
tmpfs           837M     0  837M   0% /run/user/1000

OpenEMR Version
I’m using OpenEMR version 6.0.0

Operating System
I’m using: Ubuntu 16/aws 4.4

#2

The upgrade also hangs at this point (no more database cpu usage or writes - see below). Also, it’s taking way longer than the test with the same database, and thus there is something else wrong. I’m restoring from backup images.
–RBL

(form_id,field_id,group_id,title,seq,data_type,uor,fld_length,max_length,list_id,titlecols,datacols,default_value,edit_options,description,fld_rows) VALUES (‘DEM’, ‘care_team_facility’, IFNULL(@group_id,@backup_group_id), ‘Care Team (Facility)’, @seq+1, 44, 1, 0, 0, ‘’, 1, 1, ‘’, ‘’, ‘’, 0)
INSERT INTO supported_external_dataloads (load_type, load_source, load_release_date, load_filename, load_checksum) VALUES(‘ICD10’, ‘CMS’, ‘2020-10-01’, ‘Code-Descriptions.zip’, ‘f22e7201fa662689d85b926a32359701’)
INSERT INTO supported_external_dataloads (load_type, load_source, load_release_date, load_filename, load_checksum) VALUES(‘ICD10’, ‘CMS’, ‘2020-10-01’, ‘Zip File 5 2021 ICD-10-PCS Order File (Long and Abbreviated Titles).zip’, ‘6a61cee7a8f774e23412ca1330980bbb’)
ALTER TABLE documents MODIFY hash varchar(255) DEFAULT NULL
DROP TABLE log_validator
ALTER TABLE log_comment_encrypt ADD checksum_api longtext

#3

Here is an approach I am considering the following approach for resizing the docker (has anyone here tried this? does this seem like a reasonable approach? anything I need to be aware of … maybe I can even just do this docker p:

(from https://stackoverflow.com/questions/37100065/resize-disk-usage-of-a-docker-container)

  1. (optional) If you have already downloaded any image via docker pull you need to clean them first - otherwise they won’t be resized docker rmi openemr/openemr – {is this even possible with openemr?}
  2. Edit the storage config vi /etc/sysconfig/docker-storage There should be something like DOCKER_STORAGE_OPTIONS="..." , change it to DOCKER_STORAGE_OPTIONS="... --storage-opt dm.basesize=100G"
  3. Restart the docker deamon service docker restart
  4. Pull the image docker pull openemr/openemr
  5. (optional) verification docker run -i -t openemr/openemr /bin/bash df -h

(https://stackoverflow.com/a/39529120)

answered Sep 16 '16 at 10:22
(https://stackoverflow.com/users/2440262/ondrej-svejdar)
Ondrej Svejdar
19.2k44 gold badges5050 silver badges77

#4

So I tried this:

ubuntu@ip-10-0-2-159:~$ sudo docker pull openemr/openemr
Using default tag: latest
latest: Pulling from openemr/openemr
801bfaa63ef2: Pull complete 
c0fca5602b80: Pull complete 
16ae7646d015: Pull complete 
25745d6986aa: Pull complete 
287024c5c48f: Pull complete 
7748783f7444: Pull complete 
a738d3d8c0bd: Pull complete 
fbd98a41f2ba: Pull complete 
617f72cc72f3: Pull complete 
0edf1a638799: Pull complete 
f77ec4b287c9: Pull complete 
161d4566e8be: Pull complete 
d484f49c1af7: Pull complete 
f07d96935550: Pull complete 
e3361e0f055d: Pull complete 
371e6fb1a227: Pull complete 
5020cb166b70: Pull complete 
Digest: sha256:12e3549a52df9240e8c0a0603ed3070aa482e05f6806caf2549a20982df6c000
Status: Downloaded newer image for openemr/openemr:latest

And now docker has adequate room at the moment…
ubuntu@ip-10-0-2-159:~$ df -lH
Filesystem Size Used Avail Use% Mounted on
udev 4.2G 0 4.2G 0% /dev
tmpfs 837M 9.3M 828M 2% /run
/dev/xvda1 209G 13G 196G 6% /
tmpfs 4.2G 410k 4.2G 1% /dev/shm
tmpfs 5.3M 0 5.3M 0% /run/lock
tmpfs 4.2G 0 4.2G 0% /sys/fs/cgroup
/dev/loop4 35M 35M 0 100% /snap/amazon-ssm-agent/3552
/dev/loop3 34M 34M 0 100% /snap/amazon-ssm-agent/2996
/dev/loop2 59M 59M 0 100% /snap/core18/1997
/dev/loop1 59M 59M 0 100% /snap/core18/2066
/dev/loop5 34M 34M 0 100% /snap/snapd/11841
/dev/loop0 34M 34M 0 100% /snap/snapd/11588
/dev/xvdd 43G 39G 1.8G 96% /mnt/docker
tmpfs 837M 0 837M 0% /run/user/1000

That did nothing to upgrade to v6.0.0

Soo… I simply did not migrate the documents to save docker volume space, and then the production upgrade proceeded well. Fortunately, we had switched to a different system to store patient documents so this is not a major issue and we can sort of manually hobble our way through a backup of the document tree from openemr. It would be nice to be able to use the documents, but I’m just glad I have an upgrade to 6.0.0 and then can patch to patch 1.
–RBL