Help with "Check that you can ping the server mysql"

Situation
I am running OpenEMR in AWS. Last thursday (Sept. 12th) I got an error saying the Site ID was missing - so I logged into AWS and restarted the ec2 instance, and it cleared up. However the following morning, the problem was there again - so I once again restarted the ec2 instance and started getting a different error “Check that you can ping the server mysql.”

I saw that others were having problems relatively recently in AWS, and was hoping it would just clear up. Since then I’ve restarted the instance multiple times each day, I still cannot access openemr at all - currently, with the same mysql error.

I do have experience developing in PHP, however, I have zero experience working with AWS, it sadly is overwhelming. (I don’t even know how to access the php files or ftp in to the site.)

I really need to fix this soon - any help and suggestions would be greatly appreciated. And I apologize in advance, but I’ll probably need dummy instructions on how to do anything beyond restarting the instance in AWS.

OpenEMR Version
I’m using OpenEMR version… not sure, 5.01+

Browser:
I’m using: opera, chrome, firefox

Operating System
I’m using: win 7, win 10

hi @susmitahunter, which package are you using?

I installed the express version a year ago… i think at that time it was 5.0.1 at the time

would have created a key pair to use to ssh into that instance

It did create the .pem file, and I followed those instructions… got the error message saying the permissions are too open.

I think I got in. I followed the instructions on how to use putty to get in, with the modification of using ubuntu as the username. What should I do next?

check on the dockers with

sudo bash
docker ps

further help is at openemr-devops/packages/express at master · openemr/openemr-devops · GitHub

i was able to run those commands and it gave me :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
88f8a6baf98c openemr/openemr:5.0.1 “./run_openemr.sh” 16 months ago Up 24 hours 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp lightsail_openemr_1

I then went to that first link and tried the general commands, the apache one worked, and got me to a new prompt, at which point I wasn’t sure what I was doing. I also tried the mysql docker command, but that gave me an error saying the docker exec command requires at least 2 arguments.

I’m sorry again for needing my hand held, but I’m still at a loss as to what is actually wrong and what I’m supposed to do… with tools I’ve really never played with before. So thank you again for your continued help.

ok, so the mysql container isn’t running, you should be seeing something like the below in addition to the openemr container

5bceca97b5b2        openemr/mysql-xtrabackup:latest   "docker-entrypoint..."   9 months ago        Up 11 days          3306/tcp     

from the root bash prompt something like
root@ip-10-0-1-118:~#
what does
docker ps -a
show you?

CONTAINER ID        IMAGE                             COMMAND                  C                                                                                        REATED             STATUS                    PORTS                                                                                                                              NAMES
88f8a6baf98c        openemr/openemr:5.0.1             "./run_openemr.sh"       1                                                                                        6 months ago       Up 28 hours               0.0.0.0:80->80/tcp, 0.0.0.0:443->44                                                                                        3/tcp   lightsail_openemr_1
d05167f16b1f        openemr/mysql-xtrabackup:latest   "docker-entrypoint..."   1                                                                                        6 months ago       Exited (1) 28 hours ago                                                                                                                                      lightsail_mysql_1

how about
docker logs d05167f16b1f

Initializing database
2018-04-28T18:20:14.578848Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is                                                                                                                                                              deprecated. Please use --explicit_defaults_for_timestamp server option (see doc                                                                                                                                                             umentation for more details).
2018-04-28T18:20:15.698529Z 0 [Warning] InnoDB: New log files created, LSN=45790
2018-04-28T18:20:15.952408Z 0 [Warning] InnoDB: Creating foreign key constraint                                                                                                                                                              system tables.
2018-04-28T18:20:16.020103Z 0 [Warning] No existing UUID has been found, so we a                                                                                                                                                             ssume that this is the first time that this server has been started. Generating                                                                                                                                                              a new UUID: ccac40c5-4b10-11e8-8f58-0242ac120002.
2018-04-28T18:20:16.022331Z 0 [Warning] Gtid table is not ready to be used. Tabl                                                                                                                                                             e 'mysql.gtid_executed' cannot be opened.
2018-04-28T18:20:16.023204Z 1 [Warning] root@localhost is created with an empty                                                                                                                                                              password ! Please consider switching off the --initialize-insecure option.
2018-04-28T18:20:16.878650Z 1 [Warning] 'user' entry 'root@localhost' ignored in                                                                                                                                                              --skip-name-resolve mode.
2018-04-28T18:20:16.878748Z 1 [Warning] 'user' entry 'mysql.session@localhost' i                                                                                                                                                             gnored in --skip-name-resolve mode.
2018-04-28T18:20:16.878779Z 1 [Warning] 'user' entry 'mysql.sys@localhost' ignor                                                                                                                                                             ed in --skip-name-resolve mode.
2018-04-28T18:20:16.878812Z 1 [Warning] 'db' entry 'performance_schema mysql.ses                                                                                                                                                             sion@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:16.878847Z 1 [Warning] 'db' entry 'sys mysql.sys@localhost' ign                                                                                                                                                             ored in --skip-name-resolve mode.
2018-04-28T18:20:16.878901Z 1 [Warning] 'proxies_priv' entry '@ root@localhost'                                                                                                                                                              ignored in --skip-name-resolve mode.
2018-04-28T18:20:16.878961Z 1 [Warning] 'tables_priv' entry 'user mysql.session@                                                                                                                                                             localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:16.878987Z 1 [Warning] 'tables_priv' entry 'sys_config mysql.sy                                                                                                                                                             s@localhost' ignored in --skip-name-resolve mode.
Database initialized
Initializing certificates
Generating a 2048 bit RSA private key
....+++
................................................................................                                                                                                                                                             ....................+++
unable to write 'random state'
writing new private key to 'ca-key.pem'
-----
Generating a 2048 bit RSA private key
..+++
...................+++
unable to write 'random state'
writing new private key to 'server-key.pem'
-----
Generating a 2048 bit RSA private key
......................+++
..............+++
unable to write 'random state'
writing new private key to 'client-key.pem'
-----
Certificates initialized
MySQL init process in progress...
2018-04-28T18:20:19.899688Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is                                                                                                                                                              deprecated. Please use --explicit_defaults_for_timestamp server option (see doc                                                                                                                                                             umentation for more details).
2018-04-28T18:20:19.907481Z 0 [Note] mysqld (mysqld 5.7.22) starting as process                                                                                                                                                              88 ...
2018-04-28T18:20:19.913267Z 0 [Note] InnoDB: PUNCH HOLE support available
2018-04-28T18:20:19.913331Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic                                                                                                                                                              builtins
2018-04-28T18:20:19.913354Z 0 [Note] InnoDB: Uses event mutexes
2018-04-28T18:20:19.913374Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence()                                                                                                                                                              is used for memory barrier
2018-04-28T18:20:19.913393Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-04-28T18:20:19.913411Z 0 [Note] InnoDB: Using Linux native AIO
2018-04-28T18:20:19.913806Z 0 [Note] InnoDB: Number of pools: 1
2018-04-28T18:20:19.913959Z 0 [Note] InnoDB: Using CPU crc32 instructions
2018-04-28T18:20:19.921574Z 0 [Note] InnoDB: Initializing buffer pool, total siz                                                                                                                                                             e = 128M, instances = 1, chunk size = 128M
2018-04-28T18:20:19.942729Z 0 [Note] InnoDB: Completed initialization of buffer                                                                                                                                                              pool
2018-04-28T18:20:19.945182Z 0 [Note] InnoDB: If the mysqld execution user is aut                                                                                                                                                             horized, page cleaner thread priority can be changed. See the man page of setpri                                                                                                                                                             ority().
2018-04-28T18:20:19.957760Z 0 [Note] InnoDB: Highest supported file format is Ba                                                                                                                                                             rracuda.
2018-04-28T18:20:19.991212Z 0 [Note] InnoDB: Creating shared tablespace for temp                                                                                                                                                             orary tables
2018-04-28T18:20:19.991437Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12                                                                                                                                                              MB. Physically writing the file full; Please wait ...
2018-04-28T18:20:20.035590Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-04-28T18:20:20.037144Z 0 [Note] InnoDB: 96 redo rollback segment(s) found.                                                                                                                                                              96 redo rollback segment(s) are active.
2018-04-28T18:20:20.037203Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are                                                                                                                                                              active.
2018-04-28T18:20:20.041215Z 0 [Note] InnoDB: 5.7.22 started; log sequence number                                                                                                                                                              2589156
2018-04-28T18:20:20.041773Z 0 [Note] Plugin 'FEDERATED' is disabled.
2018-04-28T18:20:20.043929Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/li                                                                                                                                                             b/mysql/ib_buffer_pool
2018-04-28T18:20:20.054279Z 0 [Note] Found ca.pem, server-cert.pem and server-ke                                                                                                                                                             y.pem in data directory. Trying to enable SSL support using them.
2018-04-28T18:20:20.055362Z 0 [Warning] CA certificate ca.pem is self signed.
2018-04-28T18:20:20.055939Z 0 [Note] InnoDB: Buffer pool(s) load completed at 18                                                                                                                                                             0428 18:20:20
2018-04-28T18:20:20.066637Z 0 [Warning] Insecure configuration for --pid-file: L                                                                                                                                                             ocation '/var/run/mysqld' in the path is accessible to all OS users. Consider ch                                                                                                                                                             oosing a different directory.
2018-04-28T18:20:20.069159Z 0 [Warning] 'user' entry 'root@localhost' ignored in                                                                                                                                                              --skip-name-resolve mode.
2018-04-28T18:20:20.069361Z 0 [Warning] 'user' entry 'mysql.session@localhost' i                                                                                                                                                             gnored in --skip-name-resolve mode.
2018-04-28T18:20:20.069449Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignor                                                                                                                                                             ed in --skip-name-resolve mode.
2018-04-28T18:20:20.069489Z 0 [Warning] 'db' entry 'performance_schema mysql.ses                                                                                                                                                             sion@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:20.069511Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ign                                                                                                                                                             ored in --skip-name-resolve mode.
2018-04-28T18:20:20.069683Z 0 [Warning] 'proxies_priv' entry '@ root@localhost'                                                                                                                                                              ignored in --skip-name-resolve mode.
2018-04-28T18:20:20.072632Z 0 [Warning] 'tables_priv' entry 'user mysql.session@                                                                                                                                                             localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:20.072692Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sy                                                                                                                                                             s@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:20.084728Z 0 [Note] Event Scheduler: Loaded 0 events
2018-04-28T18:20:20.085204Z 0 [Note] mysqld: ready for connections.
Version: '5.7.22'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Communi                                                                                                                                                             ty Server (GPL)
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping                                                                                                                                                              it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Sk                                                                                                                                                             ipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it                                                                                                                                                             .
2018-04-28T18:20:25.864150Z 4 [Warning] 'user' entry 'root@localhost' ignored in                                                                                                                                                              --skip-name-resolve mode.
2018-04-28T18:20:25.864263Z 4 [Warning] 'user' entry 'mysql.sys@localhost' ignor                                                                                                                                                             ed in --skip-name-resolve mode.
2018-04-28T18:20:25.864312Z 4 [Warning] 'db' entry 'performance_schema mysql.ses                                                                                                                                                             sion@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:25.864385Z 4 [Warning] 'db' entry 'sys mysql.sys@localhost' ign                                                                                                                                                             ored in --skip-name-resolve mode.
2018-04-28T18:20:25.864431Z 4 [Warning] 'proxies_priv' entry '@ root@localhost'                                                                                                                                                              ignored in --skip-name-resolve mode.
2018-04-28T18:20:25.864495Z 4 [Warning] 'tables_priv' entry 'user mysql.session@                                                                                                                                                             localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:25.864523Z 4 [Warning] 'tables_priv' entry 'sys_config mysql.sy                                                                                                                                                             s@localhost' ignored in --skip-name-resolve mode.

2018-04-28T18:20:25.878199Z 0 [Note] Giving 0 client threads a chance to die gra                                                                                                                                                             cefully
2018-04-28T18:20:25.878262Z 0 [Note] Shutting down slave threads
2018-04-28T18:20:25.878285Z 0 [Note] Forcefully disconnecting 0 remaining client                                                                                                                                                             s
2018-04-28T18:20:25.878306Z 0 [Note] Event Scheduler: Purging the queue. 0 event                                                                                                                                                             s
2018-04-28T18:20:25.878801Z 0 [Note] Binlog end
2018-04-28T18:20:25.879586Z 0 [Note] Shutting down plugin 'ngram'
2018-04-28T18:20:25.879737Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2018-04-28T18:20:25.879764Z 0 [Note] Shutting down plugin 'partition'
2018-04-28T18:20:25.879808Z 0 [Note] Shutting down plugin 'ARCHIVE'
2018-04-28T18:20:25.879827Z 0 [Note] Shutting down plugin 'MEMORY'
2018-04-28T18:20:25.879847Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2018-04-28T18:20:25.879864Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2018-04-28T18:20:25.879881Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACE                                                                                                                                                             S'
2018-04-28T18:20:25.879897Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_CO                                                                                                                                                             LS'
2018-04-28T18:20:25.879912Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2018-04-28T18:20:25.880054Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2018-04-28T18:20:25.880078Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2018-04-28T18:20:25.880094Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2018-04-28T18:20:25.880156Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS                                                                                                                                                             '
2018-04-28T18:20:25.880177Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2018-04-28T18:20:25.880192Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE                                                                                                                                                             '
2018-04-28T18:20:25.880208Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE                                                                                                                                                             '
2018-04-28T18:20:25.880224Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2018-04-28T18:20:25.880590Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELET                                                                                                                                                             ED'
2018-04-28T18:20:25.880615Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2018-04-28T18:20:25.880632Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STO                                                                                                                                                             PWORD'
2018-04-28T18:20:25.880673Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2018-04-28T18:20:25.880691Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INF                                                                                                                                                             O'
2018-04-28T18:20:25.880707Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_ST                                                                                                                                                             ATS'
2018-04-28T18:20:25.880724Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LR                                                                                                                                                             U'
2018-04-28T18:20:25.880780Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2018-04-28T18:20:25.880801Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_                                                                                                                                                             RESET'
2018-04-28T18:20:25.880856Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2018-04-28T18:20:25.880877Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2018-04-28T18:20:25.880893Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2018-04-28T18:20:25.880909Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2018-04-28T18:20:25.880925Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2018-04-28T18:20:25.880940Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2018-04-28T18:20:25.880956Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2018-04-28T18:20:25.880971Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2018-04-28T18:20:25.881128Z 0 [Note] Shutting down plugin 'InnoDB'
2018-04-28T18:20:25.881224Z 0 [Note] InnoDB: FTS optimize thread exiting.
2018-04-28T18:20:25.881467Z 0 [Note] InnoDB: Starting shutdown...
2018-04-28T18:20:25.981957Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/                                                                                                                                                             mysql/ib_buffer_pool
2018-04-28T18:20:25.982633Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 18                                                                                                                                                             0428 18:20:25
2018-04-28T18:20:27.292584Z 0 [Note] InnoDB: Shutdown completed; log sequence nu                                                                                                                                                             mber 12193308
2018-04-28T18:20:27.293330Z 0 [Note] InnoDB: Removed temporary tablespace data f                                                                                                                                                             ile: "ibtmp1"
2018-04-28T18:20:27.293482Z 0 [Note] Shutting down plugin 'MyISAM'
2018-04-28T18:20:27.293548Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2018-04-28T18:20:27.293588Z 0 [Note] Shutting down plugin 'CSV'
2018-04-28T18:20:27.293606Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2018-04-28T18:20:27.293649Z 0 [Note] Shutting down plugin 'sha256_password'
2018-04-28T18:20:27.293675Z 0 [Note] Shutting down plugin 'mysql_native_password                                                                                                                                                             '
2018-04-28T18:20:27.293975Z 0 [Note] Shutting down plugin 'binlog'
2018-04-28T18:20:27.295173Z 0 [Note] mysqld: Shutdown complete


MySQL init process done. Ready for start up.

2018-04-28T18:20:27.561704Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is                                                                                                                                                              deprecated. Please use --explicit_defaults_for_timestamp server option (see doc                                                                                                                                                             umentation for more details).
2018-04-28T18:20:27.565916Z 0 [Note] mysqld (mysqld 5.7.22) starting as process                                                                                                                                                              1 ...
2018-04-28T18:20:27.581875Z 0 [Note] InnoDB: PUNCH HOLE support available
2018-04-28T18:20:27.582061Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic                                                                                                                                                              builtins
2018-04-28T18:20:27.582100Z 0 [Note] InnoDB: Uses event mutexes
2018-04-28T18:20:27.582140Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence()                                                                                                                                                              is used for memory barrier
2018-04-28T18:20:27.582159Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-04-28T18:20:27.582178Z 0 [Note] InnoDB: Using Linux native AIO
2018-04-28T18:20:27.582894Z 0 [Note] InnoDB: Number of pools: 1
2018-04-28T18:20:27.583181Z 0 [Note] InnoDB: Using CPU crc32 instructions
2018-04-28T18:20:27.587805Z 0 [Note] InnoDB: Initializing buffer pool, total siz                                                                                                                                                             e = 128M, instances = 1, chunk size = 128M
2018-04-28T18:20:27.604328Z 0 [Note] InnoDB: Completed initialization of buffer                                                                                                                                                              pool
2018-04-28T18:20:27.607841Z 0 [Note] InnoDB: If the mysqld execution user is aut                                                                                                                                                             horized, page cleaner thread priority can be changed. See the man page of setpri                                                                                                                                                             ority().
2018-04-28T18:20:27.621769Z 0 [Note] InnoDB: Highest supported file format is Ba                                                                                                                                                             rracuda.
2018-04-28T18:20:27.646979Z 0 [Note] InnoDB: Creating shared tablespace for temp                                                                                                                                                             orary tables
2018-04-28T18:20:27.647101Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12                                                                                                                                                              MB. Physically writing the file full; Please wait ...
2018-04-28T18:20:27.694897Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-04-28T18:20:27.696539Z 0 [Note] InnoDB: 96 redo rollback segment(s) found.                                                                                                                                                              96 redo rollback segment(s) are active.
2018-04-28T18:20:27.696594Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are                                                                                                                                                              active.
2018-04-28T18:20:27.697368Z 0 [Note] InnoDB: Waiting for purge to start
2018-04-28T18:20:27.747651Z 0 [Note] InnoDB: 5.7.22 started; log sequence number                                                                                                                                                              12193308
2018-04-28T18:20:27.748244Z 0 [Note] Plugin 'FEDERATED' is disabled.
2018-04-28T18:20:27.748569Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/li                                                                                                                                                             b/mysql/ib_buffer_pool
2018-04-28T18:20:27.760257Z 0 [Note] Found ca.pem, server-cert.pem and server-ke                                                                                                                                                             y.pem in data directory. Trying to enable SSL support using them.
2018-04-28T18:20:27.760722Z 0 [Warning] CA certificate ca.pem is self signed.
2018-04-28T18:20:27.764237Z 0 [Note] Server hostname (bind-address): '*'; port:                                                                                                                                                              3306
2018-04-28T18:20:27.764524Z 0 [Note] IPv6 is available.
2018-04-28T18:20:27.764676Z 0 [Note]   - '::' resolves to '::';
2018-04-28T18:20:27.764727Z 0 [Note] Server socket created on IP: '::'.
2018-04-28T18:20:27.768375Z 0 [Note] InnoDB: Buffer pool(s) load completed at 18                                                                                                                                                             0428 18:20:27
2018-04-28T18:20:27.768848Z 0 [Warning] Insecure configuration for --pid-file: L                                                                                                                                                             ocation '/var/run/mysqld' in the path is accessible to all OS users. Consider ch                                                                                                                                                             oosing a different directory.
2018-04-28T18:20:27.772682Z 0 [Warning] 'user' entry 'root@localhost' ignored in                                                                                                                                                              --skip-name-resolve mode.
2018-04-28T18:20:27.772780Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignor                                                                                                                                                             ed in --skip-name-resolve mode.
2018-04-28T18:20:27.772821Z 0 [Warning] 'db' entry 'performance_schema mysql.ses                                                                                                                                                             sion@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:27.772886Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ign                                                                                                                                                             ored in --skip-name-resolve mode.
2018-04-28T18:20:27.772914Z 0 [Warning] 'proxies_priv' entry '@ root@localhost'                                                                                                                                                              ignored in --skip-name-resolve mode.
2018-04-28T18:20:27.779197Z 0 [Warning] 'tables_priv' entry 'user mysql.session@                                                                                                                                                             localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:27.779269Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sy                                                                                                                                                             s@localhost' ignored in --skip-name-resolve mode.
2018-04-28T18:20:27.829512Z 0 [Note] Event Scheduler: Loaded 0 events
2018-04-28T18:20:27.829855Z 0 [Note] mysqld: ready for connections.
Version: '5.7.22'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Comm                                                                                                                                                             unity Server (GPL)
2018-04-28T18:21:36.864024Z 0 [Note] Giving 0 client threads a chance to die gra                                                                                                                                                             cefully
2018-04-28T18:21:36.864054Z 0 [Note] Shutting down slave threads
2018-04-28T18:21:36.864059Z 0 [Note] Forcefully disconnecting 0 remaining client                                                                                                                                                             s
2018-04-28T18:21:36.864068Z 0 [Note] Event Scheduler: Purging the queue. 0 event                                                                                                                                                             s
2018-04-28T18:21:36.864103Z 0 [Note] Binlog end
2018-04-28T18:21:36.865478Z 0 [Note] Shutting down plugin 'ngram'
2018-04-28T18:21:36.865527Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2018-04-28T18:21:36.865548Z 0 [Note] Shutting down plugin 'partition'
2018-04-28T18:21:36.865566Z 0 [Note] Shutting down plugin 'ARCHIVE'
2018-04-28T18:21:36.865581Z 0 [Note] Shutting down plugin 'MEMORY'
2018-04-28T18:21:36.865599Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2018-04-28T18:21:36.865615Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2018-04-28T18:21:36.865632Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACE                                                                                                                                                             S'
2018-04-28T18:21:36.865648Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_CO                                                                                                                                                             LS'
2018-04-28T18:21:36.865685Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2018-04-28T18:21:36.865771Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2018-04-28T18:21:36.865791Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2018-04-28T18:21:36.865807Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2018-04-28T18:21:36.865824Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS                                                                                                                                                             '
2018-04-28T18:21:36.865864Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2018-04-28T18:21:36.865880Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE                                                                                                                                                             '
2018-04-28T18:21:36.865896Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE                                                                                                                                                             '
2018-04-28T18:21:36.865911Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2018-04-28T18:21:36.865927Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELET                                                                                                                                                             ED'
2018-04-28T18:21:36.865943Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2018-04-28T18:21:36.865958Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STO                                                                                                                                                             PWORD'
2018-04-28T18:21:36.865974Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2018-04-28T18:21:36.865996Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INF                                                                                                                                                             O'
2018-04-28T18:21:36.866026Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_ST                                                                                                                                                             ATS'
2018-04-28T18:21:36.866042Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LR                                                                                                                                                             U'
2018-04-28T18:21:36.866058Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2018-04-28T18:21:36.866081Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_                                                                                                                                                             RESET'
2018-04-28T18:21:36.866097Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2018-04-28T18:21:36.866113Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2018-04-28T18:21:36.866129Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2018-04-28T18:21:36.866145Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2018-04-28T18:21:36.866179Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2018-04-28T18:21:36.866196Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2018-04-28T18:21:36.866212Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2018-04-28T18:21:36.866228Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2018-04-28T18:21:36.866243Z 0 [Note] Shutting down plugin 'InnoDB'
2018-04-28T18:21:36.867007Z 0 [Note] InnoDB: FTS optimize thread exiting.
2018-04-28T18:21:36.867146Z 0 [Note] InnoDB: Starting shutdown...
2018-04-28T18:21:36.967473Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/                                                                                                                                                             mysql/ib_buffer_pool
2018-04-28T18:21:36.967748Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 18                                                                                                                                                             0428 18:21:36
2018-04-28T18:21:38.282482Z 0 [Note] InnoDB: Shutdown completed; log sequence nu                                                                                                                                                             mber 40839846
2018-04-28T18:21:38.282945Z 0 [Note] InnoDB: Removed temporary tablespace data f                                                                                                                                                             ile: "ibtmp1"
2018-04-28T18:21:38.282997Z 0 [Note] Shutting down plugin 'MyISAM'
2018-04-28T18:21:38.283025Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2018-04-28T18:21:38.283067Z 0 [Note] Shutting down plugin 'CSV'
2018-04-28T18:21:38.283083Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2018-04-28T18:21:38.283123Z 0 [Note] Shutting down plugin 'sha256_password'
2018-04-28T18:21:38.283161Z 0 [Note] Shutting down plugin 'mysql_native_password                                                                                                                                                             '
2018-04-28T18:21:38.283295Z 0 [Note] Shutting down plugin 'binlog'
2018-04-28T18:21:38.284312Z 0 [Note] mysqld: Shutdown complete

2018-06-12T19:12:16.861328Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is                                                                                                                                                              deprecated. Please use --explicit_defaults_for_timestamp server option (see doc                                                                                                                                                             umentation for more details).
2018-06-12T19:12:16.862868Z 0 [Note] mysqld (mysqld 5.7.22) starting as process                                                                                                                                                              1 ...
2018-06-12T19:12:16.867103Z 0 [Note] InnoDB: PUNCH HOLE support available
2018-06-12T19:12:16.867162Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic                                                                                                                                                              builtins
2018-06-12T19:12:16.867185Z 0 [Note] InnoDB: Uses event mutexes
2018-06-12T19:12:16.867205Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence()                                                                                                                                                              is used for memory barrier
2018-06-12T19:12:16.867223Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-06-12T19:12:16.867241Z 0 [Note] InnoDB: Using Linux native AIO
2018-06-12T19:12:16.868450Z 0 [Note] InnoDB: Number of pools: 1
2018-06-12T19:12:16.870631Z 0 [Note] InnoDB: Using CPU crc32 instructions
2018-06-12T19:12:16.872342Z 0 [Note] InnoDB: Initializing buffer pool, total siz                                                                                                                                                             e = 128M, instances = 1, chunk size = 128M
2018-06-12T19:12:16.881099Z 0 [Note] InnoDB: Completed initialization of buffer                                                                                                                                                              pool
2018-06-12T19:12:16.883847Z 0 [Note] InnoDB: If the mysqld execution user is aut                                                                                                                                                             horized, page cleaner thread priority can be changed. See the man page of setpri                                                                                                                                                             ority().
2018-06-12T19:12:16.903051Z 0 [Note] InnoDB: Highest supported file format is Ba                                                                                                                                                             rracuda.
2018-06-12T19:12:17.016466Z 0 [Note] InnoDB: Creating shared tablespace for temp                                                                                                                                                             orary tables
2018-06-12T19:12:17.016616Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12                                                                                                                                                              MB. Physically writing the file full; Please wait ...
2018-06-12T19:12:17.079635Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-06-12T19:12:17.080620Z 0 [Note] InnoDB: 96 redo rollback segment(s) found.                                                                                                                                                              96 redo rollback segment(s) are active.
2018-06-12T19:12:17.080675Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are                                                                                                                                                              active.
2018-06-12T19:12:17.081505Z 0 [Note] InnoDB: Waiting for purge to start
2018-06-12T19:12:17.131756Z 0 [Note] InnoDB: 5.7.22 started; log sequence number                                                                                                                                                              40839846
2018-06-12T19:12:17.132230Z 0 [Note] Plugin 'FEDERATED' is disabled.
2018-06-12T19:12:17.132539Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/li                                                                                                                                                             b/mysql/ib_buffer_pool
2018-06-12T19:12:17.148606Z 0 [Note] Found ca.pem, server-cert.pem and server-ke                                                                                                                                                             y.pem in data directory. Trying to enable SSL support using them.
2018-06-12T19:12:17.149500Z 0 [Warning] CA certificate ca.pem is self signed.
2018-06-12T19:12:17.153243Z 0 [Note] Server hostname (bind-address): '*'; port:                                                                                                                                                              3306
2018-06-12T19:12:17.153326Z 0 [Note] IPv6 is available.
2018-06-12T19:12:17.153353Z 0 [Note]   - '::' resolves to '::';
2018-06-12T19:12:17.153396Z 0 [Note] Server socket created on IP: '::'.
2018-06-12T19:12:17.156349Z 0 [Warning] Insecure configuration for --pid-file: L                                                                                                                                                             ocation '/var/run/mysqld' in the path is accessible to all OS users. Consider ch                                                                                                                                                             oosing a different directory.
2018-06-12T19:12:17.303192Z 0 [Warning] 'user' entry 'root@localhost' ignored in                                                                                                                                                              --skip-name-resolve mode.
2018-06-12T19:12:17.303311Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignor                                                                                                                                                             ed in --skip-name-resolve mode.
2018-06-12T19:12:17.303586Z 0 [Warning] 'db' entry 'performance_schema mysql.ses                                                                                                                                                             sion@localhost' ignored in --skip-name-resolve mode.
2018-06-12T19:12:17.303634Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ign                                                                                                                                                             ored in --skip-name-resolve mode.
error from daemon in stream: Error grabbing logs: invalid character 'l' after ob                                                                                                                                                             ject key:value pair

the last error is 2018-06-12, probably should be something more recent if it was working as recently as a few days ago

yes, was definitely working as of last week. what should i be trying to do at this point?

how about

 docker logs --tail d05167f16b1f

maybe i’m typing that in wrong, i get the error that “docker logs” requires exactly 1 argument(s).

right, try 20 so

 docker logs --tail 20 d05167f16b1f
root@ip-172-31-31-206:~# docker logs --tail 20 d05167f16b1f
2019-09-18T13:38:35.439327Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-09-18T13:38:35.441269Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-09-18T13:38:35.453940Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-09-18T13:38:35.458411Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-09-18T13:38:35.490242Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-09-18T13:38:35.681606Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-09-18T13:38:35.681677Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-09-18T13:38:35.681837Z 0 [Warning] InnoDB: Retry attempts for writing partial data failed.
2019-09-18T13:38:35.681849Z 0 [ERROR] InnoDB: Write to file ./ibtmp1failed at offset 0, 1048576 bytes should have been written, only 0 were written. Operating system error number 28. Check that your OS and file system support files of this size. Check also that the disk is not full or a disk quota exceeded.
2019-09-18T13:38:35.681859Z 0 [ERROR] InnoDB: Error number 28 means 'No space left on device'
2019-09-18T13:38:35.681863Z 0 [Note] InnoDB: Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
2019-09-18T13:38:35.681867Z 0 [ERROR] InnoDB: Could not set the file size of './ibtmp1'. Probably out of disk space
2019-09-18T13:38:35.681871Z 0 [ERROR] InnoDB: Unable to create the shared innodb_temporary
2019-09-18T13:38:35.681874Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2019-09-18T13:38:36.182872Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2019-09-18T13:38:36.182900Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2019-09-18T13:38:36.182905Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-09-18T13:38:36.182910Z 0 [ERROR] Failed to initialize builtin plugins.
2019-09-18T13:38:36.182912Z 0 [ERROR] Aborting

ok, there it is, no more room on the storage, maybe @jesdynf has some easy to follow advice on how to create more room on the mysql volume?