log

Viewing the system operation log is the basic method to diagnose cluster failures.

1 YMatrix Log

YMatrix has two types of logs:

Cluster Management Log
Query operation log

1.1 Cluster Management Log

The default directory of cluster management logs is $HOME/gpAdminLogs. Different tools have different log files in the directory, such as: mxstart_20210701.log, mxstop_20210701.log, etc.

The log file naming rules are: {administrative tool name}_{date}.log, as shown below:

gpconfig_20210723.log
mxstart_20210701.log
mxstart_20210708.log
mxstop_20210701.log
mxstop_20210723.log
......

The log format is: date + time + tool name + host name + user name + log level + log content. like:

20230308:17:29:32:001644 mxstart:mdw:mxadmin-[INFO]:-Greenplum Catalog Version: '302012031'
20230308:17:29:32:001644 mxstart:mdw:mxadmin-[WARNING]:-postmaster.pid file exists on Master, checking if recovery startup required
......

The log levels are divided into INFO, WARNING, FATAL, etc.

INFO is normal output
WARNING encountered an exception during execution, but it can still continue to execute
FATAL encountered a serious error, the entire process could not be carried out, and had to stop

Therefore, when the tool execution fails, it can be diagnosed by viewing the WARNING and FATAL records of the corresponding log.

1.2 Query the operation log

The query operation log is located in the log directory under the master node and data node (Segment) node data directory. The file name format is gpdb-{date}_{time}.csv, such as:

gpdb-2021-07-01_164653.csv
gpdb-2021-07-01_164655.csv
gpdb-2021-07-02_000000.csv
......

This log is a PostgreSQL log and can be used to locate the following problems:

Query execution error
Node failure

The file content includes the date, time, user name, process number, connection ID, log level and specific error information of the log generation.

The following log content is the error log of the user trying to connect to a non-existent database:

2021-07-26 10:21:34.692106 CST,"mxadmin","postgggg",p10188,th-1455884160,"[local]",,2021-07-26 10:21:34 CST,0,con14,,seg-1,,,,sx1,"FATAL","3D000","database ""postgggg"" does not exist",,,,,,,0,,"postinit.c",1024,

The meanings of each field of the log are as follows:

Log content	Meaning
2021-07-26 10:21:34.692106 CST	Log time
mxadmin	Contacted username
postgggg	Connected database name
p10188	Connected process number
th-1455884160	Thread number
[local]	Connection method is local connection
con14	Connection number
seg-1	Node Number
FATAL	Log Level
database "postgggg" does not exist	Error message

2. MatrixGate log

The log of MatrixGate is also in the gpAdminLogs directory, and the file name is matrixgate.{date}_{time}.log, as shown below:

matrixgate.2023-04-10_141001-2369.log

Log formats are similar to other management tools:

20210726:13:47:17 matrixgate:mxadmin:mdw:010889-[INFO]:-Connected to Database, version PostgreSQL 12 (MatrixDB 5.0.0+enter
prise) (Greenplum Database 7.0.0+dev.20438.g08c3e56457 build commit:08c3e5645733a58360aa681b717293ab9a916968) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9), 64-bit compiled on Mar 23 2023 11:45:21
20210726:13:47:17 matrixgate:mxadmin:mdw:010889-[INFO]:-[Writer.Stream] Initialized with 100ms load interval and 10 prepared insert
20210726:13:47:17 matrixgate:mxadmin:mdw:010889-[INFO]:-[Writer.Stream] Started, listening on :12120
20210726:13:47:17 matrixgate:mxadmin:mdw:010889-[INFO]:-[Trans.Plain] Started, parallel level 8
20210726:13:47:17 matrixgate:mxadmin:mdw:010889-[INFO]:-[Source.HTTP] http listening on :8086

MatrixGate logs can also be viewed using the mxgate log command:

[mxadmin@mdw gpAdminLogs]$  mxgate log
**********************************************************
 __  __       _        _       ____       _
|  \/  | __ _| |_ _ __(_)_  __/ ___| __ _| |_ ___
| |\/| |/ _` | __| '__| \ \/ / |  _ / _` | __/ _ \
| |  | | (_| | |_| |  | |>  <| |_| | (_| | ||  __/
|_|  |_|\__,_|\__|_|  |_/_/\_\\____|\__,_|\__\___|
  Version: v5.0.0 (git: HEAD 53e21dfa)
  Your Copy is Licensed to: yMatrix.cn; 2023-06-23; any
**********************************************************
==> [pid:436] matrixgate.2023-04-10_141001-2369.log <==
github.com/ymatrix-data/matrixgate/src/job.startUpPrepare
    /home/runner/work/matrixdb-ci/matrixdb-ci/src/job/job.go:55
github.com/ymatrix-data/matrixgate/src/server.(*Server).Start
    /home/runner/work/matrixdb-ci/matrixdb-ci/src/server/server.go:84
main.main
    /home/runner/work/matrixdb-ci/matrixdb-ci/src/bin/mxgated/main.go:111
runtime.main
    /opt/hostedtoolcache/go/1.19.7/x64/src/runtime/proc.go:250
runtime.goexit
    /opt/hostedtoolcache/go/1.19.7/x64/src/runtime/asm_amd64.s:1594
==> [pid:2369] matrixgate.2023-04-10_141001-2369.log <==
streaming.go:741:2023-04-10:14:33:46.710 matrixgate:mxadmin:mdw:010889-[INFO]:-[Writer.Stream] Initialized with 100ms load interval and 10 prepared insert
streaming.go:741:2023-04-10:14:33:57.230 matrixgate:mxadmin:mdw:010889-[INFO]:-[Writer.Stream] Started, listening on :12120
streaming.go:741:2023-04-10:14:34:07.729 matrixgate:mxadmin:mdw:010889-[INFO]:-[Trans.Plain] Started, parallel level 8
streaming.go:741:2023-04-10:14:34:18.239 matrixgate:mxadmin:mdw:010889-[INFO]:-[Source.HTTP] http listening on :8086

Note! The prerequisite for viewing logs using the mxgate log command is that the mxgate process must be running, and the log contents seen are also generated by the running process.