MatrixGate's fault tolerance mechanism

This document introduces the fault tolerance mechanism of MatrixGate.

Because MatrixGate uses an external table mechanism internally, microbatches insert data into the target table. Therefore, any piece of data is entered in batches together with other submitted data when it is entered into the database. If any of the data formats are incorrect, the entire batch of data will fail to be stored.

Starting from 4.3, MatrixGate has added a fault tolerance mechanism. A single data format error will not affect other data entry, and will return error data information and record error logs.

Notes!
Fault tolerance is only for format errors, and if the constraint rules are violated (unique index) it will still fail in batches.

1 Error message

Unlike the previous error data that directly returns HTTP 500, the HTTP code returned after fault tolerance is still 200. The response body will contain error line information, for example:

At line: 2
missing data for column "c3"

2 Threshold Control

Of course, it is not unlimited tolerance errors, and tolerance thresholds are related to GUC: gp_initial_bad_row_limit. When the number of wrong data rows exceeds 5 * gp_initial_bad_row_limit, batch write failed.

For more detailed usage methods, please refer to MatrixGate.