Storage Engine Overview

The storage engine is the storage base of the database system. The database uses the storage engine to create, query, update and delete data. Depending on your needs, different storage engines will provide different storage mechanisms and can be combined with different execution engines to become database microkernels suitable for specific scenarios.

Currently, the YMatrix database system supports the use of several storage engines:

PostgreSQL-based HEAP
Greenplum-based AO (Append-optimized)
MARS2 developed by YMatrix
MARS3 optimized based on MARS2

Storage Engine	Description
HEAP	The storage engine provided by PostgreSQL database, also known as heap tables, uses row storage. This type of table supports a large number of concurrent read and write, transaction, index and other features
AO	The storage engine provided by the Greenplum database, also known as append optimization table, uses Append-optimized (AO) storage format to load and read data in batches, and can provide performance advantages on HEAP tables. Append optimized storage provides checksums for data protection, compression, and row/column orientation. Tables with row or column addition optimization can be compressed
MARS2	The storage engine developed on MARS reduces the number of I/O addressing times with its physically ordered merging method, thereby improving the query performance of data in the table. MARS2 supports compression, column storage, automatic archiving, pre-aggregation and other functions, and performs superiorly in timing scenarios. In terms of compression, MARS2 can provide customized compression solutions: *Coding chain (mxcustom)*. Enable coding chains to perform detailed, targeted and customized deep compression of the table. MARS2 tables do not support data updates and deletion
MARS3	The storage engine developed on MARS2 not only inherits the performance advantages of MARS2 timing scenarios, but also adds support for AP and TP scenarios. In addition, in addition to inheriting many of the advantages and characteristics of MARS2, such as coding chain compression, column storage and other functions, it also optimizes the writing performance of data, and can cope with different forms of write scenarios (such as high-frequency, small-batch TP scenarios, low-frequency, large-batch AP scenarios). MARS3 supports the update and deletion of general data, supports the MVCC mechanism, supports the addition and deletion of columns, and supports Brin index

The basic features of MARS2 and MARS3 are as follows:

Features	MARS2	MARS3
Storage method	Column storage	Mixed storage or column storage
Compression	Support coding chain	Support coding chain
Complete data update and deletion through `UPDATE` and `DELETE` clauses	Not supported	Supported
Add or delete columns	Not supported	Support
MVCC	Not supported	Supported
Runtimefilter	Support	Not supported
Storage Diagnosis	Support	Not supported
Data Batch Merge (UPSERT)	Support	Support
INSERT INTO ... VALUES ... ON CONFLICT ... DO UPDATE Syntax	Not supported	Not supported
Index	When creating a table, you must create a `mars2_btree` index to specify the sort key, otherwise the table will not take effect (the `ORDER BY` clause of the MARS3 table)	One or more `mars3_brin` indexes can be optionally created to improve query efficiency (the minmax index of the MARS2 table)
Table parameter configuration method	`WITH(a=1,b=2,...)`	`WITH(mars3options='a=1,b=2,...')`