back to list
5/17/2017, 11:52:23 AM
Improve MyRocks' Random Read using TerarkDB

1. MyRocks Introduction

MyRocks is a MySQL distribution that uses RocksDB as the storage engine. It is created and maintained by Facebook. Lots of companies are using it these days. By using Terark's cutting edge algorithms, we've greatly improved its random read performance.

Comparing to InnoDB, MyRocks has a much lower write amplification (LSM-Tree is much more friendly to SSD), and has a much better compression.

MyRocks is a single instance of RocksDB, with a higher resource utilization in a multi-tenant environment (such as cloud databases, multi-sector shared databases within large companies).

2. Block Compression and MyRocks Random Read Problem

MyRocks itself is based on block compression for data storage. When accessing a large number of data items in random read, the frequent block decompression and frequent IO due to the blocks' decompression (decompression requires a lot of CPU computing) results in performance degradation.

Block compression is currently the most widely used compression technology for storage engines such as InnoDB, LevelDB, RocksDB, TukoDB, WiredTiger, and so on. In order to understand how we optimize MyRocks, we must first understand the principle of block compression.

As the name suggests, block compression works according to a set block size. Multiple records are packaged into a block and when the user needs to read one of them, the positioning to the target block is identified through the index and then the block is extracted and decompressed. Finally, the target record is retrieved from the decompressed data.

As a consequence, if we increase the size of the block the number of data in a single block will be larger and that will increase the compression rate, but it will also increase the amount of "useless" data decompressed. Inversely, if we reduce the block size, there is less "useless" data to decompress in order to reach the target record, but the compression rate will be poorer.

In addition, when the user searches for data, the block compressed and decompression data will be cached in memory, resulting in a double cache problem. To mitigate this problem, some products use DIRECT IO technology, for memory-only stored decompressed data, which to a certain extent can reduce the memory utilization.

In general, the block compression itself has a lot of insurmountable flaws: memory usage is too large, the compression rate is too low, random read is too slow. Terark's searchable compression technology, which can be directly search on compressed data, naturally avoids these problems.

3. TerarkDB is Built for Random Read

Terark is one of the few teams in China to innovate on storage engines. As a start-up company focused on this area, Terark achieved a storage engine, TerarkDB, based on its own searchable compression technology.

Strategically, TerarkDB integrates its searchable compression technology into RocksDB, which is compatible with a variety of existing database systems through RocksDB's ecosystem.

Technically, TerarkDB uses global compression, which greatly improves the compression ratio on the one hand and can search directly on the compressed data. Ultimately, with the minimum amount of memory, it achieves extremely high random read performance.

And, according to TerarkDB's access mode to memory, with the further development of hardware, especially 3D XPoint memory, TerarkDB can further reduce the amount of RAM and improve performance.

Replacing RocksDB in MyRocks directly with TerarkDB makes MyRocks gain the benefits of TerarkDB:

  • Higher compression ratio
    • smaller file on disk, longer SSD life
    • The larger the data size, the higher the compression rate
  • memory usage is smaller
    • only mmap compressed files, unparalleled cache problem
  • Random access is faster
    • Direct access to compressed data
    • no "useless" decompression
    • Adapt to the hardware development trend, random small amount of read + high IOPS

4. MySQL on TerarkDB

Based on the TerarkDB storage engine and MyRocks, we have released MySQL on TerarkDB. It combines the above technologies into MySQL. MySQL on TerarkDB is available through our official website and it can also be purchased through QingCloud's AppCenter.

4.1. MySQL on TerarkDB Architecture

4.2. Performance

  • Amazon movie data (~ 8 million reviews), About 9.1GB (after removed meta information, valuable data is about 8.1GB )
  • The graph on the left is the random read performance when memory is limited to 4G
  • The two graphs on the right are memory usage and random read on unlimited memory