喜迎
元旦
看懂ceph日志系列之rocksdb | 奋斗的松鼠 - Blog

看懂ceph日志系列之rocksdb

字数统计: 13.4k字   |   阅读时长≈ 79分

本文是看懂ceph日志系列文章关于rocksdb部分日志的分析

看懂ceph日志系列之rocksdb

写在前面

松鼠哥的ceph专业课程上线啦！
面向新手同学，从0实战，全面入门ceph安装部署与运维，有需要的同学赶紧联系松鼠哥订购吧：

本文是看懂ceph日志系列的rocksdb专题，关于rocksdb的介绍和调优，之前就写过两篇博文，理解rocksdb的机制和行为，对ceph的性能调优至关重要，之前的两篇文章介绍的比较浅，对其原理没有太过深入的讲解，本文也是对其的一个补充

rocksdb的代码本身很好读，facebook的文章都很全，而且社区很活跃，学会用难度还不是很大，它有它天生的优势和缺陷，更深入的代码剖析及原理介绍请期待后续文章，这里先读懂ceph日志

第一阶段 - immutable的flush

ceph使用rocksdb的第一阶段是将immutable刷到level0处，相关参数为：

1
2
3
4
5
6
7
8
max_write_buffer_number - 最大的memtable数量，一般配置可以大点，例如16
min_write_buffer_number_to_merge - 拥有多少个immutable后进行flush，配置时也要考虑level0的值
recycle_log_file_num - 用完之后的write ahead log文件回收的数量
write_buffer_size - 每个memtable的大小
writable_file_max_buffer_size - 追加内容到文件的时候使用的buffer大小，默认是0，即直接io的方式写入
flusher_threads - 进行flush的线程数
max_subcompactions - 将一个文件合并入下一个level时并行的线程数
info_log_level - rocksdb的日志级别，默认是info，想折腾机制可以开大点

因为是rocksdb的补充，所以啰嗦说明一下osd的rocksdb写过程

1
2
3
4
5
6
7
8
1、osd将写入内容提交到rocksdb。提交的内容有两种情况：
    一种是仅写入osd的数据的元数据信息，例如omap、xattr等
    另一种是写入osd的数据的元数据信息及其数据部分
    对于第二种情况的写入，是覆盖写的特殊场景，覆盖写的时候如果io不对齐，就会将数据全部先写入wal，对于对象存储的场景，没有覆盖写
2、rocksdb首先将请求写入wal，wal写完成后再写入memtable，至此写入即返回
3、memtalbe写满后，变成只读的immutable，等immutable数量达到阈值，就会一次性刷入到level0，在level0中所有immutable写到一个文件中
4、rocksdb释放immutable空间，并删除(标记删除，可能会复用)wal文件
5、计算每个level的score值，判断是否进行compaction，每个level可以同时发生compaction，其中level0 的compaction可以继续增加sub-thread来加快速度，而每层的score超过1就会触发compaction，rocksdb会优先选取score最大的层进行合并

下面通过日志来观察rocksdb的flush过程：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
2018-12-19 04:06:21.908053 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357225 from recycle list
2018-12-19 04:06:21.908196 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357343. Immutable memtables: 0.
2018-12-19 04:07:30.316143 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357282 from recycle list
2018-12-19 04:07:30.316287 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357344. Immutable memtables: 1.
2018-12-19 04:09:17.135698 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357283 from recycle list
2018-12-19 04:09:17.136086 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357345. Immutable memtables: 2.
2018-12-19 04:12:15.052290 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357284 from recycle list
2018-12-19 04:12:15.052449 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357346. Immutable memtables: 3.
2018-12-19 04:16:59.212210 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357285 from recycle list
2018-12-19 04:16:59.212348 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357347. Immutable memtables: 4.
2018-12-19 04:18:40.659759 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357286 from recycle list
2018-12-19 04:18:40.659916 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357348. Immutable memtables: 5.
2018-12-19 04:20:33.984140 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357287 from recycle list
2018-12-19 04:20:33.984244 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357349. Immutable memtables: 6.
2018-12-19 04:26:33.561736 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:684] reusing log 357288 from recycle list
2018-12-19 04:26:33.561907 7fa0d7bd9700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_write.cc:725] [default] New memtable created with log file: #357350. Immutable memtables: 7.

从日志中看出，04:06:21.908053开始写第一个memtable，到04:26:33.561907的时候，已经有了8个immutable，这个过程花费了20分12秒，环境中min_write_buffer_number_to_merge=8，因此满足了flush的条件，这个过程中，重用了此前存在的wal文件，免去了重新创建文件的开销

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
2018-12-19 04:26:33.562723 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:33.562387) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1158] Calling FlushMemTableToOutputFile with column family [default], flush slots available 8, compaction slots allowed 1, compaction slots scheduled 1
2018-12-19 04:26:33.562741 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357343
2018-12-19 04:26:33.562747 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357344
2018-12-19 04:26:33.562751 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357345
2018-12-19 04:26:33.562755 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357346
2018-12-19 04:26:33.562757 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357347
2018-12-19 04:26:33.562763 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357348
2018-12-19 04:26:33.562766 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357349
2018-12-19 04:26:33.562769 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:264] [default] [JOB 4779] Flushing memtable with next log file: 357350
2018-12-19 04:26:33.562785 7fa0e23ee700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164793562775, "job": 4779, "event": "flush_started", "num_memtables": 8, "num_entries": 4841257, "num_deletes": 1453570, "memory_usage": 1040241320}
2018-12-19 04:26:33.562794 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:293] [default] [JOB 4779] Level-0 flush table #357351: started
2018-12-19 04:26:35.180043 7fa0e23ee700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164795180015, "cf_name": "default", "job": 4779, "event": "table_file_creation", "file_number": 357351, "file_size": 61443510, "table_properties": {"data_size": 56444754, "index_size": 939165, "filter_size": 4058601, "raw_key_size": 89723951, "raw_average_key_size": 57, "raw_value_size": 25078952, "raw_average_value_size": 15, "num_data_blocks": 14259, "num_entries": 1571526, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "1453570", "kMergeOperands": "2"}}
2018-12-19 04:26:35.180072 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/flush_job.cc:319] [default] [JOB 4779] Level-0 flush table #357351: 61443510 bytes OK
2018-12-19 04:26:35.185433 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357289 to recycle list
2018-12-19 04:26:35.185441 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357343 to recycle list
2018-12-19 04:26:35.185443 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357344 to recycle list
2018-12-19 04:26:35.185444 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357345 to recycle list
2018-12-19 04:26:35.185448 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357346 to recycle list
2018-12-19 04:26:35.185450 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357347 to recycle list
2018-12-19 04:26:35.185452 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357348 to recycle list
2018-12-19 04:26:35.185453 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:242] adding log 357349 to recycle list
2018-12-19 04:26:35.185483 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.180092) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:360] [default] Level-0 commit table #357351 started
2018-12-19 04:26:35.185507 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185106) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #1 done
2018-12-19 04:26:35.185519 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185110) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #2 done
2018-12-19 04:26:35.185531 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185112) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #3 done
2018-12-19 04:26:35.185542 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185113) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #4 done
2018-12-19 04:26:35.185557 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185114) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #5 done
2018-12-19 04:26:35.185569 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185115) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #6 done
2018-12-19 04:26:35.185581 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185116) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #7 done
2018-12-19 04:26:35.185593 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185117) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/memtable_list.cc:383] [default] Level-0 commit table #357351: memtable #8 done
2018-12-19 04:26:35.185603 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185135) EVENT_LOG_v1 {"time_micros": 1545164795185126, "job": 4779, "event": "flush_finished", "lsm_state": [1, 12, 118, 1117, 2993, 0, 0], "immutable_memtables": 0}

rocksdb将immutable在内存中按序合并，合并完成即将log文件提交到recycle list，等待一段时间后这些log文件即被回收

至此，immutable的flush就完成了，日志显示此时的lsm_state为[1, 12, 118, 1117, 2993, 0, 0]，分别指level0、level1、… 、level6的sst数量，看到此时level0只有1个文件，其大小为61443510B，文件编号为357351

第二阶段 - level之间的compaction

sst从memtable flush到level0后，会立即检查各个level的score，选取score最大的level，判断是否需要进行compaction，如果score符合，则开始level的compaction。

这里先做个勘误，在ceph性能调优历程-rocksdb篇(2)
中有提到level0 -> level1的合并条件，原文强调只有符合level0_file_num_compaction_trigger才会触发compaction，实际上还有个条件就是level0的总大小超过max_bytes_for_level_base，也可以触发compaction，目前已经纠正 -.-，facebook的官方说明如下

Compaction Picking

When multiple levels trigger the compaction condition, RocksDB needs to pick which level to compact first. A score is generated for each level:

  • For non-zero levels, the score is total size of the level divided by the target size. If there is already files picked that are being compacted into the next level, the size of those files are not included into the total size, because they will soon go away.

  • for level-0, the score is the total number of files, divided by level0\_file_num_compaction_trigger, or total size over max_bytes_for_level_base, which ever is larger. (if the file size is smaller than level0_file_num_compaction_trigger, compaction won’t trigger from level 0, no matter how big the score is)

We compare the score of each level, and the level with highest score takes the priority to compact.

原文出处：Leveled Compaction

接上面的日志，继续看看

1
2
3
4
5
6
7
2018-12-19 04:26:35.185618 7fa0e23ee700  4 rocksdb: (Original Log Time 2018/12/19-04:26:35.185372) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:132] [default] Level summary: base level 1 max bytes base 375809638 files[1 12 118 1117 2993 0 0] max score 1.83
2018-12-19 04:26:35.186139 7fa0e3bf1700  3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:447] ------- DUMPING STATS -------
2018-12-19 04:26:35.186151 7fa0e3bf1700  3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl.cc:448]
2018-12-19 04:26:35.186512 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 4780] Compacting 1@0 + 12@1 files to L1, score 1.83
2018-12-19 04:26:35.186535 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 4779 Base level 0, inputs: [357351(58MB)], [357303(32MB) 357304(32MB) 357294(32MB) 357297(32MB) 357299(32MB) 357293(32MB) 357296(32MB) 357298(1138B) 357300(32MB) 357302(8102KB) 357291(32MB) 357295(8907KB)]
2018-12-19 04:26:35.186559 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164795186543, "job": 4780, "event": "compaction_started", "files_L0": [357351], "files_L1": [357303, 357304, 357294, 357297, 357299, 357293, 357296, 357298, 357300, 357302, 357291, 357295], "score": 1.82675, "input_data_size": 389032785}
2018-12-19 04:26:35.187062 7fa0e23ee700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_files.cc:388] [JOB 4779] Try to delete WAL files size 947714270, prev total WAL file size 949123808, number of live WAL files 9.

合并的统计说明，首先判断score最大的是1.83的level0，Compacting 1@0 + 12@1 files to L1z表示了要从level0选取1个文件，从level1选取12个文件合并到L1，并在Compaction start summary中详细列出参与合并的文件的大小信息，环境中每个sst文件大小配置为32MB，注意到从level0输入的文件恰好就是刚才flush完成的编号为357351的sst文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
2018-12-19 04:26:35.836924 7fa0bab9f700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357353: 97370 keys, 34467695 bytes
2018-12-19 04:26:35.836973 7fa0bab9f700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164795836950, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357353, "file_size": 34467695, "table_properties": {"data_size": 33558449, "index_size": 586721, "filter_size": 321539, "raw_key_size": 7624011, "raw_average_key_size": 78, "raw_value_size": 29432145, "raw_average_value_size": 302, "num_data_blocks": 8750, "num_entries": 97370, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:35.837339 7fa0bb3a0700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357355: 97561 keys, 34462349 bytes
2018-12-19 04:26:35.837386 7fa0bb3a0700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164795837364, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357355, "file_size": 34462349, "table_properties": {"data_size": 33555708, "index_size": 583860, "filter_size": 321795, "raw_key_size": 7611923, "raw_average_key_size": 78, "raw_value_size": 29433254, "raw_average_value_size": 301, "num_data_blocks": 8731, "num_entries": 97561, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "22", "kMergeOperands": "0"}}
2018-12-19 04:26:35.838037 7fa0ba39e700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357354: 97605 keys, 34463314 bytes
2018-12-19 04:26:35.838105 7fa0ba39e700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164795838076, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357354, "file_size": 34463314, "table_properties": {"data_size": 33557472, "index_size": 582964, "filter_size": 321892, "raw_key_size": 7606416, "raw_average_key_size": 77, "raw_value_size": 29434206, "raw_average_value_size": 301, "num_data_blocks": 8731, "num_entries": 97605, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "33", "kMergeOperands": "0"}}
2018-12-19 04:26:36.190185 7fa0ba39e700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357358: 97409 keys, 34460804 bytes
2018-12-19 04:26:36.190233 7fa0ba39e700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796190208, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357358, "file_size": 34460804, "table_properties": {"data_size": 33554865, "index_size": 583326, "filter_size": 321627, "raw_key_size": 7614186, "raw_average_key_size": 78, "raw_value_size": 29407367, "raw_average_value_size": 301, "num_data_blocks": 8752, "num_entries": 97409, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:36.294964 7fa0bab9f700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357356: 97359 keys, 34464018 bytes
2018-12-19 04:26:36.295025 7fa0bab9f700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796294994, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357356, "file_size": 34464018, "table_properties": {"data_size": 33556048, "index_size": 585467, "filter_size": 321517, "raw_key_size": 7623948, "raw_average_key_size": 78, "raw_value_size": 29417390, "raw_average_value_size": 302, "num_data_blocks": 8748, "num_entries": 97359, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "38", "kMergeOperands": "0"}}
2018-12-19 04:26:36.295343 7fa0bb3a0700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357357: 97618 keys, 34462713 bytes
2018-12-19 04:26:36.295387 7fa0bb3a0700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796295364, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357357, "file_size": 34462713, "table_properties": {"data_size": 33558039, "index_size": 581904, "filter_size": 321784, "raw_key_size": 7609928, "raw_average_key_size": 77, "raw_value_size": 29437767, "raw_average_value_size": 301, "num_data_blocks": 8711, "num_entries": 97618, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "45", "kMergeOperands": "0"}}
2018-12-19 04:26:36.296797 7fa0ba39e700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357359: 1 keys, 1138 bytes
2018-12-19 04:26:36.296851 7fa0ba39e700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796296829, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357359, "file_size": 1138, "table_properties": {"data_size": 128, "index_size": 27, "filter_size": 18, "raw_key_size": 19, "raw_average_key_size": 19, "raw_value_size": 93, "raw_average_value_size": 93, "num_data_blocks": 1, "num_entries": 1, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:36.326694 7fa0bab9f700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357360: 8624 keys, 2898733 bytes
2018-12-19 04:26:36.326732 7fa0bab9f700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796326712, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357360, "file_size": 2898733, "table_properties": {"data_size": 2823011, "index_size": 46627, "filter_size": 28110, "raw_key_size": 693175, "raw_average_key_size": 80, "raw_value_size": 2406956, "raw_average_value_size": 279, "num_data_blocks": 733, "num_entries": 8624, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "753", "kMergeOperands": "3"}}
2018-12-19 04:26:36.634711 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357352: 1428525 keys, 37746293 bytes
2018-12-19 04:26:36.634760 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796634735, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357352, "file_size": 37746293, "table_properties": {"data_size": 33554795, "index_size": 542452, "filter_size": 3648056, "raw_key_size": 78468695, "raw_average_key_size": 54, "raw_value_size": 8019945, "raw_average_value_size": 5, "num_data_blocks": 8338, "num_entries": 1428525, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "1366998", "kMergeOperands": "0"}}
2018-12-19 04:26:36.660564 7fa0ba39e700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357362: 97270 keys, 34463615 bytes
2018-12-19 04:26:36.660610 7fa0ba39e700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796660588, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357362, "file_size": 34463615, "table_properties": {"data_size": 33555257, "index_size": 585974, "filter_size": 321398, "raw_key_size": 7624037, "raw_average_key_size": 78, "raw_value_size": 29409785, "raw_average_value_size": 302, "num_data_blocks": 8759, "num_entries": 97270, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:36.667111 7fa0bb3a0700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357361: 97611 keys, 34464174 bytes
2018-12-19 04:26:36.667142 7fa0bb3a0700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796667126, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357361, "file_size": 34464174, "table_properties": {"data_size": 33558528, "index_size": 582826, "filter_size": 321834, "raw_key_size": 7607498, "raw_average_key_size": 77, "raw_value_size": 29439593, "raw_average_value_size": 301, "num_data_blocks": 8722, "num_entries": 97611, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "19", "kMergeOperands": "0"}}
2018-12-19 04:26:36.708933 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357363: 86544 keys, 3703034 bytes
2018-12-19 04:26:36.708974 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796708952, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357363, "file_size": 3703034, "table_properties": {"data_size": 3420867, "index_size": 57080, "filter_size": 224101, "raw_key_size": 6834203, "raw_average_key_size": 78, "raw_value_size": 1161, "raw_average_value_size": 0, "num_data_blocks": 848, "num_entries": 86544, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "86539", "kMergeOperands": "0"}}
2018-12-19 04:26:36.995042 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357364: 97660 keys, 34464021 bytes
2018-12-19 04:26:36.995086 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164796995063, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357364, "file_size": 34464021, "table_properties": {"data_size": 33556462, "index_size": 584567, "filter_size": 322005, "raw_key_size": 7623733, "raw_average_key_size": 78, "raw_value_size": 29435632, "raw_average_value_size": 301, "num_data_blocks": 8724, "num_entries": 97660, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "137", "kMergeOperands": "0"}}
2018-12-19 04:26:37.241830 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4780] Generated table #357365: 97427 keys, 34463446 bytes
2018-12-19 04:26:37.241877 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797241854, "cf_name": "default", "job": 4780, "event": "table_file_creation", "file_number": 357365, "file_size": 34463446, "table_properties": {"data_size": 33556089, "index_size": 584645, "filter_size": 321726, "raw_key_size": 7614898, "raw_average_key_size": 78, "raw_value_size": 29412101, "raw_average_value_size": 301, "num_data_blocks": 8761, "num_entries": 97427, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:37.241998 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1173] [default] [JOB 4780] Compacted 1@0 + 12@1 files to L1 => 388985347 bytes
2018-12-19 04:26:37.249199 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:37.248437) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 375809638 files[0 14 118 1117 2993 0 0] max score 2.77, MB/sec: 189.3 rd, 189.3 wr, level 1, files in(1, 12) out(14) MB in(58.6, 312.4) out(371.0), read-write-amplify(12.7) write-amplify(6.3) OK, records in: 2498658, records dr
2018-12-19 04:26:37.249218 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:37.248485) EVENT_LOG_v1 {"time_micros": 1545164797248464, "job": 4780, "event": "compaction_finished", "compaction_time_micros": 2055398, "output_level": 1, "num_output_files": 14, "total_output_size": 388985347, "num_input_records": 2498660, "num_output_records": 2498584, "num_subcompactions": 4, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 14, 118, 1117, 2993, 0, 0]}
2018-12-19 04:26:37.250431 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250424, "job": 4780, "event": "table_file_deletion", "file_number": 357351}
2018-12-19 04:26:37.250459 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250456, "job": 4780, "event": "table_file_deletion", "file_number": 357304}
2018-12-19 04:26:37.250478 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250476, "job": 4780, "event": "table_file_deletion", "file_number": 357303}
2018-12-19 04:26:37.250500 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250497, "job": 4780, "event": "table_file_deletion", "file_number": 357302}
2018-12-19 04:26:37.250524 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250522, "job": 4780, "event": "table_file_deletion", "file_number": 357300}
2018-12-19 04:26:37.250545 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250543, "job": 4780, "event": "table_file_deletion", "file_number": 357299}
2018-12-19 04:26:37.250562 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250560, "job": 4780, "event": "table_file_deletion", "file_number": 357298}
2018-12-19 04:26:37.250581 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250579, "job": 4780, "event": "table_file_deletion", "file_number": 357297}
2018-12-19 04:26:37.250602 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250600, "job": 4780, "event": "table_file_deletion", "file_number": 357296}
2018-12-19 04:26:37.250622 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250620, "job": 4780, "event": "table_file_deletion", "file_number": 357295}
2018-12-19 04:26:37.250645 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250643, "job": 4780, "event": "table_file_deletion", "file_number": 357294}
2018-12-19 04:26:37.250669 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250667, "job": 4780, "event": "table_file_deletion", "file_number": 357293}
2018-12-19 04:26:37.250691 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797250688, "job": 4780, "event": "table_file_deletion", "file_number": 357291}

通过读出level0与level1中的sst文件并进行合并，排序后依次产生了编号为357352 - 357365的新的sst文件，共14个，从而合并完成后，level1共有14个文件，此时lsm_state为**[0, 14, 118, 1117, 2993, 0, 0]**，合并完成后，原来level1的旧文件被删除，这次参与合并的文件，输入输出大小为:in(58.6, 312.4) out(371.0)，从而读写放大情况为read-write-amplify(12.7) write-amplify(6.3)；这里啰嗦一下，从日志中看到，合并level0与level1时，显示“num_subcompactions”: 4，这个值可以再稍微调大点，加快这个过程

1
2
3
2018-12-19 04:26:37.255854 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:37.250949) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1509] [default] Moving #357352 to level-2 37746293 bytes
2018-12-19 04:26:37.255871 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:37.255654) EVENT_LOG_v1 {"time_micros": 1545164797255644, "job": 4781, "event": "trivial_move", "destination_level": 2, "files": 1, "total_files_size": 37746293}
2018-12-19 04:26:37.255880 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:37.255668) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/db_impl_compaction_flush.cc:1537] [default] Moved #1 files to level-2 37746293 bytes OK: base level 1 max bytes base 375809638 files[0 13 119 1117 2993 0 0] max score 1.15

leve0与level1合并完成后，发现有sst文件可以直接加入到level2，所以不通过合并而使用move的方式使某个level的sst文件进入下一个level，在下个level为空的时候常常发生这种情况，在下个level不空的时候也有可能触发这种情况，虽然并不常见；移动文件后，level1的文件数变成13，而level2的文件数变成119，score超过了1，为1.15，将触发level2与level3的文件合并

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
2018-12-19 04:26:37.257102 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 4782] Compacting 1@2 + 6@3 files to L3, score 1.15
2018-12-19 04:26:37.257124 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 4781 Base level 2, inputs: [357352(35MB)], [357318(583KB) 357319(32MB) 357320(32MB) 357321(1042B) 357322(2671KB) 357324(4258KB)]
2018-12-19 04:26:37.257148 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164797257132, "job": 4782, "event": "compaction_started", "files_L2": [357352], "files_L3": [357318, 357319, 357320, 357321, 357322, 357324], "score": 1.1476, "input_data_size": 114387991}
2018-12-19 04:26:38.174969 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357366: 1286630 keys, 31675111 bytes
2018-12-19 04:26:38.175015 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798174993, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357366, "file_size": 31675111, "table_properties": {"data_size": 27945520, "index_size": 447928, "filter_size": 3280673, "raw_key_size": 66924251, "raw_average_key_size": 52, "raw_value_size": 8349321, "raw_average_value_size": 6, "num_data_blocks": 6950, "num_entries": 1286630, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "1222400", "kMergeOperands": "0"}}
2018-12-19 04:26:38.422904 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357367: 97847 keys, 34473171 bytes
2018-12-19 04:26:38.422950 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798422926, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357367, "file_size": 34473171, "table_properties": {"data_size": 33556364, "index_size": 593290, "filter_size": 322531, "raw_key_size": 7655159, "raw_average_key_size": 78, "raw_value_size": 29482212, "raw_average_value_size": 301, "num_data_blocks": 8750, "num_entries": 97847, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "86", "kMergeOperands": "0"}}
2018-12-19 04:26:38.630370 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357368: 97722 keys, 34473304 bytes
2018-12-19 04:26:38.630393 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798630381, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357368, "file_size": 34473304, "table_properties": {"data_size": 33557943, "index_size": 592146, "filter_size": 322229, "raw_key_size": 7636886, "raw_average_key_size": 78, "raw_value_size": 29488092, "raw_average_value_size": 301, "num_data_blocks": 8748, "num_entries": 97722, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:38.639662 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357369: 1 keys, 1042 bytes
2018-12-19 04:26:38.639705 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798639682, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357369, "file_size": 1042, "table_properties": {"data_size": 35, "index_size": 27, "filter_size": 18, "raw_key_size": 19, "raw_average_key_size": 19, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 1, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "1", "kMergeOperands": "0"}}
2018-12-19 04:26:38.696361 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357370: 63887 keys, 2736124 bytes
2018-12-19 04:26:38.696404 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798696382, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357370, "file_size": 2736124, "table_properties": {"data_size": 2527764, "index_size": 41957, "filter_size": 165417, "raw_key_size": 5041766, "raw_average_key_size": 78, "raw_value_size": 186, "raw_average_value_size": 0, "num_data_blocks": 627, "num_entries": 63887, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "63885", "kMergeOperands": "0"}}
2018-12-19 04:26:38.801866 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357371: 135001 keys, 5749369 bytes
2018-12-19 04:26:38.801909 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798801887, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357371, "file_size": 5749369, "table_properties": {"data_size": 5310161, "index_size": 88735, "filter_size": 349487, "raw_key_size": 10629259, "raw_average_key_size": 78, "raw_value_size": 372, "raw_average_value_size": 0, "num_data_blocks": 1316, "num_entries": 135001, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "134997", "kMergeOperands": "0"}}
2018-12-19 04:26:38.915027 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4782] Generated table #357372: 124094 keys, 5279290 bytes
2018-12-19 04:26:38.915071 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798915049, "cf_name": "default", "job": 4782, "event": "table_file_creation", "file_number": 357372, "file_size": 5279290, "table_properties": {"data_size": 4875991, "index_size": 81065, "filter_size": 321248, "raw_key_size": 9730626, "raw_average_key_size": 78, "raw_value_size": 186, "raw_average_value_size": 0, "num_data_blocks": 1208, "num_entries": 124094, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "124092", "kMergeOperands": "0"}}
2018-12-19 04:26:38.915119 7fa0e3bf1700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1173] [default] [JOB 4782] Compacted 1@2 + 6@3 files to L3 => 114387411 bytes
2018-12-19 04:26:38.923642 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:38.921931) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 375809638 files[0 13 118 1118 2993 0 0] max score 1.04, MB/sec: 69.0 rd, 69.0 wr, level 3, files in(1, 6) out(7) MB in(36.0, 73.1) out(109.1), read-write-amplify(6.1) write-amplify(3.0) OK, records in: 1805186, records dropped:
2018-12-19 04:26:38.923666 7fa0e3bf1700  4 rocksdb: (Original Log Time 2018/12/19-04:26:38.921986) EVENT_LOG_v1 {"time_micros": 1545164798921962, "job": 4782, "event": "compaction_finished", "compaction_time_micros": 1657943, "output_level": 3, "num_output_files": 7, "total_output_size": 114387411, "num_input_records": 1805186, "num_output_records": 1805182, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 13, 118, 1118, 2993, 0, 0]}
2018-12-19 04:26:38.924861 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924853, "job": 4782, "event": "table_file_deletion", "file_number": 357352}
2018-12-19 04:26:38.924889 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924886, "job": 4782, "event": "table_file_deletion", "file_number": 357324}
2018-12-19 04:26:38.924907 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924904, "job": 4782, "event": "table_file_deletion", "file_number": 357322}
2018-12-19 04:26:38.924924 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924921, "job": 4782, "event": "table_file_deletion", "file_number": 357321}
2018-12-19 04:26:38.924942 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924940, "job": 4782, "event": "table_file_deletion", "file_number": 357320}
2018-12-19 04:26:38.924961 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924958, "job": 4782, "event": "table_file_deletion", "file_number": 357319}
2018-12-19 04:26:38.924982 7fa0e3bf1700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798924979, "job": 4782, "event": "table_file_deletion", "file_number": 357318}

类似的，rocksdb从level2选择了一个文件，从level3选择了6个文件进行compaction，共产生了7个文件，所以level3在合并完成后文件数加1，最后删除了旧的sst文件，level2文件数减1，此时lsm_state为**[0, 13, 118, 1118, 2993, 0, 0]，本次合并读写放大为：read-write-amplify(6.1) write-amplify(3.0)**

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
2018-12-19 04:26:38.926154 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 4783] Compacting 1@1 + 11@2 files to L2, score 1.04
2018-12-19 04:26:38.926185 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 4782 Base level 1, inputs: [357363(3616KB)], [357306(32MB) 357307(1358KB) 357308(14KB) 357309(32MB) 357310(32MB) 357311(32MB) 357312(32MB) 357313(1138B) 357314(32MB) 357315(103KB) 357316(32MB)]
2018-12-19 04:26:38.926214 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164798926194, "job": 4783, "event": "compaction_started", "files_L1": [357363], "files_L2": [357306, 357307, 357308, 357309, 357310, 357311, 357312, 357313, 357314, 357315, 357316], "score": 1.04238, "input_data_size": 245745887}
2018-12-19 04:26:39.017142 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357373: 86539 keys, 3701676 bytes
2018-12-19 04:26:39.017191 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799017167, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357373, "file_size": 3701676, "table_properties": {"data_size": 3419527, "index_size": 57080, "filter_size": 224084, "raw_key_size": 6833968, "raw_average_key_size": 78, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 848, "num_entries": 86539, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "86539", "kMergeOperands": "0"}}
2018-12-19 04:26:39.263972 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357374: 97730 keys, 34317981 bytes
2018-12-19 04:26:39.264015 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799263992, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357374, "file_size": 34317981, "table_properties": {"data_size": 33412493, "index_size": 582644, "filter_size": 321857, "raw_key_size": 7614219, "raw_average_key_size": 77, "raw_value_size": 29288975, "raw_average_value_size": 299, "num_data_blocks": 8710, "num_entries": 97730, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "337", "kMergeOperands": "0"}}
2018-12-19 04:26:39.298012 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357375: 36496 keys, 1391303 bytes
2018-12-19 04:26:39.298057 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799298034, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357375, "file_size": 1391303, "table_properties": {"data_size": 1279551, "index_size": 16628, "filter_size": 94142, "raw_key_size": 2253130, "raw_average_key_size": 61, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 316, "num_entries": 36496, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "36496", "kMergeOperands": "0"}}
2018-12-19 04:26:39.338609 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357376: 183 keys, 14428 bytes
2018-12-19 04:26:39.338652 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799338629, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357376, "file_size": 14428, "table_properties": {"data_size": 12779, "index_size": 186, "filter_size": 491, "raw_key_size": 11731, "raw_average_key_size": 64, "raw_value_size": 4092, "raw_average_value_size": 22, "num_data_blocks": 4, "num_entries": 183, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "139", "kMergeOperands": "0"}}
2018-12-19 04:26:39.584140 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357377: 97069 keys, 34420591 bytes
2018-12-19 04:26:39.584185 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799584161, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357377, "file_size": 34420591, "table_properties": {"data_size": 33507241, "index_size": 591084, "filter_size": 321279, "raw_key_size": 7620733, "raw_average_key_size": 78, "raw_value_size": 29372913, "raw_average_value_size": 302, "num_data_blocks": 8803, "num_entries": 97069, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "152", "kMergeOperands": "0"}}
2018-12-19 04:26:39.826577 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357378: 96767 keys, 34423501 bytes
2018-12-19 04:26:39.826623 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164799826599, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357378, "file_size": 34423501, "table_properties": {"data_size": 33510279, "index_size": 591931, "filter_size": 320305, "raw_key_size": 7655211, "raw_average_key_size": 79, "raw_value_size": 29331477, "raw_average_value_size": 303, "num_data_blocks": 8810, "num_entries": 96767, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:40.070646 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357379: 96072 keys, 33888290 bytes
2018-12-19 04:26:40.070692 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800070668, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357379, "file_size": 33888290, "table_properties": {"data_size": 32992372, "index_size": 577729, "filter_size": 317203, "raw_key_size": 7495430, "raw_average_key_size": 78, "raw_value_size": 28918019, "raw_average_value_size": 301, "num_data_blocks": 8659, "num_entries": 96072, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "1", "kMergeOperands": "0"}}
2018-12-19 04:26:40.279863 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357380: 97445 keys, 34455775 bytes
2018-12-19 04:26:40.279907 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800279885, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357380, "file_size": 34455775, "table_properties": {"data_size": 33555167, "index_size": 577891, "filter_size": 321731, "raw_key_size": 7622517, "raw_average_key_size": 78, "raw_value_size": 29341840, "raw_average_value_size": 301, "num_data_blocks": 8755, "num_entries": 97445, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "23", "kMergeOperands": "0"}}
2018-12-19 04:26:40.318228 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357381: 1 keys, 1138 bytes
2018-12-19 04:26:40.318271 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800318248, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357381, "file_size": 1138, "table_properties": {"data_size": 128, "index_size": 27, "filter_size": 18, "raw_key_size": 19, "raw_average_key_size": 19, "raw_value_size": 93, "raw_average_value_size": 93, "num_data_blocks": 1, "num_entries": 1, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:40.532176 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357382: 96550 keys, 34468123 bytes
2018-12-19 04:26:40.532223 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800532200, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357382, "file_size": 34468123, "table_properties": {"data_size": 33556566, "index_size": 590646, "filter_size": 319925, "raw_key_size": 7662812, "raw_average_key_size": 79, "raw_value_size": 29359353, "raw_average_value_size": 304, "num_data_blocks": 8807, "num_entries": 96550, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:40.575280 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357383: 2288 keys, 106362 bytes
2018-12-19 04:26:40.575323 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800575300, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357383, "file_size": 106362, "table_properties": {"data_size": 97957, "index_size": 1480, "filter_size": 5949, "raw_key_size": 178824, "raw_average_key_size": 78, "raw_value_size": 2511, "raw_average_value_size": 1, "num_data_blocks": 25, "num_entries": 2288, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "2261", "kMergeOperands": "0"}}
2018-12-19 04:26:40.807200 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4783] Generated table #357384: 135510 keys, 34556650 bytes
2018-12-19 04:26:40.807246 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800807222, "cf_name": "default", "job": 4783, "event": "table_file_creation", "file_number": 357384, "file_size": 34556650, "table_properties": {"data_size": 33556399, "index_size": 582125, "filter_size": 417138, "raw_key_size": 10614250, "raw_average_key_size": 78, "raw_value_size": 27843428, "raw_average_value_size": 205, "num_data_blocks": 8765, "num_entries": 135510, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "43068", "kMergeOperands": "0"}}
2018-12-19 04:26:40.807310 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1173] [default] [JOB 4783] Compacted 1@1 + 11@2 files to L2 => 245745818 bytes
2018-12-19 04:26:40.813403 7fa0e2bef700  4 rocksdb: (Original Log Time 2018/12/19-04:26:40.812754) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 375809638 files[0 12 119 1118 2993 0 0] max score 1.01, MB/sec: 130.6 rd, 130.6 wr, level 2, files in(1, 11) out(12) MB in(3.5, 230.8) out(234.4), read-write-amplify(132.7) write-amplify(66.4) OK, records in: 842651, records dr
2018-12-19 04:26:40.813426 7fa0e2bef700  4 rocksdb: (Original Log Time 2018/12/19-04:26:40.812801) EVENT_LOG_v1 {"time_micros": 1545164800812781, "job": 4783, "event": "compaction_finished", "compaction_time_micros": 1881052, "output_level": 2, "num_output_files": 12, "total_output_size": 245745818, "num_input_records": 842651, "num_output_records": 842650, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 12, 119, 1118, 2993, 0, 0]}
2018-12-19 04:26:40.814973 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800814965, "job": 4783, "event": "table_file_deletion", "file_number": 357363}
2018-12-19 04:26:40.815006 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815003, "job": 4783, "event": "table_file_deletion", "file_number": 357316}
2018-12-19 04:26:40.815029 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815027, "job": 4783, "event": "table_file_deletion", "file_number": 357315}
2018-12-19 04:26:40.815050 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815048, "job": 4783, "event": "table_file_deletion", "file_number": 357314}
2018-12-19 04:26:40.815070 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815067, "job": 4783, "event": "table_file_deletion", "file_number": 357313}
2018-12-19 04:26:40.815087 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815085, "job": 4783, "event": "table_file_deletion", "file_number": 357312}
2018-12-19 04:26:40.815108 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815105, "job": 4783, "event": "table_file_deletion", "file_number": 357311}
2018-12-19 04:26:40.815130 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815127, "job": 4783, "event": "table_file_deletion", "file_number": 357310}
2018-12-19 04:26:40.815150 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815147, "job": 4783, "event": "table_file_deletion", "file_number": 357309}
2018-12-19 04:26:40.815170 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815167, "job": 4783, "event": "table_file_deletion", "file_number": 357308}
2018-12-19 04:26:40.815191 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815188, "job": 4783, "event": "table_file_deletion", "file_number": 357307}
2018-12-19 04:26:40.815213 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815210, "job": 4783, "event": "table_file_deletion", "file_number": 357306}

接上，判断level1的score为1.04，触发compaction，值得一提的是本次的读写放大情况为：read-write-amplify(132.7) write-amplify(66.4)，呵呵，开始变得夸张起来，合并完成后lsm_state为[0, 12, 119, 1118, 2993, 0, 0]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
2018-12-19 04:26:40.815612 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1403] [default] [JOB 4784] Compacting 1@3 + 4@4 files to L4, score 1.01
2018-12-19 04:26:40.815630 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1407] [default] Compaction start summary: Base version 4783 Base level 3, inputs: [357366(30MB)], [357326(33MB) 357327(32MB) 357255(33MB) 357256(32MB)]
2018-12-19 04:26:40.815654 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164800815637, "job": 4784, "event": "compaction_started", "files_L3": [357366], "files_L4": [357326, 357327, 357255, 357256], "score": 1.01321, "input_data_size": 169855357}
2018-12-19 04:26:41.280397 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4784] Generated table #357385: 227167 keys, 34739884 bytes
2018-12-19 04:26:41.280443 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164801280419, "cf_name": "default", "job": 4784, "event": "table_file_creation", "file_number": 357385, "file_size": 34739884, "table_properties": {"data_size": 33557037, "index_size": 538833, "filter_size": 643028, "raw_key_size": 11786808, "raw_average_key_size": 51, "raw_value_size": 29632837, "raw_average_value_size": 130, "num_data_blocks": 8471, "num_entries": 227167, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:41.690487 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4784] Generated table #357386: 216591 keys, 34733017 bytes
2018-12-19 04:26:41.690534 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164801690511, "cf_name": "default", "job": 4784, "event": "table_file_creation", "file_number": 357386, "file_size": 34733017, "table_properties": {"data_size": 33558511, "index_size": 556531, "filter_size": 616989, "raw_key_size": 11509650, "raw_average_key_size": 53, "raw_value_size": 29673634, "raw_average_value_size": 137, "num_data_blocks": 8487, "num_entries": 216591, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:41.934608 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4784] Generated table #357387: 87575 keys, 34754209 bytes
2018-12-19 04:26:41.934663 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164801934632, "cf_name": "default", "job": 4784, "event": "table_file_creation", "file_number": 357387, "file_size": 34754209, "table_properties": {"data_size": 33557954, "index_size": 902014, "filter_size": 293255, "raw_key_size": 10007753, "raw_average_key_size": 114, "raw_value_size": 29473394, "raw_average_value_size": 336, "num_data_blocks": 8759, "num_entries": 87575, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:42.135047 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1116] [default] [JOB 4784] Generated table #357388: 103009 keys, 34148460 bytes
2018-12-19 04:26:42.135093 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802135070, "cf_name": "default", "job": 4784, "event": "table_file_creation", "file_number": 357388, "file_size": 34148460, "table_properties": {"data_size": 33349993, "index_size": 465324, "filter_size": 332157, "raw_key_size": 6410636, "raw_average_key_size": 62, "raw_value_size": 29243968, "raw_average_value_size": 283, "num_data_blocks": 8682, "num_entries": 103009, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}}
2018-12-19 04:26:42.135327 7fa0e2bef700  4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:1173] [default] [JOB 4784] Compacted 1@3 + 4@4 files to L4 => 138375570 bytes
2018-12-19 04:26:42.141246 7fa0e2bef700  4 rocksdb: (Original Log Time 2018/12/19-04:26:42.140832) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.7/rpm/el7/BUILD/ceph-12.2.7/src/rocksdb/db/compaction_job.cc:621] [default] compacted to: base level 1 max bytes base 375809638 files[0 12 119 1117 2993 0 0] max score 1.00, MB/sec: 128.7 rd, 104.9 wr, level 4, files in(1, 4) out(4) MB in(30.2, 131.8) out(132.0), read-write-amplify(9.7) write-amplify(4.4) OK, records in: 1920098, records dropp
2018-12-19 04:26:42.141268 7fa0e2bef700  4 rocksdb: (Original Log Time 2018/12/19-04:26:42.140875) EVENT_LOG_v1 {"time_micros": 1545164802140858, "job": 4784, "event": "compaction_finished", "compaction_time_micros": 1319648, "output_level": 4, "num_output_files": 4, "total_output_size": 138375570, "num_input_records": 1920098, "num_output_records": 634342, "num_subcompactions": 1, "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 12, 119, 1117, 2993, 0, 0]}
2018-12-19 04:26:42.142498 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802142491, "job": 4784, "event": "table_file_deletion", "file_number": 357366}
2018-12-19 04:26:42.142525 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802142522, "job": 4784, "event": "table_file_deletion", "file_number": 357327}
2018-12-19 04:26:42.142543 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802142541, "job": 4784, "event": "table_file_deletion", "file_number": 357326}
2018-12-19 04:26:42.142568 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802142566, "job": 4784, "event": "table_file_deletion", "file_number": 357256}
2018-12-19 04:26:42.142589 7fa0e2bef700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1545164802142587, "job": 4784, "event": "table_file_deletion", "file_number": 357255}

level3 -> level4的compaction，score为1.01

总结

rocksdb的compaction过程本身还是比较简单的，其核心就是lsm，每次触发compaction的条件都搞清楚，并分析出各个过程产生的缺陷，就可以在一定程度上通过硬件的配置、参数的调整来适应实际环境的使用，例如compaction时产生极高的io请求，此时如果可以保证wal和memtable写入流畅，则不会影响写入

参考资料

赏

谢谢你请我吃糖果

支付宝
微信
  • 本文作者： 奋斗的松鼠
  • 本文链接： http://www.strugglesquirrel.com/2019/01/02/看懂ceph日志系列之rocksdb/
  • 版权声明： 本博客所有文章除特别声明外，创作版权均为作者个人所有，未经允许禁止转载！

扫一扫，分享到微信

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia-plus根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

松鼠哥，存储研发专家
曾就职于深信服北京研发中心，负责存储相关产品的调研测试
在分布式存储领域深耕近十年，实战经验和理论经验功力深厚
累计管理及上线生产集群超过600个，机器超3万台，磁盘总数超过80万块，总容量达到EB级别。