-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
10 changed files
with
195 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Submodule rocksdb
updated
20 files
+7 −0 | cmake/modules/CxxFlags.cmake | |
+29 −0 | cmake/modules/FindJeMalloc.cmake | |
+29 −0 | cmake/modules/FindNUMA.cmake | |
+29 −0 | cmake/modules/FindSnappy.cmake | |
+33 −0 | cmake/modules/FindTBB.cmake | |
+29 −0 | cmake/modules/Findgflags.cmake | |
+29 −0 | cmake/modules/Findlz4.cmake | |
+26 −0 | cmake/modules/Finduring.cmake | |
+29 −0 | cmake/modules/Findzstd.cmake | |
+10 −0 | cmake/modules/ReadVersion.cmake | |
+36 −0 | db/db_impl/db_impl.cc | |
+2 −0 | db/db_impl/db_impl.h | |
+3 −2 | db/db_impl/db_impl_open.cc | |
+4 −0 | db/write_batch.cc | |
+2 −1 | env/env_posix.cc | |
+4 −1 | env/file_system_tracer.cc | |
+36 −0 | ycsb_trace/time_test.cpp | |
+4 −4 | ycsb_trace/trace_analyze.sh | |
+58 −58 | ycsb_trace/ycsb_run.cc | |
+4 −4 | ycsb_trace/ycsb_run.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
Something weird, | ||
|
||
|
||
I found out that there is no db end tracing log . | ||
|
||
|
||
There is 'end tracing ' initial install phase. | ||
The rocksdb instance is shutdown gracefully . | ||
|
||
How can I run a graceful shutdown. | ||
|
||
|
||
|
||
- Gracefully shutdown a process | ||
https://stackoverflow.com/questions/690415/in-what-order-should-i-send-signals-to-gracefully-shutdown-processes | ||
|
||
`kill -15 $pid` | ||
|
||
Now there is end tracing log in the LOG | ||
|
||
I need to check the startTrace code to see if the new trace file will overwrite the existing one. | ||
|
||
But there is still error while parsing op trace file. | ||
|
||
|
||
I check the `env/fs_posix.cc` | ||
The file will be truncated. | ||
|
||
|
||
## Fix a bug which I don't know why | ||
|
||
- Bug description | ||
When I run `trace_analyzer` to parse the op_trace_file, the | ||
`Iterate()` function in `write_batch.cc` reports error while executing | ||
`MarkNoop()` function, the handler does not over write the `MarkNoop()` | ||
function from `Handler`, so it calls methods from parents which returns | ||
invalid status, thus causing a error. | ||
|
||
|
||
- My fix | ||
I just add the `MarkNoop()` override function in the `Classifier` Handler. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
|
||
|
||
|
||
# Table format | ||
op_human_readable_trace | ||
|
||
[type_id cf_id value_size time_in_micorsec <key>] | ||
|
||
[key, type_id, cf_id, value_size , time in microsec] | ||
|
||
Let's just plot something here. | ||
|
||
using 5:4 | ||
|
||
|
||
|
||
- block cache human file | ||
|
||
Access timestamp: 1 | ||
Block ID: 2 | ||
Level: 7 | ||
SST file number: 8 | ||
Cache hit : 14 | ||
|
||
|
||
|
||
- io_trace_res | ||
|
||
time_stamp : 1 | ||
file_op : 2 | ||
latency : 3 | ||
io status : 4 | ||
file name : 5 | ||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
|
||
|
||
I think I found a bug ? | ||
|
||
The timestamp in io trace file does not make sense | ||
record timestamp:100853315672962 | ||
|
||
|
||
start timstamp: 1661002238043735 | ||
|
||
time I get with the bash command `date +%s%N` | ||
1661073847698401833 | ||
|
||
Turns out that record timestamp is return by NowNanos() | ||
And start timestamp is return by NowMicros(); | ||
|
||
I don't know why the it is, I think it's a bug. | ||
|
||
So I will just submit a pr, fix this. | ||
|
||
|
||
How linux get nano time | ||
``` | ||
uint64_t NowNanos() override { | ||
#if defined(OS_LINUX) || defined(OS_FREEBSD) || defined(OS_GNU_KFREEBSD) || \ | ||
defined(OS_AIX) | ||
struct timespec ts; | ||
clock_gettime(CLOCK_MONOTONIC, &ts); | ||
return static_cast<uint64_t>(ts.tv_sec) * 1000000000 + ts.tv_nsec; | ||
#elif defined(OS_SOLARIS) | ||
``` | ||
|
||
|
||
|
||
Check the source code and run a test by myself, | ||
|
||
Found that it's because there is subttle difference. | ||
|
||
``` | ||
# define CLOCK_REALTIME 0 | ||
/* Monotonic system-wide clock. */ | ||
# define CLOCK_MONOTONIC 1 | ||
/* | ||
``` | ||
|
||
|
||
``` | ||
uint64_t NowNanos() { | ||
struct timespec ts; | ||
clock_gettime(CLOCK_MONOTONIC, &ts); | ||
printf("seconds: %lu\n", static_cast<uint64_t>(ts.tv_sec)); | ||
return (static_cast<uint64_t>(ts.tv_sec) * 1000000000) + ts.tv_nsec; | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
|
||
sysbench_dir="/home/bily/sysbench/sysbench/src/lua" | ||
|
||
./install.pl | ||
|
||
|
||
./start_server.pl | ||
|
||
pushd $sysbench_dir | ||
|
||
sleep 3 | ||
|
||
|
||
sysbench ./oltp_read_write.lua --mysql-port=3306 \ | ||
--mysql-user=root --mysql-db=test \ | ||
--tables=4 --table_size=1000000 \ | ||
--mysql_storage_engine=rocksdb \ | ||
--mysql-socket=/tmp/mysql.sock \ | ||
--threads=64 prepare | ||
|
||
|
||
sysbench ./oltp_read_write.lua --mysql-port=3306 \ | ||
--mysql-user=root --mysql-db=test \ | ||
--tables=4 --table_size=1000000 \ | ||
--mysql_storage_engine=rocksdb \ | ||
--mysql-socket=/tmp/mysql.sock \ | ||
--report-interval=3 \ | ||
--time=60 \ | ||
--threads=64 run | ||
popd | ||
|
||
./shut_server.pl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,6 +10,6 @@ | |
chomp($pid); | ||
|
||
|
||
kill 9,$pid; | ||
kill 15,$pid; | ||
|
||
|