-
Notifications
You must be signed in to change notification settings - Fork 41
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: migrate from MySQL in v0.9 (#1258)
- Loading branch information
Showing
3 changed files
with
181 additions
and
1 deletion.
There are no files selected for viewing
83 changes: 83 additions & 0 deletions
83
...content-docs/version-0.9/user-guide/migrate-to-greptimedb/migrate-from-mysql.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# 从 MySQL 迁移 | ||
|
||
本文档将指引您完成从 MySQL 迁移到 GreptimeDB。 | ||
|
||
## 在开始迁移之前 | ||
|
||
请注意,尽管 GreptimeDB 支持 MySQL 的协议,并不意味着 GreptimeDB 实现了 MySQL | ||
的所有功能。你可以参考 "[ANSI 兼容性](/reference/sql/compatibility.md)" 查看在 GreptimeDB 中使用 SQL 的约束。 | ||
|
||
## 迁移步骤 | ||
|
||
### 在 GreptimeDB 中创建数据库和表 | ||
|
||
在从 MySQL 迁移数据之前,你首先需要在 GreptimeDB 中创建相应的数据库和表。 | ||
由于 GreptimeDB 有自己的 SQL 语法用于创建表,因此你不能直接重用 MySQL 生成的建表 SQL。 | ||
|
||
当你为 GreptimeDB 编写创建表的 SQL 时,首先请了解其“[数据模型](/user-guide/concepts/data-model.md)”。然后,在创建表的 | ||
SQL 中请考虑以下几点: | ||
|
||
1. 由于 time index 列在表创建后无法更改,所以你需要仔细选择 time index | ||
列。时间索引最好设置为数据生成时的自然时间戳,因为它提供了查询数据的最直观方式,以及最佳的查询性能。例如,在 IOT | ||
场景中,你可以使用传感器采集数据时的时间作为 time index;或者在可观测场景中使用事件的发生时间。 | ||
2. 不建议在此迁移过程中另造一个时间戳用作时间索引,例如使用 `DEFAULT current_timestamp()` 创建的新列。也不建议使用具有随机时间戳的列。 | ||
3. 选择合适的 time index 精度也至关重要。和 time index 的选择一样,一旦表创建完毕,time index | ||
的精度就无法变更了。请根据你的数据集在[这里](/reference/sql/data-types#data-types-compatible-with-mysql-and-postgresql) | ||
找到最适合的时间戳类型。 | ||
4. 根据您的查询模式选择最适合的 tag 列。tag 列存储经常被查询的元数据,其中的值是数据源的标签,通常用于描述数据的特征。tag 列具有索引,所以使用 tag 列的查询具备良好的性能。 | ||
|
||
请参考[CREATE](/reference/sql/create.md) SQL 文档,了解如何选择正确的数据类型以及“ttl”或“compaction”选项等。 | ||
|
||
### 双写 GreptimeDB 和 MySQL | ||
|
||
双写 GreptimeDB 和 MySQL 是迁移过程中防止数据丢失的有效策略。通过使用 MySQL 的客户端库(JDBC + 某个 MySQL | ||
驱动),你可以建立两个客户端实例 —— 一个用于 GreptimeDB,另一个用于 MySQL。有关如何使用 SQL 将数据写入 | ||
GreptimeDB,请参考[写入数据](/user-guide/ingest-data/for-iot/sql.md)部分。 | ||
|
||
若无需保留所有历史数据,你可以双写一段时间以积累业务所需的最新数据,然后停止向 MySQL 写入数据并仅使用 | ||
GreptimeDB。如果需要完整迁移所有历史数据,请按照接下来的步骤操作。 | ||
|
||
### 从 MySQL 导出数据 | ||
|
||
[mysqldump](https://dev.mysql.com/doc/refman/8.4/en/mysqldump.html) 是一个常用的、从 MySQL 导出数据的工具。使用 | ||
mysqldump,我们可以从 MySQL 中导出后续可直接导入到 GreptimeDB 的数据。例如,如果我们想要从 MySQL 导出两个数据库 `db1` 和 | ||
`db2`,我们可以使用以下命令: | ||
|
||
```bash | ||
mysqldump -h127.0.0.1 -P3306 -umysql_user -p --compact -cnt --skip-extended-insert --databases db1 db2 > /path/to/output.sql | ||
``` | ||
|
||
替换 `-h`、`-P` 和 `-u` 参数为 MySQL 服务的正确值。`--databases` 参数用于指定要导出的数据库。输出将写入 | ||
`/path/to/output.sql` 文件。 | ||
|
||
`/path/to/output.sql` 文件应该具有如下内容: | ||
|
||
```plaintext | ||
~ ❯ cat /path/to/output.sql | ||
USE `db1`; | ||
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (1,'hello',1); | ||
INSERT INTO ... | ||
USE `db2`; | ||
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (2,'greptime',2); | ||
INSERT INTO ... | ||
``` | ||
|
||
### 将数据导入 GreptimeDB | ||
|
||
[MySQL Command-Line Client](https://dev.mysql.com/doc/refman/8.4/en/mysql.html) 可用于将数据导入 | ||
GreptimeDB。继续上面的示例,假设数据导出到文件 `/path/to/output.sql`,那么我们可以使用以下命令将数据导入 GreptimeDB: | ||
|
||
```bash | ||
mysql -h127.0.0.1 -P4002 -ugreptime_user -p -e "source /path/to/output.sql" | ||
``` | ||
|
||
替换 `-h`、`-P` 和 `-u` 参数为你的 GreptimeDB 服务的值。`source` 命令用于执行 `/path/to/output.sql` 文件中的 SQL | ||
命令。若需要进行 debug,添加 `-vvv` 以查看详细的执行结果。 | ||
|
||
总结一下,数据迁移步骤如下图所示: | ||
|
||
 | ||
|
||
数据迁移完成后,你可以停止向 MySQL 写入数据,并继续使用 GreptimeDB! |
96 changes: 96 additions & 0 deletions
96
versioned_docs/version-0.9/user-guide/migrate-to-greptimedb/migrate-from-mysql.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
# Migrate from MySQL | ||
|
||
This document will guide you through the migration process from MySQL to GreptimeDB. | ||
|
||
## Before you start the migration | ||
|
||
Please be aware that though GreptimeDB supports the wire protocol of MySQL, it does not mean GreptimeDB implements all | ||
MySQL's features. You may refer to the "[ANSI Compatibility](/reference/sql/compatibility.md)" to see the | ||
constraints regarding using SQL in GreptimeDB. | ||
|
||
## Migration steps | ||
|
||
### Create the databases and tables in GreptimeDB | ||
|
||
Before migrating the data from MySQL, you first need to create the corresponding databases and tables in GreptimeDB. | ||
GreptimeDB has its own SQL syntax for creating tables, so you cannot directly reuse the table creation SQLs that are produced | ||
by MySQL. | ||
|
||
When you write the table creation SQL for GreptimeDB, it's important to understand | ||
its "[data model](/user-guide/concepts/data-model.md)" first. Then, please take the following considerations in | ||
your create table SQL: | ||
|
||
1. Since the time index column cannot be changed after the table is created, you need to choose the time index column | ||
carefully. The time index is best set to the natural timestamp when the data is generated, as it provides the most | ||
intuitive way to query the data, and the best query performance. For example, in the IOT scenes, you can use the | ||
collection time of sensor data as the time index; or the occurrence time of an event in the observability scenes. | ||
2. In this migration process, it's not recommend to create another synthetic timestamp, such as a new column created | ||
with `DEFAULT current_timestamp()` as the time index column. It's not recommend to use the random timestamp as the | ||
time index either. | ||
3. It's vital to set the most fit timestamp precision for your time index column, too. Like the chosen of time index | ||
column, the precision of it cannot be changed as well. Find the most fit timestamp type for your | ||
data set [here](/reference/sql/data-types#data-types-compatible-with-mysql-and-postgresql). | ||
4. Choose the most fit tag columns based on your query patterns. The tag columns store the metadata that is | ||
commonly queried. The values in the tag columns are labels attached to the collected sources, generally used to | ||
describe a particular characteristic of these sources. The tag columns are indexed, making queries on them | ||
performant. | ||
|
||
Finally please refer to "[CREATE](/reference/sql/create.md)" SQL document for more details for choosing the | ||
right data types and "ttl" or "compaction" options, etc. | ||
|
||
### Write data to both GreptimeDB and MySQL simultaneously | ||
|
||
Writing data to both GreptimeDB and MySQL simultaneously is a practical strategy to avoid data loss during migration. By | ||
utilizing MySQL's client libraries (JDBC + a MySQL driver), you can set up two client instances - one for GreptimeDB | ||
and another for MySQL. For guidance on writing data to GreptimeDB using SQL, please refer to the [Ingest Data](/user-guide/ingest-data/for-iot/sql.md) section. | ||
|
||
If retaining all historical data isn't necessary, you can simultaneously write data to both GreptimeDB and MySQL for a | ||
specific period to accumulate the required recent data. Subsequently, cease writing to MySQL and continue exclusively | ||
with GreptimeDB. If a complete migration of all historical data is needed, please proceed with the following steps. | ||
|
||
### Export data from MySQL | ||
|
||
[mysqldump](https://dev.mysql.com/doc/refman/8.4/en/mysqldump.html) is a commonly used tool to export data from MySQL. | ||
Using it, we can export the data that can be later imported into GreptimeDB directly. For example, if we want to export | ||
two databases, `db1` and `db2` from MySQL, we can use the following command: | ||
|
||
```bash | ||
mysqldump -h127.0.0.1 -P3306 -umysql_user -p --compact -cnt --skip-extended-insert --databases db1 db2 > /path/to/output.sql | ||
``` | ||
|
||
Replace the `-h`, `-P` and `-u` flags with the appropriate values for your MySQL server. The `--databases` flag is used | ||
to specify the databases to be exported. The output will be written to the `/path/to/output.sql` file. | ||
|
||
The content in the `/path/to/output.sql` file should be like this: | ||
|
||
```plaintext | ||
~ ❯ cat /path/to/output.sql | ||
USE `db1`; | ||
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (1,'hello',1); | ||
INSERT INTO ... | ||
USE `db2`; | ||
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (2,'greptime',2); | ||
INSERT INTO ... | ||
``` | ||
|
||
### Import data into GreptimeDB | ||
|
||
The [MySQL Command-Line Client](https://dev.mysql.com/doc/refman/8.4/en/mysql.html) can be used to import data into | ||
GreptimeDB. Continuing the above example, say the data is exported to file `/path/to/output.sql`, then we can use the | ||
following command to import the data into GreptimeDB: | ||
|
||
```bash | ||
mysql -h127.0.0.1 -P4002 -ugreptime_user -p -e "source /path/to/output.sql" | ||
``` | ||
|
||
Replace the `-h`, `-P` and `-u` flags with the appropriate values for your GreptimeDB server. The `source` command is | ||
used to execute the SQL commands in the `/path/to/output.sql` file. Add `-vvv` to see the detailed execution results for | ||
debugging purpose. | ||
|
||
To summarize, data migration steps can be illustrate as follows: | ||
|
||
 | ||
|
||
After the data migration is completed, you can stop writing data to MySQL and continue using GreptimeDB exclusively! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters