# Migrating MySQL to Kafka

UDTS supports migration from MySQL to Kafka. Supported MySQL versions include MySQL (including the Percona version) 5.6/5.7/8.x. Supported Kafka versions are 2.x: including 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, etc.

## 1. Functional Limitations

### 1.1 Source MySQL Limitations
1. For incremental/complete + incremental migration, the source database needs to have binlog enabled, with the format set to ROW and image set to FULL.

```

Query method:
show global variables like 'binlog_format';
show global variables like 'binlog_row_image';

Setting method:
set global binlog_format = "ROW";
set global binlog_row_image = "FULL";

```

### 1.2 Target Kafka Limitations

1. Set `auto.create.topics.enable` to true.
2. Set `delete.topic.enable` to true.

## 2. Migration Content

| Migration Content | Description                                                             |
| ----------------- | ----------------------------------------------------------------------- |
| Migration Structure | Database, Table structure, and data                                      |
| Migration Scope   | Only migrates the databases and tables available when creating the task. Tables added during task operation will not be automatically migrated. |
| DDL               | CREATE, ALTER, DROP statements                                          |
| DML               | Snapshot/insert/update/delete                                           |

## 3. Form Filling

### Data Source Form

| Parameter Name | Description                                                                                                                                                                                |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Address Type   | Supports three types: internal network address, external network address, and dedicated line address. Internal network address requires filling in VPC and subnet information; external network address supports both IP and domain name; dedicated line address supports both IP and domain name. If using a domain name, the user network must have an external network exit. |
| Port           | MySQL connection port                                                                                                                                                                      |
| Username       | MySQL connection username                                                                                                                                                                  |
| Password       | Password corresponding to the MySQL user                                                                                                                                                   |
| Database Name  | MySQL database name. For all databases transmission, fill in *; to specify a single database, fill in the database name; to specify multiple databases, enter multiple database names separated by a comma. (Incremental migration cannot be performed if the database name contains spaces.) |
| Table Name     | MySQL transmission table name. Only valid when "Database Name" specifies a single database. If not filled in, the default is to migrate all tables in the specified database; to specify a single table transmission, fill in the table name; to specify multiple tables, enter multiple table names separated by a comma. |
| Nolocks        | Disabled by default. For partner RDS services that cannot obtain SUPER privileges, enable this option. See FAQ for how to obtain SUPER privileges for UDB.                                |

### Transmission Target Form

| Parameter Name    | Description                                                                                                                |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------- |
| Internal Address  | Connection address of the Kafka cluster, example: 92.168.1.10:9092,192.168.1.11:9092,192.168.1.12:9093                     |
| Maximum Rate      | Limit value for Kafka transmission speed, used to adjust the transmission rate                                             |
| Topic Prefix      | UDTS creates corresponding topics on the target Kafka when migrating MySQL to Kafka. A topic is created for each table, and each topic will use this parameter as a prefix. |
| Default Partition Number | The default number of partitions for the corresponding topic created on the target Kafka when migrating MySQL to Kafka with UDTS.                   |

## 4. Kafka Transmission Data Format

The content migrated to Kafka by UDTS is in the Debezium JSON format. Downstream synchronization can use Flink CDC, specifying the data format as debezium-json.