# SQL Server Migration to Kafka
UDTS supports migration from SQL Server to Kafka. The supported SQL Server version is SQL Server 2008 R2 and above. Kafka supports version 2.x: including 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, etc.

## 1. Feature Limitations

### 1.1 Source SQL Server Limitations
1. The source database and the tables to be migrated need to have the CDC feature enabled.

```sql
-- Specify the database name
use dbname
-- Enable CDC feature
exec sys.sp_cdc_enable_db
-- Check if the CDC is enabled for the dbname database, a return value of 1 means it is enabled
select is_cdc_enabled from sys.databases where name = "dbname";
-- Enable CDC for the dbo.tablename table
exec sys.sp_cdc_enable_table @source_schema = 'dbo',@source_name = 'tablename',@role_name = null;
-- Check if the CDC is enabled for the dbo.tablename table, a return value of 1 means it is enabled
select is_tracked_by_cdc from sys.tables where name = 'tablename'
-- Modify the CDC data retention time, at least set to 1440 minutes (1 day) or more, recommended 14400 minutes (7 days).
EXECUTE sys.sp_cdc_change_job @job_type = N'cleanup', @retention = 14400;
```

### 1.2 Target Kafka Limitations

1. Set `auto.create.topics.enable` to true.
2. Set `delete.topic.enable` to true.


## 2. Migration Content

| Migration Content | Description                                                        |
| ----------------- | ------------------------------------------------------------------ |
| Migration Structure| Database, Table structure, and data                               |
| Migration Scope   | Only migrates tables that can be found when creating the task, newly added tables during the task will not be automatically migrated temporarily |
| DDL               | CREATE, ALTER, DROP statements                                     |
| DML               | Snapshot/insert/update/delete                                      |


## 3. Form Filling

### Data Source Form

| Parameter Name | Description                                                                                                         |
| -------------- | ------------------------------------------------------------------------------------------------------------------- |
| Address        | Supports two methods: internal network address and dedicated line address. Internal network address requires VPC and subnet information; dedicated line address supports both IP and domain name, if using a domain name, a user network with an external network exit is required |
| Port           | SQL Server connection port                                                                                          |
| Username       | SQL Server connection username                                                                                      |
| Password       | Password corresponding to the SQL Server database user                                                              |
| Database Name  | SQL Server database name, only supports single database migration                                                   |
| Table Name     | SQL Server table name to be transferred, use English commas to separate table names. Example: `dbo.tablename1,dbo.tablename2`, when the schema is dbo, it can be omitted, and you can fill only tablename, example: `tablename1,tablename2` |


### Target Transfer Form

| Parameter Name     | Description                                                                                       |
| ------------------ | ------------------------------------------------------------------------------------------------- |
| Internal Address   | Kafka cluster connection address, example: 92.168.1.10:9092,192.168.1.11:9092,192.168.1.12:9093   |
| Maximum Rate       | Kafka transfer rate limit, adjust the transfer rate                                               |
| Topic Prefix       | UDTS migrating MySQL to Kafka will create the corresponding Topic on the target Kafka; the rule is one Topic per table, each Topic will have this parameter as a prefix |
| Default Partition Number | The default number of partitions for the corresponding Topic when migrating MySQL to Kafka using UDTS |


## 4. Kafka Transfer Data Format

The content migrated from UDTS to Kafka is in Debezium JSON format. For downstream synchronization, Flink CDC can be used, specifying the data format as debezium-json.