com.ververica.cdc.connectors.mysql.source.utils

Class RecordUtils

java.lang.Object

com.ververica.cdc.connectors.mysql.source.utils.RecordUtils

public class RecordUtils
extends Object

Utility class to deal record.

Field Summary

Fields

Modifier and Type	Field and Description
static String	SCHEMA_CHANGE_EVENT_KEY_NAME
static String	SCHEMA_HEARTBEAT_EVENT_KEY_NAME

Method Summary

Modifier and Type	Method and Description
static BinlogOffset	getBinlogPosition(Map<String,?> offset)
static BinlogOffset	getBinlogPosition(org.apache.kafka.connect.source.SourceRecord dataRecord)
static Long	getFetchTimestamp(org.apache.kafka.connect.source.SourceRecord record) Return the timestamp when the change event is fetched in DebeziumReader.
static io.debezium.relational.history.HistoryRecord	getHistoryRecord(org.apache.kafka.connect.source.SourceRecord schemaRecord)
static Long	getMessageTimestamp(org.apache.kafka.connect.source.SourceRecord record) Return the timestamp when the change event is produced in MySQL.
static FinishedSnapshotSplitInfo	getSnapshotSplitInfo(MySqlSnapshotSplit split, org.apache.kafka.connect.source.SourceRecord highWatermark) Return the finished snapshot split information.
static Object[]	getSplitKey(org.apache.flink.table.types.logical.RowType splitBoundaryType, org.apache.kafka.connect.source.SourceRecord dataRecord, io.debezium.util.SchemaNameAdjuster nameAdjuster)
static BinlogOffset	getStartingOffsetOfBinlogSplit(List<FinishedSnapshotSplitInfo> finishedSnapshotSplits) Returns the start offset of the binlog split.
static io.debezium.relational.TableId	getTableId(org.apache.kafka.connect.source.SourceRecord dataRecord)
static BinlogOffset	getWatermark(org.apache.kafka.connect.source.SourceRecord watermarkEvent)
static boolean	isDataChangeRecord(org.apache.kafka.connect.source.SourceRecord record)
static boolean	isEndWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)
static boolean	isHeartbeatEvent(org.apache.kafka.connect.source.SourceRecord record)
static boolean	isHighWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)
static boolean	isLowWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)
static boolean	isSchemaChangeEvent(org.apache.kafka.connect.source.SourceRecord sourceRecord)
static boolean	isWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)
static List<org.apache.kafka.connect.source.SourceRecord>	normalizedSplitRecords(MySqlSnapshotSplit snapshotSplit, List<org.apache.kafka.connect.source.SourceRecord> sourceRecords, io.debezium.util.SchemaNameAdjuster nameAdjuster) Normalize the records of snapshot split which represents the split records state on high watermark.
static Object[]	rowToArray(ResultSet rs, int size) Converts a ResultSet row to an array of Objects.
static boolean	splitKeyRangeContains(Object[] key, Object[] splitKeyStart, Object[] splitKeyEnd) Returns the specific key contains in the split key range or not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SCHEMA_CHANGE_EVENT_KEY_NAME

public static final String SCHEMA_CHANGE_EVENT_KEY_NAME

See Also:

Constant Field Values

SCHEMA_HEARTBEAT_EVENT_KEY_NAME

public static final String SCHEMA_HEARTBEAT_EVENT_KEY_NAME

See Also:

Constant Field Values

Method Detail

rowToArray

public static Object[] rowToArray(ResultSet rs,
                                  int size)
                           throws SQLException

Converts a ResultSet row to an array of Objects.

Throws:

SQLException

normalizedSplitRecords

public static List<org.apache.kafka.connect.source.SourceRecord> normalizedSplitRecords(MySqlSnapshotSplit snapshotSplit,
                                                                                        List<org.apache.kafka.connect.source.SourceRecord> sourceRecords,
                                                                                        io.debezium.util.SchemaNameAdjuster nameAdjuster)

Normalize the records of snapshot split which represents the split records state on high watermark. data input: [low watermark event] [snapshot events ] [high watermark event] [binlog events] [binlog-end event] data output: [low watermark event] [normalized events] [high watermark event]

isWatermarkEvent

public static boolean isWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)

isLowWatermarkEvent

public static boolean isLowWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)

isHighWatermarkEvent

public static boolean isHighWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)

isEndWatermarkEvent

public static boolean isEndWatermarkEvent(org.apache.kafka.connect.source.SourceRecord record)

getWatermark

public static BinlogOffset getWatermark(org.apache.kafka.connect.source.SourceRecord watermarkEvent)

getMessageTimestamp

public static Long getMessageTimestamp(org.apache.kafka.connect.source.SourceRecord record)

Return the timestamp when the change event is produced in MySQL.

The field `source.ts_ms` in SourceRecord data struct is the time when the change event is operated in MySQL.

getFetchTimestamp

public static Long getFetchTimestamp(org.apache.kafka.connect.source.SourceRecord record)

Return the timestamp when the change event is fetched in DebeziumReader.

The field `ts_ms` in SourceRecord data struct is the time when the record fetched by debezium reader, use it as the process time in Source.

isSchemaChangeEvent

public static boolean isSchemaChangeEvent(org.apache.kafka.connect.source.SourceRecord sourceRecord)

isHeartbeatEvent

public static boolean isHeartbeatEvent(org.apache.kafka.connect.source.SourceRecord record)

getSnapshotSplitInfo

public static FinishedSnapshotSplitInfo getSnapshotSplitInfo(MySqlSnapshotSplit split,
                                                             org.apache.kafka.connect.source.SourceRecord highWatermark)

Return the finished snapshot split information.

Returns:

[splitId, splitStart, splitEnd, highWatermark], the information will be used to filter binlog events when read binlog of table.

getStartingOffsetOfBinlogSplit

public static BinlogOffset getStartingOffsetOfBinlogSplit(List<FinishedSnapshotSplitInfo> finishedSnapshotSplits)

Returns the start offset of the binlog split.

isDataChangeRecord

public static boolean isDataChangeRecord(org.apache.kafka.connect.source.SourceRecord record)

getTableId

public static io.debezium.relational.TableId getTableId(org.apache.kafka.connect.source.SourceRecord dataRecord)

getSplitKey

public static Object[] getSplitKey(org.apache.flink.table.types.logical.RowType splitBoundaryType,
                                   org.apache.kafka.connect.source.SourceRecord dataRecord,
                                   io.debezium.util.SchemaNameAdjuster nameAdjuster)

getBinlogPosition

public static BinlogOffset getBinlogPosition(org.apache.kafka.connect.source.SourceRecord dataRecord)

getBinlogPosition

public static BinlogOffset getBinlogPosition(Map<String,?> offset)

splitKeyRangeContains

public static boolean splitKeyRangeContains(Object[] key,
                                            Object[] splitKeyStart,
                                            Object[] splitKeyEnd)

Returns the specific key contains in the split key range or not.

getHistoryRecord

public static io.debezium.relational.history.HistoryRecord getHistoryRecord(org.apache.kafka.connect.source.SourceRecord schemaRecord)
                                                                     throws IOException

Throws:

IOException