com.ververica.cdc.connectors.mysql.source.assigners

Class MySqlSnapshotSplitAssigner

java.lang.Object

com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner

All Implemented Interfaces:

MySqlSplitAssigner

public class MySqlSnapshotSplitAssigner
extends Object
implements MySqlSplitAssigner

A MySqlSplitAssigner that splits tables into small chunk splits based on primary key range and chunk size.

See Also:

MySqlSourceOptions.SCAN_INCREMENTAL_SNAPSHOT_CHUNK_SIZE

Constructor Summary

Constructors

Constructor and Description
MySqlSnapshotSplitAssigner(MySqlSourceConfig sourceConfig, int currentParallelism, List<io.debezium.relational.TableId> remainingTables, boolean isTableIdCaseSensitive)
MySqlSnapshotSplitAssigner(MySqlSourceConfig sourceConfig, int currentParallelism, SnapshotPendingSplitsState checkpoint)

Method Summary

Modifier and Type	Method and Description
void	addSplits(Collection<MySqlSplit> splits) Adds a set of splits to this assigner.
void	close() Called to close the assigner, in case it holds on to any resources, like threads or network connections.
Map<String,MySqlSnapshotSplit>	getAssignedSplits()
AssignerStatus	getAssignerStatus() Gets the split assigner status, see AssignerStatus.
List<FinishedSnapshotSplitInfo>	getFinishedSplitInfos() Gets the finished splits information.
Optional<MySqlSplit>	getNext() Gets the next split.
Map<String,BinlogOffset>	getSplitFinishedOffsets()
boolean	noMoreSplits() Indicates there is no more splits available in this assigner.
void	notifyCheckpointComplete(long checkpointId) Notifies the listener that the checkpoint with the given checkpointId completed and was committed.
void	onFinishedSplits(Map<String,BinlogOffset> splitFinishedOffsets) Callback to handle the finished splits with finished binlog offset.
void	open() Called to open the assigner to acquire any resources, like threads or network connections.
SnapshotPendingSplitsState	snapshotState(long checkpointId) Creates a snapshot of the state of this split assigner, to be stored in a checkpoint.
void	suspend() Suspends the assigner under AssignerStatus.INITIAL_ASSIGNING_FINISHED or AssignerStatus.NEWLY_ADDED_ASSIGNING_FINISHED.
boolean	waitingForFinishedSplits() Whether the split assigner is still waiting for callback of finished splits, i.e.
void	wakeup() Wakes up the assigner under AssignerStatus.SUSPENDED.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MySqlSnapshotSplitAssigner

public MySqlSnapshotSplitAssigner(MySqlSourceConfig sourceConfig,
                                  int currentParallelism,
                                  List<io.debezium.relational.TableId> remainingTables,
                                  boolean isTableIdCaseSensitive)

MySqlSnapshotSplitAssigner

public MySqlSnapshotSplitAssigner(MySqlSourceConfig sourceConfig,
                                  int currentParallelism,
                                  SnapshotPendingSplitsState checkpoint)

Method Detail

open

public void open()

Description copied from interface: MySqlSplitAssigner

Called to open the assigner to acquire any resources, like threads or network connections.

Specified by:

open in interface MySqlSplitAssigner

getNext

public Optional<MySqlSplit> getNext()

Description copied from interface: MySqlSplitAssigner

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

Specified by:

getNext in interface MySqlSplitAssigner

waitingForFinishedSplits

public boolean waitingForFinishedSplits()

Description copied from interface: MySqlSplitAssigner

Whether the split assigner is still waiting for callback of finished splits, i.e. MySqlSplitAssigner.onFinishedSplits(Map).

Specified by:

waitingForFinishedSplits in interface MySqlSplitAssigner

getFinishedSplitInfos

public List<FinishedSnapshotSplitInfo> getFinishedSplitInfos()

Description copied from interface: MySqlSplitAssigner

Gets the finished splits information. This is useful meta data to generate a binlog split that considering finished snapshot splits.

Specified by:

getFinishedSplitInfos in interface MySqlSplitAssigner

onFinishedSplits

public void onFinishedSplits(Map<String,BinlogOffset> splitFinishedOffsets)

Description copied from interface: MySqlSplitAssigner

Callback to handle the finished splits with finished binlog offset. This is useful for determine when to generate binlog split and what binlog split to generate.

Specified by:

onFinishedSplits in interface MySqlSplitAssigner

addSplits

public void addSplits(Collection<MySqlSplit> splits)

Description copied from interface: MySqlSplitAssigner

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added.

Specified by:

addSplits in interface MySqlSplitAssigner

snapshotState

public SnapshotPendingSplitsState snapshotState(long checkpointId)

Description copied from interface: MySqlSplitAssigner

Creates a snapshot of the state of this split assigner, to be stored in a checkpoint.

The snapshot should contain the latest state of the assigner: It should assume that all operations that happened before the snapshot have successfully completed. For example all splits assigned to readers via MySqlSplitAssigner.getNext() don't need to be included in the snapshot anymore.

This method takes the ID of the checkpoint for which the state is snapshotted. Most implementations should be able to ignore this parameter, because for the contents of the snapshot, it doesn't matter for which checkpoint it gets created. This parameter can be interesting for source connectors with external systems where those systems are themselves aware of checkpoints; for example in cases where the enumerator notifies that system about a specific checkpoint being triggered.

Specified by:

snapshotState in interface MySqlSplitAssigner

Parameters:

checkpointId - The ID of the checkpoint for which the snapshot is created.

Returns:

an object containing the state of the split enumerator.

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)

Description copied from interface: MySqlSplitAssigner

Notifies the listener that the checkpoint with the given checkpointId completed and was committed.

Specified by:

notifyCheckpointComplete in interface MySqlSplitAssigner

See Also:

CheckpointListener.notifyCheckpointComplete(long)

getAssignerStatus

public AssignerStatus getAssignerStatus()

Description copied from interface: MySqlSplitAssigner

Gets the split assigner status, see AssignerStatus.

Specified by:

getAssignerStatus in interface MySqlSplitAssigner

suspend

public void suspend()

Description copied from interface: MySqlSplitAssigner

Suspends the assigner under AssignerStatus.INITIAL_ASSIGNING_FINISHED or AssignerStatus.NEWLY_ADDED_ASSIGNING_FINISHED.

Specified by:

suspend in interface MySqlSplitAssigner

wakeup

public void wakeup()

Description copied from interface: MySqlSplitAssigner

Wakes up the assigner under AssignerStatus.SUSPENDED.

Specified by:

wakeup in interface MySqlSplitAssigner

close

public void close()

Description copied from interface: MySqlSplitAssigner

Called to close the assigner, in case it holds on to any resources, like threads or network connections.

Specified by:

close in interface MySqlSplitAssigner

noMoreSplits

public boolean noMoreSplits()

Indicates there is no more splits available in this assigner.

getAssignedSplits

public Map<String,MySqlSnapshotSplit> getAssignedSplits()

getSplitFinishedOffsets

public Map<String,BinlogOffset> getSplitFinishedOffsets()