Skip to content

docs(dm): clarify Aurora incremental replication prerequisites - #24659

Open
doc-claw-bot wants to merge 4 commits into
masterfrom
docs/aurora-dm-incremental-prereqs
Open

doc-claw-bot wants to merge 4 commits into
masterfrom
docs/aurora-dm-incremental-prereqs

Conversation

@doc-claw-bot

@doc-claw-bot doc-claw-bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

What is changed, added, or deleted? (Required)

This PR clarifies the Aurora-to-TiDB migration guide for incremental replication:

  • adds the Aurora binary logging prerequisites for DM, including binlog_format=ROW
  • adds a binlog retention reminder and an example with mysql.rds_set_configuration
  • replaces the unsupported "export the snapshot within 5 minutes" guidance with wording that explains the snapshot/binlog-position alignment requirement

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • master (the latest development version)
  • v9.0 (TiDB 9.0 versions)
  • v8.5 (TiDB 8.5 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

What is the related PR or file link(s)?

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

Summary by CodeRabbit

  • Documentation
    • Clarified Aurora prerequisites for incremental replication, including required binlog settings and retaining binlogs until DM catches up.
    • Updated snapshot-export guidance to cover parameter-group settings, writer reboots, retention through catch-up, and obtaining the snapshot-aligned binlog position. Exported data reflects the selected snapshot’s point in time.
    • Added guidance to export a schema matching the snapshot, configure DM with matching replication coordinates (including the snapshot-aligned GTID set when applicable), and create a new snapshot if required settings or binlogs are unavailable.
    • Warned about incorrect recovery coordinates with enhanced binlog on affected Aurora MySQL 3 versions.

@doc-claw-bot doc-claw-bot added type/bugfix This PR fixes a bug. area/dm ONCALL Relates to documentation oncall. translation/welcome Waits for a contributor to translate this PR and create a PR to the pingcap/docs-cn repository. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. labels Nov 6, 2026
@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Nov 6, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 6, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lance6716 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review
Walkthrough

The Aurora-to-TiDB migration guide adds Aurora binlog prerequisites, snapshot-aligned schema export and recovery-position steps, and DM task configuration guidance.

Changes: Aurora to TiDB migration

Layer / File(s) Summary
Binlog prerequisites and retention
migrate-aurora-to-tidb.md
The guide specifies Aurora binlog settings, including ROW format and FULL row images. For Aurora MySQL 3, it also requires transaction compression to be off. Binlogs must remain available until DM catches up.
Snapshot-aligned schema export
migrate-aurora-to-tidb.md
The guide requires the exported schema to match the selected snapshot. If the live schema has changed, it directs users to restore the snapshot to a temporary cluster and export from that cluster. It also describes retention requirements and obtaining the snapshot’s binlog position, including a warning about enhanced binlog before Aurora MySQL 3.14.0.
DM snapshot position configuration
migrate-aurora-to-tidb.md
The guide directs users to configure DM with the snapshot-aligned binlog name and position and connect to the original Aurora source. In GTID mode, it also requires the GTID set for that position. The sample binlog coordinates are updated.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: qiancai, lilin90

Merge Risk: Low · previous review b48fb

The guide now explains how to align the schema and the DM start position with a chosen Aurora snapshot. Two edge cases are still undocumented. After a replacement snapshot, readers need to re-export the schema. For GTID-based replication, readers need GTID mode to be on before the snapshot is taken. Readers who hit these cases could set up a migration that stops or replays changes incorrectly. A small text clarification before merge would close both gaps.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: clarifying Aurora incremental replication prerequisites for the DM migration guide. The changeset adds binlog prerequisites (ROW format, FULL row image, retention) and…
Description check Passed The description follows the template structure: it fills in the 'What is changed' section with a clear list of changes, marks master as the target version, adds reference links, and ticks the AI agent…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: clarifying Aurora incremental replication prerequisites for the DM migration guide. The changeset adds binlog prerequisites (ROW format, FULL row image, retention) and rewrites snapshot/binlog-position guidance, so the title matches the primary change. It is short, specific, and uses the conventional docs prefix.

Full details: Description check

Explanation

The description follows the template structure: it fills in the 'What is changed' section with a clear list of changes, marks master as the target version, adds reference links, and ticks the AI agent involvement box. The first-time contributor checklist and the 'Do your changes match' items are left unticked, which is acceptable because they are conditional. The description does not mention the Aurora MySQL 3 compression setting or the 3.14.0 upgrade warning from the summary, and it does not state the GTID handling change. These are not required by the template, but adding them would make the description more complete.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@qiancai qiancai removed needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. labels Nov 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 21e0e01 and 0e343c2.

Files selected for processing: 1

  • migrate-aurora-to-tidb.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread migrate-aurora-to-tidb.md Resolved
Comment thread migrate-aurora-to-tidb.md Resolved
Comment thread migrate-aurora-to-tidb.md Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 0e343c2 and 24881a2.

Files selected for processing: 1

  • migrate-aurora-to-tidb.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread migrate-aurora-to-tidb.md Outdated
+----------------------------+----------+--------------+------------------+-------------------+
1 row in set (0.012 sec)
```
2. Create or select the Aurora DB cluster snapshot to export. If you plan to use DM, obtain the binlog file name and position corresponding to this snapshot: restore the snapshot to a temporary Aurora cluster using the same parameter groups as the source, then find the `Binlog position from crash recovery is <binlog-file-name> <binlog-position>` message in the restored writer instance's **Recent Events**. For detailed instructions, see [Setting up binary log replication for Aurora MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Replication.MySQL.SettingUp.html). Record these values for the DM task, which must connect to the original source cluster. You can then delete the temporary cluster.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Data Integrity & Integration · Heavy lift

Align the schema with the selected snapshot.

If an existing snapshot predates an ALTER TABLE, Step 1 exports the later schema from the live source. Lightning then imports the older snapshot into that schema, and DM can replay the same ALTER TABLE and stop. Document how to obtain the schema at the snapshot boundary and use it for the import.


Minor · Data Integrity & Integration · Quick win

Document the enhanced binlog coordinate limitation for Aurora MySQL versions earlier than 3.14.0.

Step 2.1 obtains the DM binlog position from a temporary restored cluster. Aurora MySQL 3.14.0 fixes an enhanced binlog issue that reported incorrect binary log coordinates after a snapshot restore. Before the fix, this could create an invalid binlog replication setup when transactions were rolled back (Aurora MySQL 3.14.0 release notes).

This guide does not instruct users to enable enhanced binlog. The restored cluster uses the source's parameter groups, so a source with aurora_enhanced_binlog set to 1 can carry this issue into the procedure. Update step 2 to state this limitation for Aurora MySQL versions earlier than 3.14.0 and the condition under which it applies. Specify the required action for users who meet that condition. Do not present the restored-cluster position as a safe DM starting point for those versions.

Comment thread migrate-aurora-to-tidb.md Outdated
### Step 2: Create the migration task

Create the `task1.yaml` file as follows:
Create the `task1.yaml` file as follows. Set `meta.binlog-name` and `meta.binlog-pos` to the snapshot-aligned values recorded in [Step 2.1](#21-export-an-amazon-aurora-snapshot-to-amazon-s3), and configure DM to read from the original source Aurora cluster:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Data Integrity & Integration · Heavy lift

Provide a snapshot-aligned GTID for GTID mode.

If the source requires switching, Line 190 directs users to enable GTID mode. The DM task reference then requires meta.binlog-gtid, but Step 2.1 records only a file and position. Add a procedure for obtaining and configuring the snapshot-aligned GTID set for this mode. (raw.githubusercontent.com)

@ti-chi-bot ti-chi-bot Bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Nov 20, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 24881a2 and b48fbee.

Files selected for processing: 1

  • migrate-aurora-to-tidb.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Comment thread migrate-aurora-to-tidb.md Resolved
Comment thread migrate-aurora-to-tidb.md Resolved
@doc-claw-bot doc-claw-bot added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/welcome Waits for a contributor to translate this PR and create a PR to the pingcap/docs-cn repository. labels Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dm contribution This PR is from a community contributor. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. ONCALL Relates to documentation oncall. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. type/bugfix This PR fixes a bug.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants