[feature](load) Support show load for insert select progress#65203
Open
0AyanamiRei wants to merge 5 commits into
Open
[feature](load) Support show load for insert select progress#652030AyanamiRei wants to merge 5 commits into
0AyanamiRei wants to merge 5 commits into
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: INSERT INTO SELECT is executed synchronously, but the runtime load job was not visible enough through SHOW LOAD from another SQL session. This change registers a running InsertLoadJob after transaction creation and sink finalization, initializes progress tracking, marks the job LOADING during execution, and keeps the same job/statistics through FINISHED or CANCELLED states. It also adds regression coverage for observing S3/TVF-source insert-select progress with SHOW LOAD.
### Release note
Support observing INSERT INTO SELECT load progress through SHOW LOAD.
### Check List (For Author)
- Test: Regression test / Unit Test / Manual test
- Regression test: ./run-regression-test.sh --run -d load_p0/insert -s test_show_load_insert_runtime_observation -c 'jdbc:mysql://127.0.0.1:9035/?useLocalSessionState=true&allowLoadLocalInfile=true&zeroDateTimeBehavior=round' -ha 127.0.0.1:8035
- Unit Test: ./run-fe-ut.sh --run org.apache.doris.load.loadv2.InsertLoadJobTest
- Manual test: ./build.sh --fe -j60; git diff --check
- Behavior changed: Yes. SHOW LOAD can now show running INSERT INTO SELECT load jobs by label.
- Does this need documentation: No
0AyanamiRei
requested review from
924060929,
englefly,
gavinchou,
liaoxin01,
morrySnow and
starocean999
as code owners
July 3, 2026 09:08
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
### What problem does this PR solve? Issue Number: None Related PR: apache#65203 Problem Summary: Fix follow-up FE build failures in the INSERT INTO SELECT SHOW LOAD progress implementation. The previous local iteration left an unused import in AbstractInsertExecutor, missed imports required by InsertIntoTableCommand, and kept InsertLoadJobTest on the old initRunning signature. This change aligns the imports and test code with the current runtime registration implementation. ### Release note None ### Check List (For Author) - Test: Unit Test / Manual test - Unit Test: ./run-fe-ut.sh --run org.apache.doris.load.loadv2.InsertLoadJobTest - Manual test: ./build.sh --fe -j60; git diff --check for changed FE files - Behavior changed: No. - Does this need documentation: No
Contributor
Author
|
run buildall |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29583 ms |
Contributor
TPC-DS: Total hot run time: 172611 ms |
Contributor
ClickBench: Total hot run time: 25.15 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 30041 ms |
Contributor
TPC-DS: Total hot run time: 173870 ms |
Contributor
ClickBench: Total hot run time: 25.35 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
### What problem does this PR solve? Issue Number: None Related PR: apache#65203 Problem Summary: INSERT INTO SELECT is executed synchronously, but the runtime SHOW LOAD observation path temporarily initialized the InsertLoadJob as PENDING before marking it LOADING at execution start. PENDING is part of the asynchronous load scheduler lifecycle and can be misleading for synchronous INSERT jobs. This change initializes the runtime InsertLoadJob directly as LOADING after transaction and sink initialization, removes the separate markLoading transition, and updates the focused unit test to assert the direct LOADING semantics. ### Release note None ### Check List (For Author) - Test: Unit Test / Manual test - Unit Test: ./run-fe-ut.sh --run org.apache.doris.load.loadv2.InsertLoadJobTest - Manual test: git diff --check - Behavior changed: No. Internal state management is simplified while preserving SHOW LOAD runtime observation. - Does this need documentation: No
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
INSERT INTO ... SELECT ...runs synchronously, but users need to observe its load progress from another SQL session throughSHOW LOADby label. Before this change, the running insert load job/progress information was not registered early enough forSHOW LOADto provide useful runtime visibility.This PR registers a running
InsertLoadJobafter transaction creation and sink finalization, initializes progress tracking after the job is registered, marks the job asLOADINGduring execution, and reuses the same job/statistics when the insert reachesFINISHEDorCANCELLED. It also adds regression coverage for observing S3/TVF-sourceINSERT INTO SELECTruntime progress, including non-zero rows/bytes/file/backend details.Release note
Support observing INSERT INTO SELECT load progress through SHOW LOAD.
Check List (For Author)
Test
Behavior changed:
SHOW LOADcan observe runningINSERT INTO SELECTload jobs by label, including progress and job details.Does this need documentation?
Check List (For Reviewer who merge this PR)