Repository navigation
Fix in-place gem rebuild after a one-shot build script (#2106) - #2161
Mikola Lysenko (mikolalysenko) wants to merge 4 commits into
Conversation
Assisted-by: Claude Code:claude-fable-5-2 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011f9aeRso7kGGWRcMwXQpkz
A build script can use up a file the gem ships: argon2's extconf.rb renames Makefile.real to Makefile. The installed tree then lacks the file, and the in-place rebuild of agent mode fails in the script. The first test fails on main (the apply ends in native_extension_build_failed). The second guards the fix: a build that fails leaves the gem exactly as installed. Refs #2106 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011f9aeRso7kGGWRcMwXQpkz Assisted-by: Claude Code:claude-fable-5-2
A gem that loads its library from its own tree is rebuilt in agent mode where it is installed. That build ran over the tree as the install left it, and a build script may use up a file the gem ships: argon2's extconf.rb renames Makefile.real to Makefile. Run again, the script fails, so every agent-mode patch to an argon2 build input ended in native_extension_build_failed. The staged build of the source gem already says the gem builds into its own tree. It now also keeps the files the source gem ships that the installed copy no longer holds, read before the staged build uses them up as well. The in-place build gets them back after the tree snapshot is taken, so the undo of a failed build removes them. After a build that finishes, those it left untouched are removed again: the tree then differs from the installed one by what the build made, and rollback gives the installed tree back byte for byte. A rollback that has lost its backup rebuilds the same way. Fixes #2106 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011f9aeRso7kGGWRcMwXQpkz Assisted-by: Claude Code:claude-fable-5-2
Review fixes for the in-place build of a gem with a one-shot build script. A file given back kept the time it was written, which is after the patched sources were. A build that makes one from the other (a parser the gem ships generated) would take the shipped file for current. It now keeps the time the stage's copy was written, which is before the record's files are written anywhere. The size of a file is checked against the limit before it is read. Files that cannot be given back (a full disk) no longer fail the rebuild: it runs over the tree as it is, as it did before. The end-to-end test of a failed build broke the patched source, which fails the staged build first, so nothing was ever given back in it. Its build now fails only in the installed tree, after it used the shipped file up. Refs #2106 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011f9aeRso7kGGWRcMwXQpkz Assisted-by: Claude Code:claude-fable-5-2
|
bugbot run Generated by Claude Code |
PR SummaryMedium Risk Overview
Tests: e2e Reviewed by Cursor Bugbot for commit 48c82f8. Configure here. |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 48c82f8. Configure here.
|
[agent] PR shepherd: review brief for the approver What it does. Fixes #2106. An agent-mode Risk: low. It only writes paths that don't exist yet, inside the gem dir, never through a symlink or junction, and after the tree snapshot, so the existing undo removes them on a failed build. A gem whose installed tree is complete takes the same path as before. Look here
Verified. I reviewed the whole diff for path safety, partial failures, ordering, Windows (bytes are copied raw) and debug leftovers.
Changes I made: none. Open questions: none that block a merge. Two nits for a follow-up:
Generated by Claude Code |
LLM Description written by Claude Code:claude-fable-5-2
Fixes #2106
Summary
Agent-mode
applycould never rebuild a gem whose build script uses up a file the gem ships. argon2 is the reported case: itsextconf.rbrenamesMakefile.realtoMakefile, so the installed gem has noMakefile.real, and the rebuild in the installed tree failed in the script withnative_extension_build_failed.applynow puts the files the gem's first build used up back for that rebuild, from the source gem it has already unpacked and proven to be the installed gem's own source. A failed build leaves the gem exactly as installed, and a finished one leaves nothing in the tree the build did not make. Nothing changes for a gem whose installed tree is complete, or when no source gem is at hand.Root cause
A gem that loads its library from its own tree (an FFI gem) cannot take a build staged elsewhere, so it is rebuilt where it is installed (
rebuildincrates/socket-patch-core/src/patch/gem_ext.rs). That build assumed the installed tree still held everything the build scripts read. The tree is what the install-time build left, and a build may rename or delete what the gem shipped.The change
build_in_stagereads the files the pristine source gem ships and the installed copy does not hold (used_up_files), before the staged build uses them up too. At most 64 MiB, all or none, and only for a copy RubyGems built.StageFailure::InTreecarries them torebuildwith the list of products it already carried (SeenInStage).rebuildwrites them into the gem's tree after the tree snapshot is taken (give_back), so the existing undo of a failed or interrupted build removes them. It never writes over a file the tree holds, a file of the patch record, or through a symlink in the gem dir. Each file keeps the time the staged copy was written, which is older than the patched sources, so a build that derives one file from another does not take a shipped file for current.take_away), with the directories made for them.rollbacktherefore still gives the installed tree back byte for byte.rollbackthat has lost its backup rebuilds through the same path.docs/ecosystems.md("Ruby gems: native extensions") names argon2 again and describes this.Evidence
Real gem, Ruby 3.3.6, Bundler 4.0.18, the repro from the issue:
origin/mainsocket-patch applyonargon2 2.3.0native_extension_build_failed(extconf.rb:Makefile.realmissing)lib/libargon2_wrap.soArgon2::Passwordworksapply --checkrollback, thendiff -ragainst the fresh installTests:
a_build_script_that_uses_up_a_shipped_file_is_given_it_back(e2e_gem_native_ext_build) coversFixes #2106: a gem built with the real toolchain whose script renames a shipped file. It fails onorigin/mainwith the issue's error (first commit of the branch) and passes with the fix. It asserts that only the patched source and the library changed, thatapply --checkandvexaccept the build, and thatrollbackrestores the tree with and without the backup.a_failed_build_takes_the_shipped_file_it_was_given_away_again: a build that fails in the installed tree after using the file up leaves the gem exactly as installed.gem_ext.rscover reading the files, giving them back and taking them away, and the paths that are never written (existing file, record file, unsafe path, path under a symlink).Local runs:
cargo clippy --locked --workspace --all-features -- -D warningsclean;socket-patch-corepatch::unit tests pass (1364);e2e_gem_native_ext_build(41),e2e_redirect_gem_build(49),e2e_vendor_gem_build(18) andin_process_gem_apply(3) pass with--include-ignored.e2e_gemneeds the live patch API, which this sandbox cannot reach; CI runs it.CI on the head commit:
ci-okandclippyare green, and Bugbot found no issues. The nokogiri compatibility run was cancelled at creation (the push and the ready-for-review event raced) and was re-run. Its Ruby 2.3 / nokogiri 1.6.8.1 leg then failed once inbundle install, before socket-patch ran (Bundler 1.17 resolved aminitestrelease that needs Ruby 3.1), and passed on its one re-run.Review
/code-review highon the diff gave six findings. Fixed: the failed-build test never had a file given back; given-back files were newer than the patched sources; a file was read before its size was checked; a write failure while giving files back failed the rebuild with a misleading message (the rebuild now runs over the tree as it is, as before this change).Not fixed here:
main. That is the same behavior as before, not a regression, and widening it touches everyStageFailurevariant; it is better as its own change if a real gem needs it.https://claude.ai/code/session_011f9aeRso7kGGWRcMwXQpkz
Generated by Claude Code