Repository navigation
Test NuGet hosted mode under rules_csharp on real Bazel - #2140
Conversation
socket-patch has no Bazel-specific code for NuGet, and no test ran its
NuGet rewrite in a Bazel workspace. This adds the NuGet module of the
real-Bazel tests, its fixture and its workflow, and makes one edit to
the shared harness: NETRC is removed from every Bazel child (below).
No file under crates/*/src changes.
The flow holds hosted mode for a package that rules_csharp 0.5.0 reads
through `nuget.from_lock(packages_lock_json, nuget_config)` under
Bzlmod, on Bazel 9.3.2, 8.8.2 and 7.7.2. One test,
`nuget::hosted_from_lock`, in a fresh output base:
- builds the workspace and reads the unpatched marker from
`bazel run //:consumer`; the upstream package came from the stand-in;
- runs `scan --mode hosted --yes`, which rewrites nuget.config (the
Socket source of the patch, ahead of nuget.org) and
packages.lock.json (the contentHash of the patched package), and
nothing else;
- runs the same two Bazel commands and reads the patched marker, which
holds an id made for the run: the package came from the stand-in
patch host, and Bazel rewrote MODULE.bazel.lock and no other file;
- runs `rollback --yes`: the two files are the ones of before at once,
and after the next build so is every file of the checkout, and the
build prints the unpatched marker again;
- on a second machine, where the patch host serves other bytes than
the grant pins, the build fails in both states of the checkout: over
the contentHash of packages.lock.json while Bazel's lock is stale,
and over the sha256 in MODULE.bazel.lock once that lock pins the
granted package. There Bazel also refuses the copy on nuget.org,
which the hub tries next.
Three outcomes the flow asserts are gaps of the product, not its
contract. They are asserted as the release has them, so that the
measurement does not drift; the module docs name them under one
heading, and each is marked KNOWN GAP where it is asserted, with what
a fix would change in the test:
- Under Bazel alone socket-patch does not find NuGet packages from
packages.lock.json. `scan` finds a NuGet package only where a restore
unpacked it: a folder that `obj/project.assets.json` records, the
`packages/` folder of a packages.config project, or NuGet's global
packages folder. In a checkout without a restore record, on a
machine whose global packages folder does not hold the package (a
runner that only ever ran Bazel), `scan --mode hosted --yes` exits
0, finds no package and writes no file: hosted mode does nothing.
The flow asserts that case. Its patched path exists only because the
test then stages the global folder by hand, as a restore leaves it;
no dotnet runs.
- `vex` on the machine that ran the scan exits 1 and writes no
document (`not_applied`) while Bazel builds the patched package: it
reads the unpatched copy in NuGet's folder. Where the folder holds
nothing, `vex` attests the package from the pin in the two files.
- rules_csharp asks the Socket source for every package of the lock.
For the package the source does not hold, the stand-in answers 404,
and Bazel prints a warning with the URL it asked, which holds the
access token of the grant. The package then comes from nuget.org as
before.
What the flow asserts beside that, because a user meets it:
- The scan warns `redirect_nuget_stale_global_package`: the folder
still holds the unpatched copy. That copy is what `dotnet restore`
would go on using; no Bazel build reads it.
- The scan leaves MODULE.bazel.lock stale, and so does the rollback: a
`--lockfile_mode=error` build fails ("MODULE.bazel.lock is no longer
up-to-date") until a build that may update the lock has run.
- A second `scan` reports the patch applied again and rewrites no file.
The names of the two fixture packages end in an id made for the run.
nuget.org has no scopes, so an id that is committed here could be
pushed there by anyone; an id that is new for every run cannot. The
committed project, BUILD file and lock hold `@RUN@` where a checkout
holds that id, and NuGet takes no `@` in a package id, so those files
name no package that could exist either: a hermetic test reads every
id the three files name and holds it to that. The downloader config of
the test sends every URL of api.nuget.org and of patch.socket.dev to
the loopback stand-in and blocks both hosts, so no build asks either.
The lock of a checkout pins bytes that the run made, so no cache holds
a package under test.
The consumer is a genrule over the assembly that the hub's label
(`@nuget//:<package>`, what a C# rule names in deps) hands out. The
fixture's assembly is a line of text, no C# rule is built, and no .NET
SDK is downloaded. With an empty repository cache a run fetched 0.7 MB
on Bazel 9.3.2, 0.4 MB on 7.7.2 and 13.0 MB on 8.8.2, from
bcr.bazel.build, github.com and release-assets.githubusercontent.com
(macOS arm64). The rules_csharp archive is 142 kB of that; Bazel 8.8.2
fetches protobuf 29.0, rules_python, rules_java and rules_cc for its
own modules, as it does in a workspace without rules_csharp.
The fixture has one era, for Bazel 7.1.0 and later: the hub of
rules_csharp 0.5.0 downloads with `block = False`, which Bazel has
from 7.1.0. Only the three releases above ran.
The harness edit. Bazel's `http_archive` and its downloader read the
credentials file that NETRC names, and rules_csharp reads it for every
package it downloads; the harness passed the variable of the caller's
shell on to every Bazel child. With NETRC naming a file that does not
exist, the NuGet flow failed at its first fetch, in Bazel's own
`http_archive` (FileNotFoundException). With a file that holds a login
for 127.0.0.1, the stand-in received that login with each package
download; a login for api.nuget.org in the same file did not reach it.
NETRC now joins the variables removed from every Bazel child, for
every ecosystem's flow, and the hermetic test that guards that list
holds the name.
- crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs: the packages,
the stand-in (the Socket API, nuget.org's flat container, the Socket
source of a patch), the staged packages folder and the flow; two
hermetic tests hold the fixture, the downloader config and the path
lists of the workflow.
- crates/socket-patch-cli/tests/e2e_bazel_build/harness.rs: NETRC, in
the list of what every Bazel child loses and in the list of the test
that guards it.
- crates/socket-patch-cli/tests/fixtures/bazel/rules_csharp/bzlmod: the
workspace, with a placeholder where a checkout holds the id of its
run and the contentHash of its packages.
- .github/workflows/bazel-nuget.yml: three Linux legs on a pull request
that touches these files or a NuGet file the flow runs through, on a
main push that also touches the shared engine code, nightly and on
dispatch; one macOS leg nightly and on dispatch. Not required for
merge.
Not covered: what the real patch host answers for a package it does
not hold (the stand-in answers 404), signed packages, a real
`dotnet restore`, a C# compile, vendored and agent mode, rules_dotnet,
and a hub without a nuget.config.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
PR SummaryLow Risk Overview Introduces Harness: strips Reviewed by Cursor Bugbot for commit 1b61ea7. Configure here. |
|
[agent] PR shepherd: review brief for the approver What it does. Adds a real-Bazel end-to-end flow for NuGet in hosted mode under rules_csharp 0.5.0 ( Risk: low. This PR only adds tests. The real-Bazel test is Look here
Verified
Changes I made: none. Open questions (the description's "Decisions for the owner"; none blocks merging this test-only PR):
Generated by Claude Code |
Wenxin Jiang (Wenxin-Jiang)
left a comment
There was a problem hiding this comment.
[agent] Reviewed at 1b61ea7. Tests, fixture and workflow only, plus the harness scrubbing NETRC. The flow ran and passed on 7.7.2, 8.8.2 and 9.3.2, asserting exact nuget.config and packages.lock.json contents, the patched marker from the build, and tamper refusal on both contentHash and the MODULE.bazel.lock sha256. Non-blocking: the test pins that vex withholds the patch (not_applied, exit 1) while the global packages folder still holds the unpatched copy, which is a user-facing limitation worth documenting.
No user-visible change.
What this is
Tests only: the NuGet module of the real-Bazel tests, its fixture and its workflow, and one edit to the shared test harness.
socket-patchhas no Bazel-specific code for NuGet, and until now no test ran its NuGet rewrite in a Bazel workspace. No file undercrates/*/srcchanges.The flow holds hosted mode for a package that rules_csharp 0.5.0 reads through
nuget.from_lock(packages_lock_json = "//:packages.lock.json", nuget_config = "//:nuget.config")under Bzlmod: the lockdotnet restorewrites beside a project, and anuget.configwhose one source is nuget.org. rules_csharp 0.5.0 is the ruleset for Bazel 9.3.2, 8.8.2 and 7.7.2 alike (one fixture era). The fixture's table gives that era to Bazel 7.1.0 and later: the ruleset's hub downloads withblock = False, a parameter Bazel has from 7.1.0 (read in Bazel's source at the tags 7.0.0, 7.0.2 and 7.1.0). Only the three releases above ran.Contents
crates/*/src)crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs(947), six lines inharness.rsand onemodline inmain.rscrates/socket-patch-cli/tests/fixtures/bazel/rules_csharp/bzlmod/{common,e9}, five text files, no binaryPlus
.github/workflows/bazel-nuget.yml(137 lines), a caller ofbazel-compatibility.ymlmodelled onbazel-npm.yml. The diff is nine files and 1156 added lines. The reusable workflow and the other ecosystems' files are untouched.nuget.rs: the two fixture packages, built per run (a zip of a manifest and one placeholder assembly, no signature); the stand-in for the Socket API, nuget.org's flat container and the Socket source of a patch (service index and flat container); NuGet's global packages folder staged by hand asdotnet restoreleaves it; one flow,nuget::hosted_from_lock; two hermetic tests that hold the fixture, the downloader config and the workflow's path lists to the test code.harness.rs, the one shared edit:NETRCjoins the variables removed from every Bazel child, and the list of the hermetic test that guards them. Bazel'shttp_archiveand its downloader read the credentials file that variable names, and rules_csharp reads it for every package it downloads. Measured on Bazel 9.3.2 before the edit: withNETRCnaming a file that does not exist, the flow failed at its first fetch (aFileNotFoundExceptionin Bazel's ownhttp_archive); with a file that holds a login for127.0.0.1and one forapi.nuget.org, the stand-in received the loopback login with both package downloads, and not the other one. After the edit the flow passes withNETRC=/nonexistentexported, and the stand-in receives noAuthorizationheader. The edit applies to every ecosystem's flow.MODULE.bazel,nuget.config,packages.lock.json,App.csprojand aBUILD.bazelwhose consumer is agenruleover@nuget//:<package>, the label a C# rule names indeps. The fixture's assembly is a line of text; the rule writes it into the script it builds. No C# rule is built and no .NET SDK is downloaded.SP.Victim.R<32 hex>). nuget.org has no scopes, so a name committed here could be pushed there by anyone; a name that is new for every run cannot. The three committed files that name a package hold@RUN@where a checkout holds that id. NuGet takes no@in a package id, so the committed project and lock name no package that could exist on nuget.org either. A hermetic test reads every id those files name (the references of the project, the keys of the lock, the label in the BUILD file) and holds each to ending in the placeholder. The lock of a checkout pins bytes the run made, so no cache holds a package under test.What it proves, and where
One flow, in a fresh output base:
bazel buildandbazel run //:consumerprint the unpatched marker; the upstream package came from the stand-in.scan --mode hosted --yesrewritesnuget.config(the Socket source of the patch, ahead of nuget.org) andpackages.lock.json(thecontentHashof the patched package), and nothing else.MODULE.bazel.lockand no other file. The next build changes nothing and a--lockfile_mode=errorbuild passes. A second scan and twovexruns after that change no file of the checkout.rollback --yesrestores the two files byte for byte at once; after the next build every file of the checkout is the one of before and the build prints the unpatched marker.contentHashofpackages.lock.jsonwhile Bazel's lock is stale (nuget.org is not asked for the package), and over the sha256 inMODULE.bazel.lockonce that lock pins the granted package, where Bazel also refuses the copy on nuget.org that the hub tries next.Run on macOS arm64 at this commit, as the workflow's run step runs it (the ignored tests of the module by exact name, the release required), with a fresh output root and an empty repository cache per row. The last four rows are the other ecosystems' flows, run once because
harness.rschanged:nuget::hosted_from_locknuget::hosted_from_locknuget::hosted_from_lockgo::hosted_go_depsnpm::(two tests)pypi::hosted_pip_parseruby::hosted_bundle_fetchThe machine was under load from other builds; the NuGet flow took between 12 and 27 s per release over the runs of this review.
Changes that must fail the tests. Before the review, seventeen were tried and each failed at the intended assertion. Fourteen on Bazel 9.3.2: the scan left out; upstream bytes served under the patched name; the second patch host serving the granted bytes, in the first state and in the second state alone; a frozen build while Bazel's lock is current; a build between the rollback and the frozen build;
vexfirst on the machine without the package; the patched package in NuGet's folder before the firstvex; the first scan on a machine that has the package; a second scan that rewrites files;vexbefore any pin; the second tamper build with a lock that does not pin the granted package; the stand-in registry without the upstream package; and the downloader config without the nuget.org rewrite. Three without Bazel: the era table without Bazel 7, the downloader config withoutblock nuget.org, and a placeholder that could be part of a package id. Of those seventeen, only the second scan that rewrites files and the placeholder were run again after the review.For the review's changes, on the code of this commit, each of these failed at the assertion it is aimed at. On Bazel 9.3.2: the step after the first patched build expecting the unpatched marker; that step given the checkout of before the build (Bazel's lock then changes under it); a rollback run before the check that ends the step; a second scan that rewrites files. Without Bazel: a pattern of the workflow under a directory that does not exist; a file in the fixture that is not text (the message names it); a file in a subdirectory of the fixture that holds the placeholder; an id without the placeholder in the project, in the lock and in the BUILD file, one at a time; a placeholder without
@in the code and the three files; a lock whose pinning changes more than the two hashes; the old order of the un-pinning with a run tag that occurs in a hash;NETRCtaken out of the removed variables; andNETRCtaken out of the guarding test's list. Two controls passed, as they must: a subdirectory of the fixture with a text file that holds no placeholder, and the new order of the un-pinning with that same run tag.Linux x86_64 (the CI runner): the flow passed on its first run there, on all three releases (run 44393008302; checks
nuget / bazel (9.3.2, e9),(8.8.2, e9),(7.7.2, e9)): 13 s on 9.3.2, 14 s on 8.8.2, 15 s on 7.7.2 of test time, with no cache saved yet. Because this pull request adds one name to the shared harness, the same run covered the Go, npm, PyPI and RubyGems legs as well; all 68 checks pass. The macOS leg has not run on a GitHub runner.What starts on main
.github/workflows/bazel-nuget.ymlcallsbazel-compatibility.ymlwith three Linux legs (Bazel 9.3.2, 8.8.2, 7.7.2) and one macOS leg (9.3.2).nuget.rs,harness.rs, the fixture, or one of the NuGet files of the product the flow runs through:formats/nuget/, the NuGet crawler,vendor/nuget_config.rs,vendor/nuget_feed.rs,vex/discover/nuget.rs, the upstream restore ofrollbackand the hosted scan's NuGet check.patch/redirect/,hosted/,vex/, the CLI commands), the shared test helpers, the Cargo manifests and lock, or the toolchain files.workflow_dispatch. Only those two runs have the macOS leg, and no other workflow uses that minute.Every path of both lists names at least one tracked file, and the pull request list is a subset of the push list; a hermetic test holds the subset, every plain path to a file and every pattern to the directory before its first
*.Because this pull request changes
harness.rs, which is in the pull request list of every caller, it also starts the Go, npm, PyPI and Ruby legs.Outside hosts of a leg, beyond what the reusable workflow needs for itself:
bcr.bazel.build,github.comandrelease-assets.githubusercontent.com. With an empty repository cache a leg received 0.7 MB on Bazel 9.3.2, 0.4 MB on 7.7.2 and 13.0 MB on 8.8.2 (macOS arm64, measured before the review, which changed nothing a leg downloads). The rules_csharp archive is 142 kB of that; Bazel 8.8.2 fetches protobuf 29.0 (11.1 MB), rules_python, rules_java and rules_cc for its own modules, as it does in a workspace without rules_csharp. The repository cache of a leg holds 1.1 MB, 13 MB and 0.5 MB. No leg asks nuget.org or the patch host: the test's downloader config sends every URL ofapi.nuget.organdpatch.socket.devto the loopback stand-in and blocks both hosts.Not required for merge: no
merge_grouptrigger, and no job is aneedsofci-ok.Known behaviour the flow records
The first three are gaps of the product, not its contract. The flow asserts them as the release has them, so that the measurement does not drift; the module docs name them under one heading, and each assertion is marked
KNOWN GAPin the test with what a fix would change there.socket-patchdoes not find NuGet packages frompackages.lock.jsontoday. It needs a restore record (a folder that anobj/project.assets.jsonin the checkout names), apackages/folder of a packages.config project, or NuGet's global packages folder. In a checkout without any of them, on a machine where nobody randotnet restore(a runner that only ever runs Bazel),scan --mode hosted --yesexits 0, reports success, finds no package and writes no file: hosted mode does nothing. The flow asserts that. Its patched path exists only because the test then stages the two packages in NuGet's global folder by hand. The other two places are not in the flow; in one-off runs on Bazel 9.3.2 before the review, a checkout whoseobj/project.assets.jsonnames a folder with the two packages was scanned and pinned on a machine without a global folder, and a checkout with the packages underpackages/had both found. So today the scan has to run where the project was restored, and the two rewritten files have to be committed.vexon the machine that ran the scan exits 1 and writes no document (not_applied) while Bazel builds the patched package. It reads the unpatched copy in NuGet's folder, which is not what Bazel built. On a machine where the folder holds nothing,vexexits 0 and attests the package from the pin in the two files.redirect_nuget_stale_global_package: the folder still holds the unpatched copy. That is whatdotnet restorewould go on using; no Bazel build reads it.MODULE.bazel.lockstale, and so does the rollback. A--lockfile_mode=errorbuild fails ("MODULE.bazel.lock is no longer up-to-date") until a build that may update the lock, orbazel mod deps --lockfile_mode=update, has run and the lock is committed. The failure is asserted on the three releases; thatbazel mod depsis enough was tried once on each release before the review and is not in the flow.scan, as a pipeline runs it, reports the patch applied again and rewrites no file.rollbackbrings back the two files at once; Bazel's own lock comes back with the next build.rollbackalso warnsreinstall_required, which under Bazel needs no step: the next build is on the upstream package again. The warning is not asserted.Not claimed
contentHashis then not the sha512 of the file; the fixture's packages are unsigned.dotnet restore. The folder is staged by hand, and the project'sobjdirectory, which a restore in the checkout also leaves, is not written. A restore of the fixture directory as committed, with its placeholder, was not run either.csharp_libraryorcsharp_binaryis built.nuget.config,feedsorlocal_package_dirson the tag, and central package management.--lockfile_mode=error, so Bazel has to take the lock as current; with the granted bytes served in that state the build passes.defaultentry. That such an entry would go to the stand-in like the loopback one is read from Bazel's source, not run. The other ecosystems' flows ran withoutNETRCset.Decisions for the owner
packages.lock.jsonin a later pull request, make the scan say what a Bazel-only user has to do, or leave it?vexwithholds a package that the Bazel build has patched, until the upstream copy leaves NuGet's folder. Is that the verdict wanted for a Bazel workspace, or should it be changed in a later pull request?csharp_binary, with the .NET SDK the ruleset downloads) is not here. Add one on one release, or leave the leg without an SDK?MODULE.bazel.lock(about 55 lines) and the test of the workflow's path lists (about 30 lines). Keep them, or cut?Code review
A review of the commit found ten things. Each was checked by reading, and by a run where one was possible.
contentHash. It holds: with the tagRAin place ofR1, the hash ofSP.Victim.RAcontainsRAand the old order failed, while today's two hashes happen not to containR1. The hashes are now put back first, and a comment says why.*. For a pattern, the directory before its first*now has to exist.harness.rs, the one shared edit.NETRCis removed from every Bazel child and is in the list of the hermetic test that guards the removed variables. One part of the finding did not hold as written: a login forapi.nuget.orgin that file does not reach the stand-in, because Bazel looks up credentials by the URL it fetches after the rewrite. What does reach it is the login for the loopback host (measured) or adefaultentry (read in Bazel's source). The failure with a file that does not exist was reproduced, and is gone.common/as text and panicked on a stray file or a subdirectory. It now walks the regular files at any depth, and a file that is not UTF-8 fails with a message that names it.@only restated a constant. The test now reads every id the committed files name and holds each to ending in the placeholder and holding an@.vexwithholding on the scanning machine, the token in Bazel's warning) are asserted as before. Each is now markedKNOWN GAPin the test with what a fix would change, the module docs name them under one heading, and they come first in the section above.patch/redirect/mod.rs, where the NuGet rewrite lives. That is the stated policy of every caller: shared engine code runs on the push to main and nightly. The sentence in the workflow's header that says so is still true.frozen_build_failsinnuget.rsis a second copy of the frozen-lock step thatpypi.rsand the harness have. Folding them is a harness change of its own.sha256_hex,textand the workflow check are copies of helpers in the sibling modules (ruby.rs,npm.rs,pypi.rs). Same reason.vexruns, as the module docs always had it. The step is four Bazel commands, down from six. The two removed commands took 0.22 s on Bazel 9.3.2, 0.30 s on 8.8.2 and 0.26 s on 7.7.2 in a timed run: about a quarter of a second per leg, less than the run-to-run spread of a leg on this machine.Generated by Claude Code