Skip to content

Refuse to autogenerate a NOT NULL column filled only by a dataclass default_factory - #7815

Merged
masenf merged 4 commits into
mainfrom
fix/default-factory-not-null
Nov 11, 2026
Merged

masenf merged 4 commits into
mainfrom
fix/default-factory-not-null

Conversation

@masenf

@masenf masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Based on #7797, now merged. It does not depend on #7809, which handles FINDING-005/006/017.

Fixes FINDING-012 from the reflex 0.11.0a1 pre-release testing (artifacts on branch claude/vibrant-pasteur-3921g3, prerelease-testing/runs/2026-11-11-v0.11.0a1/).

Defect

Take a non-empty table and add score: Mapped[int] = mapped_column(default_factory=lambda: 7) on a MappedAsDataclass model. Autogenerate wrote add_column(... nullable=False) with no server_default, so reflex db migrate failed with NOT NULL constraint failed. When the revision rebuilt the table in sqlite batch mode, alembic also left an _alembic_tmp_* table behind that broke the next migrate.

The cause is the env.py template's add-column hook, which only reads Column.default. A dataclass default_factory is stored on the dataclass field only, so Column.default is None.

Fix

The hook now handles a NOT NULL added column that has no server_default and no Column.default like this:

  • It looks the column up in the mapped dataclasses of the registered or detected model bases. It uses table.c[op.column.key] (keyed by Column.key, which can differ from the name), then mapper.get_property_by_column(column).key, then __dataclass_fields__. A mapper that doesn't map the column, as with a single-table-inheritance parent, is skipped.

  • If the column's attribute has a default_factory, the hook raises a CommandError rather than guess a value:

    Cannot add the NOT NULL column 'item.score' with only a default_factory: the migration has no value for rows that a database already holds. Set server_default=..., a literal default= or insert_default=, or make the column nullable.

I put the refusal at autogenerate time because it fails before a broken revision is written, so no _alembic_tmp_* leftover can come from this path. It also fires in alembic check, so the startup schema check reports it.

Docs: docs/database/tables.md gets a sentence about default_factory and migrations.

Known behaviour

This was confirmed by the release owner.

  • A callable Column.default is still evaluated once, not refused. Examples are sqlmodel Field(default_factory=...) and mapped_column(insert_default=callable). The value is taken when the revision is generated and becomes the server default for existing rows, as in 0.10.1. test_callable_default_is_evaluated_for_existing_rows covers it. Only a dataclass default_factory that leaves Column.default empty is refused.
  • A non-callable Column.default still becomes the server default, and a nullable column is never refused.
  • The column is refused on an empty table too. The revision also runs on other databases, such as production, that may hold rows.

Test plan

  • New test test_default_factory_column_is_refused in tests/units/utils/test_migrations.py (real alembic and sqlite, MappedAsDataclass app). It failed before the fix: makemigrations exited 0 and wrote ..._factory.py. After the fix it checks:

    • makemigrations exits 1 with the message naming 'thing.score', and no revision is written.
    • Bare alembic check fails with the same message.
    • A column with a custom key, mapped_column('points', key='points_key', default_factory=...), is refused too, naming 'thing.points'. This case was added for the review thread and failed before the key fix.
    • A literal default=7 gets server_default=sa.text('7') and migrates the existing row.
    • A nullable default_factory column is accepted.
    • The test app has an rx.Model table beside Base, so autogenerate compares merged copies of the columns and the hook must find the column by key.
    • A column that a single-table-inheritance subclass adds to its parent's table is refused, naming 'thing.extra'. Only the subclass mapper maps it.
  • uv run pytest tests/units --cov (without tests/units/reflex_bench/drivers/test_browser.py, since there is no Chromium in this sandbox): 15645 passed, 66 skipped, coverage 81.44%. After the rebase onto 93beac9, the migration and model tests pass again: 43/43 locally, and test_migrations 30/30 on SQLite 3.45.

  • uv run ruff check . and uv run ruff format --check . are clean. uv run pyright reflex tests gives 0 errors.

  • End to end with the verifier's verify_dbm/v3_default_factory.sh, run on an editable install:

    Variant Result
    df makemigrations rc 1 with the message, no revision; migrate rc 0 and its schema check logs the refusal; row untouched; no _alembic_tmp_* table
    dfbatch same as df
    dflt server_default=sa.text('7'), rows [(1, 'a', 7)]
    ins server_default=sa.text('7'), rows [(1, 'a', 7)]
    rxdf server_default=sa.text('7'), rows [(1, 'a', 7)]

Notes

  • The fix lives in the env.py template. Apps upgrading from 0.10.x get it when their stock env.py is replaced. An env.py that 0.11.0a1's reflex db init wrote keeps the old hook until it is moved aside and re-initialized.
  • I did not add cleanup of an _alembic_tmp_* table left by an already-written broken revision.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

@masenf
masenf requested review from a team and Alek99 as code owners November 11, 2026 09:13
@masenf masenf added this to the v0.11.0 milestone Nov 11, 2026
@masenf masenf added the on deck PRs lined up to review / merge next label Nov 11, 2026
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Refuses to generate migrations for added NOT NULL columns whose only default is a dataclass default_factory.

  • Migration generation refuses required columns backed only by dataclass factories.

Reviews (7) · Last reviewed commit: "Reword the default_factory refusal messa..." · Reviewed by Greptile

Comment thread reflex/.templates/alembic/env.py Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 14 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/database/overview.md Resolved
Comment thread reflex/model.py Resolved
Comment thread reflex/.templates/alembic/env.py Resolved
@masenf

masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

app-harness-playwright (redis, 3.11, 1) is red because of an ERROR at teardown of tests/integration/tests_playwright/test_client_error.py::test_state_mismatch_prompts_reload[prod-chromium]. The test itself passed on all three reruns. The raise_console_error fixture caught the [Reflex Frontend Exception] Skipped state update for unknown substate(s) log that this test provokes on purpose, so the log landed in the wrong capture window. The test exercises a frontend/backend substate mismatch and touches none of alembic, env.py, DeclarativeBase or reflex db, which is all this PR (and #7797 under it) changes. The same job passed on #7797 and #7809. The workflow is still running other jobs, so the failed job can't be re-run yet. It will be re-run once when the run finishes, or by the next push to this branch.


Generated by Claude Code

@codspeed

codspeed Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 11.32%

2 warnings

⚠️ 22 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚡ 1 improved benchmark
✅ 150 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_state_collection_assignment[1000] 12 ms 10.8 ms +11.32%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/default-factory-not-null (8af7b98) with main (a6f2f8c)

Open in CodSpeed

@masenf

masenf commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

CodSpeed is red here (test_route_matching[1000] -5.8%, test_process_event_burst_independent_tokens[1] -4.0%), but this PR doesn't cause it. CodSpeed flags both as measured on different hardware, and both show the same 4–6% drop on every PR in this batch. This PR (and #7797 under it) changes only migration generation: the alembic env.py template, reflex/model.py base discovery and reflex/utils/migrations.py. None of those runs in route matching or event processing. CodSpeed is not a required check; leaving it for acknowledgement on CodSpeed.


Generated by Claude Code

@masenf
masenf force-pushed the fix/default-factory-not-null branch from 3faf080 to 342353a Compare November 11, 2026 09:29
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/default-factory-not-null branch 2 times, most recently from fcfb9ff to 3e9f0a8 Compare November 11, 2026 09:34

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

`mapped_column(default_factory=...)` on a MappedAsDataclass model is the
dataclass's default alone: the Column has no default, so the env.py
add-column hook gave the revision no server_default and adding the
column to a table with rows failed with "NOT NULL constraint failed"
(on sqlite batch mode leaving an `_alembic_tmp_*` table behind).

The hook now looks the column up in the mapped dataclasses and, for a
NOT NULL column whose only default is a default_factory, raises a
CommandError naming the table and column before any revision is
written (and in `alembic check`). Column defaults, literal or callable,
still become the server default as before; a nullable column needs none.

Fixes FINDING-012 of the 0.11.0a1 pre-release testing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
`Table.c` is keyed by `Column.key`, so a column with an explicit key
other than its name was missed and its default_factory not refused.
The property is now found with `mapper.get_property_by_column` and the
dataclass field with `__dataclass_fields__`, instead of scanning
`mapper.columns` and `dataclasses.fields`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/default-factory-not-null branch from 3e9f0a8 to 2e4a9d3 Compare November 11, 2026 17:40
masenf pushed a commit that referenced this pull request Nov 11, 2026
…onto main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf

masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

CodSpeed on 2e4a9d3 flags test_route_matching[10] (−5.8%) and test_process_event_burst_independent_tokens[1] (−4.2%). Neither comes from this PR. Its only source change is the alembic env.py template (reflex/.templates/alembic/env.py), which runs only during migration autogeneration, never in either benchmark. Both benchmarks drift on unrelated PRs, and CodSpeed reports two benchmarks in this run measured on different hardware.


Generated by Claude Code

…tory refusal test

An rx.Model table beside Base makes autogenerate compare copies of the
columns, so the hook must look the column up by key. A subclass that
shares its parent's table adds a column that only the subclass mapper
maps. The test reached neither path before.
State the cause plainly. The old text also said the table holds rows,
which is wrong for an empty table that is refused too.
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread reflex/.templates/alembic/env.py Resolved

@masenf masenf left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@masenf
masenf merged commit 3184cad into main Nov 11, 2026
168 checks passed
@masenf
masenf deleted the fix/default-factory-not-null branch November 11, 2026 18:32
masenf added a commit that referenced this pull request Nov 11, 2026
…efault_factory (#7815)

* Refuse a NOT NULL column filled only by a dataclass default_factory

`mapped_column(default_factory=...)` on a MappedAsDataclass model is the
dataclass's default alone: the Column has no default, so the env.py
add-column hook gave the revision no server_default and adding the
column to a table with rows failed with "NOT NULL constraint failed"
(on sqlite batch mode leaving an `_alembic_tmp_*` table behind).

The hook now looks the column up in the mapped dataclasses and, for a
NOT NULL column whose only default is a default_factory, raises a
CommandError naming the table and column before any revision is
written (and in `alembic check`). Column defaults, literal or callable,
still become the server default as before; a nullable column needs none.

Fixes FINDING-012 of the 0.11.0a1 pre-release testing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

* Look up the added column by key and its property directly

`Table.c` is keyed by `Column.key`, so a column with an explicit key
other than its name was missed and its default_factory not refused.
The property is now found with `mapper.get_property_by_column` and the
dataclass field with `__dataclass_fields__`, instead of scanning
`mapper.columns` and `dataclasses.fields`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

* Cover merged metadata and single-table inheritance in the default_factory refusal test

An rx.Model table beside Base makes autogenerate compare copies of the
columns, so the hook must look the column up by key. A subclass that
shares its parent's table adds a column that only the subclass mapper
maps. The test reached neither path before.

* Reword the default_factory refusal message

State the cause plainly. The old text also said the table holds rows,
which is wrong for an empty table that is refused too.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Farhan <www.mfarvirus@gmail.com>
(cherry picked from commit 3184cad)
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf masenf added the release-train Issues found during agentic /prerelease-testing fanout, not necessarily validated bugs. label Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

on deck PRs lined up to review / merge next release-train Issues found during agentic /prerelease-testing fanout, not necessarily validated bugs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants