Skip to content

3.1: delete the 2.x Lexicon translation — refuse 2.x pickles, shorten the old-spelling errors #1003

Open

Description

@derek73

Background

3.0 configures a Lexicon by reading (add(w, kind=...), remove(w, kind=...)). Two migration aids share one translation of 2.x's compiled sets into readings, in nameparser/_lexicon.py:

  1. the refusal of a 2.x spelling (add/remove with a set-field keyword, Lexicon(...) or dataclasses.replace with one) names the 3.0 calls that replace it, word by word;
  2. a Lexicon pickled by 2.x (and a Parser or locale pack holding one) loads as the vocabulary it was written with, and warns with a DeprecationWarning since the PR that made this deprecation real.

decisions/vocabulary-model's 2026-11-02 #754 entry, "the 2.x translation's lifetime", decided both go in 3.1. This issue is that deletion.

Delete (in nameparser/_lexicon.py)

Code Serves What replaces it
_target_readings the translation nothing
_MARKER_READ_WITH, _inert_dropped the translation (a marker left behind) nothing
_edits_toward the translation nothing
_translation, _SHOWN_WORDS, _SHOWN_CALLS, _rendered (if nothing else uses it) the per-word fix in a refusal nothing
Lexicon._changed a refusal's sets-it-would-leave nothing
_refusal's last branch (the set-field one) the old-spelling TypeError a fixed message: f"{where} takes no {field} since 3.0: a Lexicon is configured by reading, add(*words, kind=...) and remove(*words, kind=...); docs/migrate.rst's Lexicon table gives the kind for each 2.x field". The list and mask branches stay as they are: they never translated
_translated_2x, _PICKLE_2X_DEPRECATION, the written_by_2x branch of Lexicon.__setstate__ the 2.x pickle load if "titles" in state: raise ValueError("this Lexicon pickle was written by nameparser 2.x, which 3.1 no longer loads: load it under 3.0 and pickle the loaded object again")
_warn_at_caller's category parameter the deprecation warning only revert to UserWarning if no other caller passes one

Check before deleting each: grep -n for its name in nameparser/ and tools/; _fields_of, _pool, _built and _normset(warn=False) may have other users.

Unchanged: the set attribute read (lexicon.titles) already raises a fixed AttributeError with no translation.

Tests that go (tests/v2/test_lexicon.py)

  • _state_2x, _loaded, _WARNED_2X, _LEFT_BEHIND, _REFUSED_CALLS
  • test_a_lexicon_parser_and_locale_pickled_by_2_8_0_load → becomes a refusal test (see below)
  • test_a_2x_pickle_warns_once_per_lexicon_at_the_callers_line, test_the_2x_pickle_warnings_remedy_runs
  • test_a_2x_default_loads_as_the_default, test_a_2x_title_listed_beside_its_gate_loads_as_a_title
  • test_a_2x_state_no_readings_express_is_refused_naming_the_fields, test_a_2x_state_missing_a_field_or_not_normalized_is_refused, test_a_2x_state_with_a_malformed_field_is_refused, test_a_2x_multi_word_title_is_refused_as_a_pickle_3_0_cannot_express
  • test_a_2x_removal_leaving_an_inert_marker_loads_as_the_removal, test_a_marker_left_behind_goes_with_its_word
  • test_the_translation_keeps_a_held_suffix_kind
  • test_a_2x_spelling_is_refused_with_calls_that_run, test_a_2x_spelling_with_no_exact_reading_offers_the_nearest_that_runs, test_each_2x_spelling_is_translated_or_said_to_have_none, test_a_spelling_that_already_reads_so_says_no_call_is_needed

Keep, with matches loosened to the fixed message: test_every_2x_set_field_is_refused_at_all_four_sites, test_a_list_or_the_masks_through_add_is_refused_with_its_spelling, test_add_capitalization_exceptions_raises_pointing_at_replace, test_a_2x_spelling_with_a_malformed_value_still_says_since_3_0. Re-read the parametrized test_a_malformed_argument_is_refused_by_name and test_add_and_remove_refuse_with_the_fix rows: they test 3.0 calls and should stay.

Add: a 2.8.0 Lexicon, Parser and locale pack each refused with the re-pickle message (one parametrized test over the fixture's three objects, pytest.raises(ValueError, match="re-pickle|pickle the loaded object again")), and a 3.0 pickle still loading.

Fixtures

  • tests/v2/data/lexicon_v28.pickle: keep, as the refusal test's input (it is the only 2.x Lexicon pickle in the tree).
  • tests/v2/data/parsers_4ib.pickle: unaffected (a 3.0 layout).
  • parsedname_v2*.pickle, policy_v28.pickle: unaffected (no Lexicon in them; confirm with a load under the change).

Docs

  • docs/migrate.rst: the "A Lexicon pickled by 2.x ..." paragraphs and the marker-left-behind paragraph become one sentence: a 2.x pickle is refused; load it under 3.0 and re-pickle. The kind table stays — it is what the shortened error points at.
  • docs/release_log.rst, under the 3.1 Unreleased entry: Remove loading a Lexicon, Parser or locale pack pickled by 2.x, which 3.0 deprecated. pickle.loads of a 2.8.0 Parser raises ValueError saying to re-pickle under 3.0, where 3.0.x loaded it with a DeprecationWarning; the error for a 2.x Lexicon spelling names :doc:migrate's table instead of each word's calls. :issue:754
  • docs/design/decisions/vocabulary-model.md: a dated entry recording it was carried out; strike or amend item 5 ("Pickle") and item 6 ("Migration") of the 2026-11-02 Lexicon surface entry by SUPERSEDES, and the 2026-11-02 "a marker left behind" entry (its contract dies with the load).
  • AGENTS.md's Pickling bullet: the sentences on _lexicon._translated_2x and the 2.8 fixtures loading go.

Done when

The functions above are gone, uv run pytest, -n0 -m serial, mypy, ruff, the Sphinx doctest build and --python 3.15 pass, and tests/v2/test_snapshot.py is byte-identical (no default parse reads this code).

Metadata

Metadata

Assignees

Labels

breaking-changeBackwards-incompatible API change

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions