edit: plan every edit from the file the editor holds, not its path
FileEditor re-opened its path to plan each edit but wrote through the file it held open, and the Python 'r+' handle re-opened the path after every edit to read. When the path came to name another file between edits (a rename or replacement, or a relative path after os.chdir), an edit was laid out from the other file's metadata and written into the held one, corrupting it, and later reads came from the other file (the review's repro: h5py then reports "invalid dataset size, likely file corruption"). The editor now plans from a mapping of its own file (a clone of the held descriptor, dropped before the edit writes) and canonicalises its path at open. New FileEditor::reader() opens the held file anew for reading, without sharing the editor's flock (a mapping of a cloned descriptor holds the lock until unmapped): through /proc/self/fd on Linux, which follows a renamed file; elsewhere by path, refused on Unix when the path no longer names the held file. The Python handle reads through it and keeps no path; a 'w' file is written at the absolute path it was opened with. Tests: edit_tests.rs edits_go_to_the_file_held_not_the_path; test_edit.py test_relative_path_and_chdir and test_path_replaced_between_edits. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -2,6 +2,26 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Correctness: edits planned from another file after a rename or `chdir` (2026-09-27)
|
||||
- **`FileEditor` planned each edit by re-opening its path but wrote
|
||||
through the file it held open** (fixed 2026-09-27; on main since PR #18,
|
||||
no release). When the path came to name another file between edits — a
|
||||
rename or replacement, or, for a relative path, a change of working
|
||||
directory — an edit was laid out from the other file's metadata and
|
||||
written into the held one, corrupting it (h5py: "invalid dataset size,
|
||||
likely file corruption"). The Python `'r+'` handle had the same flaw in
|
||||
its reads: it reopened the path after every edit, so reads came from the
|
||||
other file. The editor now plans every edit from the file it holds, and
|
||||
its path is canonicalised at open. New `FileEditor::reader()` opens the
|
||||
held file anew for reading (on Linux through `/proc/self/fd`, so it
|
||||
follows a renamed file; elsewhere by the path, refused when the path no
|
||||
longer names the held file), without sharing the editor's lock; the
|
||||
Python handle reads through it, and a `'w'` file is written at the
|
||||
absolute path it was opened with. Tests: `edit_tests.rs`'s
|
||||
`edits_go_to_the_file_held_not_the_path`; `test_edit.py`'s
|
||||
`test_relative_path_and_chdir` and `test_path_replaced_between_edits`
|
||||
(the review's repro).
|
||||
|
||||
### Correctness: zero extents in Fixed/Extensible Array chunk indexes (2026-09-27)
|
||||
- **A chunked dataset whose maximum (or, with none recorded, current)
|
||||
extent is 0 along a dimension made the reader divide by zero** (fixed
|
||||
|
||||
Reference in New Issue
Block a user