clawhdf5: a dropped FileEditor releases its lock at once #23

2026-09-29T02:38:29Z

osobh commented

2026-09-29 02:38:29 +00:00

Merge order: this is 1 of 4 (#23 → #24 → #25 → #26). Each later PR is stacked on the one before it, so until that one is merged its diff also shows the earlier PRs' commits.

FileEditor now releases its lock as soon as it is dropped.

The bug: FileEditor holds an flock. When another thread in the same program starts a child process, the child briefly shares the locked descriptor, until it calls exec. During that window, reopening the file right after dropping the editor was refused with Error::Locked. This is what made edit_interop::editor_locks_the_file fail once in a full parallel test run.

The fix: Drop unlocks the file before closing it, which releases the lock for every descriptor that shares it. The agent store's lock file had the same pattern, previously hidden by its 250 ms retry on open, and gets the same fix.

Reproducer: edit_tests::drop_releases_the_lock_while_other_threads_spawn_processes (tank, 2026-09-28):

Before the fix: 1483 of 2000 reopens were refused.
After the fix: none refused, in 30 runs.

Lock type unchanged: an F_OFD_SETLK lock would not help. A forked child inherits it the same way, and on Linux it doesn't conflict with libhdf5's flock, so h5py would no longer be refused. The existing test that h5py is refused still passes.

🤖 Generated with Claude Code

**Merge order: this is 1 of 4 (#23 → #24 → #25 → #26).** Each later PR is stacked on the one before it, so until that one is merged its diff also shows the earlier PRs' commits. `FileEditor` now releases its lock as soon as it is dropped. **The bug:** `FileEditor` holds an `flock`. When another thread in the same program starts a child process, the child briefly shares the locked descriptor, until it calls `exec`. During that window, reopening the file right after dropping the editor was refused with `Error::Locked`. This is what made `edit_interop::editor_locks_the_file` fail once in a full parallel test run. **The fix:** `Drop` unlocks the file before closing it, which releases the lock for every descriptor that shares it. The agent store's lock file had the same pattern, previously hidden by its 250 ms retry on open, and gets the same fix. **Reproducer:** `edit_tests::drop_releases_the_lock_while_other_threads_spawn_processes` (tank, 2026-09-28): - Before the fix: 1483 of 2000 reopens were refused. - After the fix: none refused, in 30 runs. **Lock type unchanged:** an `F_OFD_SETLK` lock would not help. A forked child inherits it the same way, and on Linux it doesn't conflict with libhdf5's `flock`, so h5py would no longer be refused. The existing test that h5py is refused still passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

osobh added 1 commit 2026-09-29 02:38:30 +00:00

clawhdf5: a dropped FileEditor releases its lock at once

CI / test-arm64 (pull_request) Successful in 1m38s

Details

CI / test (pull_request) Successful in 28m11s

Details

3eca5d8334

FileEditor's flock belongs to the open file description. When another
thread forks to spawn a process, the child shares the locked descriptor
until it execs, so a reopen right after the drop could be refused with
Error::Locked (a one-off failure of edit_interop::editor_locks_the_file in
a parallel test run). Drop now unlocks before closing, which releases the
lock for every descriptor sharing it.

Reproducer edit_tests::drop_releases_the_lock_while_other_threads_spawn_processes
(4 threads running `true`, 2000 open/drop rounds): 1483 of 2000 reopens
refused before, 0 in 30 runs after (tank). An OFD lock would not help: it
is inherited across fork the same way and does not conflict with
libhdf5's flock. The agent store's lock file unlocks on drop too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

osobh referenced this pull request

2026-09-29 02:38:31 +00:00

clawhdf5-netcdf4: unlimited dimensions report their length #24

osobh referenced this pull request

2026-09-29 02:38:33 +00:00

Write complex numbers, incl. HDF5 2.0 native complex (class 11) #26

osobh referenced this issue from a commit

2026-09-29 02:38:39 +00:00

docs: known-issues — cite PRs #23 and #24

osobh merged commit d279ee06a2 into main

2026-09-29 03:10:44 +00:00

osobh deleted branch fix/editor-lock-fork-race