clawhdf5/docs/superpowers/plans/2026-06-29-format-write-extensions.md

# Format Write Extensions Implementation Plan

> **Status (2026-08-03):** Implemented. Tasks 1–3 (external links, VDS mapping serialization, VDS `FileWriter` API) shipped in commit `d6c4d4f` (2026-06-30). Tasks 4–5 (superblock v4 read/write) were not part of that commit and were completed separately as part of this cleanup pass (2026-08-03) — see `Superblock::parse_v4`/`serialize` and `FileWriter::with_page_size` in `crates/clawhdf5-format`. This doc was authored 2026-06-29 as the pre-work plan and committed to the repo retroactively; checkboxes below have been marked complete to match current state. Treat this as a historical record, not an open task list.

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add three write-side features to clawhdf5-format: (1) external link creation via `GroupBuilder`, (2) external VDS (Virtual Dataset Source) layout writes, and (3) superblock v4 read/write for page-buffering-aware files.

**Architecture:** External links reuse the existing `LinkMessage::serialize()` which already handles `LinkTarget::External` — only the `GroupBuilder` API needs wiring up. External VDS adds `write_vds_layout()` in `file_writer.rs` and `serialize_vds_mappings()` in a new `data_layout_write.rs`. Superblock v4 extends `Superblock::parse` with a new `parse_v4` branch (identical structure to v3 with an extra `page_size` field) and updates `Superblock::serialize` to optionally write v4.

**Tech Stack:** Pure Rust, no new dependencies. All changes in `crates/clawhdf5-format/`.

## Global Constraints

- All code in `crates/clawhdf5-format/`.
- No new Cargo dependencies.
- External links: written as `LinkTarget::External`, readable by h5py (verified in tests).
- VDS: uses data layout version 4, class 3. Global heap at end of file.
- Superblock v4: only adds `page_size: u32` field after the v2/v3 body; checksum placement unchanged.
- Run `cargo test -p clawhdf5-format` after every task.

---

### Task 1: External link write API in GroupBuilder

**Background:** `LinkMessage::serialize()` in `link_message.rs:76–175` already handles `LinkTarget::External { filename, object_path }` (writes link_type byte = 64, then packed filename+path). What's missing is a public API in `file_writer.rs` to create external links from a `GroupBuilder`. Currently `GroupBuilder` only creates datasets and sub-groups via `create_dataset` / `create_group`.

**Files:**
- Modify: `crates/clawhdf5-format/src/file_writer.rs` (add `GroupBuilder::add_external_link`)
- Modify: `crates/clawhdf5-format/src/lib.rs` (re-export `LinkTarget` if not already exported)
- Test: `crates/clawhdf5-format/src/file_writer.rs` (new test in `#[cfg(test)]`)

**Interfaces:**
- Produces: `GroupBuilder::add_external_link(&mut self, name: &str, target_file: &str, target_path: &str) -> &mut Self`

- [x] **Step 1: Write the failing test**

At the bottom of the `#[cfg(test)]` block in `crates/clawhdf5-format/src/file_writer.rs`, add:

```rust
#[test]
fn external_link_write_roundtrip() {
    use crate::group_v2::resolve_path_any;
    use crate::link_message::{LinkMessage, LinkTarget};
    use crate::message_type::MessageType;
    use crate::object_header::ObjectHeader;
    use crate::signature::find_signature;
    use crate::superblock::Superblock;

    let mut fw = FileWriter::new();
    let mut grp = fw.create_group("links");
    grp.add_external_link("remote_data", "other_file.h5", "/sensors/temp");
    fw.add_group(grp.finish());
    let bytes = fw.finish().unwrap();

    let sig = find_signature(&bytes).unwrap();
    let sb = Superblock::parse(&bytes, sig).unwrap();

    // Navigate to /links group
    let links_addr = resolve_path_any(&bytes, &sb, "links").unwrap();
    let links_oh = ObjectHeader::parse(
        &bytes, links_addr as usize, sb.offset_size, sb.length_size,
    ).unwrap();

    // Find the link message for "remote_data"
    let link_msg = links_oh.messages.iter()
        .filter(|m| m.msg_type == MessageType::Link)
        .find_map(|m| {
            let lm = LinkMessage::parse(&m.data, sb.offset_size).ok()?;
            if lm.name == "remote_data" { Some(lm) } else { None }
        })
        .expect("external link message not found");

    assert_eq!(
        link_msg.link_target,
        LinkTarget::External {
            filename: "other_file.h5".into(),
            object_path: "/sensors/temp".into(),
        }
    );
}
```

- [x] **Step 2: Run test to verify it fails**

```bash
cargo test -p clawhdf5-format external_link_write_roundtrip 2>&1 | head -20
```

Expected: compile error — `add_external_link` not found.

- [x] **Step 3: Find GroupBuilder in file_writer.rs and add the method**

Locate `GroupBuilder` in `crates/clawhdf5-format/src/file_writer.rs`. It tracks its items as a `Vec` of internal builders. Add a field for external links and the method:

First, locate the `GroupBuilder` struct definition and add a field:

```rust
pub struct GroupBuilder {
    name: String,
    datasets: Vec<DatasetBuilder>,
    groups: Vec<FinishedGroup>,
    external_links: Vec<(String, String, String)>, // (name, filename, object_path)
}
```

Update `GroupBuilder::new()` (or equivalent constructor) to initialize `external_links: Vec::new()`.

Add the public method immediately after the existing `create_dataset`/`create_group` methods:

```rust
/// Add a link in this group that points to an object in another HDF5 file.
///
/// `name` is the link name within this group.
/// `target_file` is the relative or absolute path to the target .h5 file.
/// `target_path` is the HDF5 path of the object within the target file.
pub fn add_external_link(
    &mut self,
    name: &str,
    target_file: &str,
    target_path: &str,
) -> &mut Self {
    self.external_links.push((
        name.to_string(),
        target_file.to_string(),
        target_path.to_string(),
    ));
    self
}
```

- [x] **Step 4: Wire external links into the group serialization**

Find where the `GroupBuilder` emits `LinkMessage` bytes during `finish()` / `build_group()`. For each external link, emit a `LinkMessage` with `LinkTarget::External`:

```rust
use crate::link_message::{LinkMessage, LinkTarget};
use crate::datatype::CharacterSet;

// Inside the loop/block that serializes links:
for (link_name, filename, object_path) in &self.external_links {
    let msg = LinkMessage {
        name: link_name.clone(),
        link_target: LinkTarget::External {
            filename: filename.clone(),
            object_path: object_path.clone(),
        },
        creation_order: None,
        charset: CharacterSet::Utf8,
    };
    let msg_bytes = msg.serialize(offset_size);
    // Emit as a Link message (MessageType::Link = 0x0006) into the object header
    emit_message(&mut oh_buf, MessageType::Link, &msg_bytes);
}
```

(Follow the exact pattern used for hard links and soft links in the same codebase — find where hard-link `LinkMessage` bytes are pushed and add the external links in the same loop.)

- [x] **Step 5: Run the failing test**

```bash
cargo test -p clawhdf5-format external_link_write_roundtrip 2>&1
```

Expected: PASS.

- [x] **Step 6: Run full suite**

```bash
cargo test -p clawhdf5-format 2>&1 | tail -10
```

Expected: all tests pass.

- [x] **Step 7: Commit**

```bash
git add crates/clawhdf5-format/src/file_writer.rs
git commit -m "feat: add GroupBuilder::add_external_link for writing cross-file HDF5 links"
```

---

### Task 2: VDS mapping serialization helper

**Background:** Reading VDS mappings from a global heap object is done by `parse_vds_mappings()` in `data_layout.rs:70–155`. Writing the inverse — serializing a `Vec<VdsMapping>` into the same binary layout — does not exist. This task creates `serialize_vds_mappings()`.

**Files:**
- Create: `crates/clawhdf5-format/src/data_layout_write.rs`
- Modify: `crates/clawhdf5-format/src/lib.rs` (declare module)

**Interfaces:**
- Consumes: `VdsMapping { source_file_name: String, source_dataset_name: String, source_selection: Vec<u8>, virtual_selection: Vec<u8> }` (existing struct from `data_layout.rs`).
- Produces: `pub fn serialize_vds_mappings(mappings: &[VdsMapping], length_size: u8) -> Vec<u8>`

**Binary layout (from `data_layout.rs:72–91` doc comment):**
```
version: u8 (0 = external file, 1 = same-file marker)
nused: length_size bytes (number of mappings)
for each mapping:
  if version==0: source_file_name (null-terminated)
  else: marker byte (0xFF or similar; same-file means empty filename)
  source_dataset_name: null-terminated string
  source_selection: length(length_size) + bytes
  virtual_selection: length(length_size) + bytes
```

- [x] **Step 1: Write the failing tests**

Create `crates/clawhdf5-format/src/data_layout_write.rs`:

```rust
//! Write-side helpers for VDS (Virtual Dataset Source) mapping serialization.

use crate::data_layout::{parse_vds_mappings, VdsMapping};
use crate::error::FormatError;

/// Serialize a slice of VDS mappings into the global-heap object byte format.
///
/// The output can be stored directly in a global heap object and referenced
/// from a Data Layout v4 class=3 (Virtual) message.
pub fn serialize_vds_mappings(mappings: &[VdsMapping], length_size: u8) -> Vec<u8> {
    let mut buf = Vec::new();

    // Determine if all sources are same-file (empty source_file_name)
    let has_external = mappings.iter().any(|m| !m.source_file_name.is_empty());
    let version: u8 = if has_external { 0 } else { 1 };
    buf.push(version);

    // nused: number of mappings
    write_length(&mut buf, mappings.len() as u64, length_size);

    for m in mappings {
        if version == 0 {
            // External: null-terminated filename
            buf.extend_from_slice(m.source_file_name.as_bytes());
            buf.push(0);
        } else {
            // Same-file: marker byte (0x00, which parse_vds_mappings treats as empty)
            buf.push(0);
        }
        // source dataset name: null-terminated
        buf.extend_from_slice(m.source_dataset_name.as_bytes());
        buf.push(0);
        // source selection: length + bytes
        write_length(&mut buf, m.source_selection.len() as u64, length_size);
        buf.extend_from_slice(&m.source_selection);
        // virtual selection: length + bytes
        write_length(&mut buf, m.virtual_selection.len() as u64, length_size);
        buf.extend_from_slice(&m.virtual_selection);
    }

    buf
}

fn write_length(buf: &mut Vec<u8>, val: u64, size: u8) {
    match size {
        2 => buf.extend_from_slice(&(val as u16).to_le_bytes()),
        4 => buf.extend_from_slice(&(val as u32).to_le_bytes()),
        8 => buf.extend_from_slice(&val.to_le_bytes()),
        _ => buf.extend_from_slice(&val.to_le_bytes()),
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    fn all_sel() -> Vec<u8> {
        // Minimal H5S ALL selection bytes: type=3 (ALL), version=1, flags=0, unused*4
        let mut v = Vec::new();
        v.extend_from_slice(&3u32.to_le_bytes()); // type = H5S_SEL_ALL
        v.push(1); // version
        v.push(0); // flags
        v.extend_from_slice(&[0u8; 4]); // unused
        v
    }

    #[test]
    fn roundtrip_same_file_two_mappings() {
        let sel = all_sel();
        let mappings = vec![
            VdsMapping {
                source_file_name: String::new(),
                source_dataset_name: "/src_a".into(),
                source_selection: sel.clone(),
                virtual_selection: sel.clone(),
            },
            VdsMapping {
                source_file_name: String::new(),
                source_dataset_name: "/src_b".into(),
                source_selection: sel.clone(),
                virtual_selection: sel.clone(),
            },
        ];
        let bytes = serialize_vds_mappings(&mappings, 8);
        let parsed = parse_vds_mappings(&bytes, 8).unwrap();
        assert_eq!(parsed.len(), 2);
        assert_eq!(parsed[0].source_dataset_name, "/src_a");
        assert_eq!(parsed[1].source_dataset_name, "/src_b");
    }

    #[test]
    fn roundtrip_external_file_mapping() {
        let sel = all_sel();
        let mappings = vec![VdsMapping {
            source_file_name: "source.h5".into(),
            source_dataset_name: "/data".into(),
            source_selection: sel.clone(),
            virtual_selection: sel.clone(),
        }];
        let bytes = serialize_vds_mappings(&mappings, 8);
        let parsed = parse_vds_mappings(&bytes, 8).unwrap();
        assert_eq!(parsed.len(), 1);
        assert_eq!(parsed[0].source_file_name, "source.h5");
        assert_eq!(parsed[0].source_dataset_name, "/data");
    }

    #[test]
    fn empty_mappings_roundtrip() {
        let bytes = serialize_vds_mappings(&[], 8);
        let parsed = parse_vds_mappings(&bytes, 8).unwrap();
        assert!(parsed.is_empty());
    }
}
```

- [x] **Step 2: Run the failing tests**

```bash
cargo test -p clawhdf5-format roundtrip_same_file_two_mappings roundtrip_external_file_mapping 2>&1 | head -20
```

Expected: compile errors (module not declared).

- [x] **Step 3: Declare module in lib.rs**

In `crates/clawhdf5-format/src/lib.rs`, add:

```rust
pub mod data_layout_write;
```

- [x] **Step 4: Run tests**

```bash
cargo test -p clawhdf5-format data_layout_write 2>&1
```

Expected: all 3 tests PASS. If `parse_vds_mappings` expects a slightly different format for the version byte or the marker byte, adjust `serialize_vds_mappings` to match what the parser consumes (read `data_layout.rs:92–155` carefully to align).

- [x] **Step 5: Commit**

```bash
git add crates/clawhdf5-format/src/data_layout_write.rs \
        crates/clawhdf5-format/src/lib.rs
git commit -m "feat: add serialize_vds_mappings for writing VDS global-heap objects"
```

---

### Task 3: FileWriter API for virtual datasets

**Background:** This task wires `serialize_vds_mappings()` into the `FileWriter` flow so callers can create a virtual dataset. It adds a new `DatasetBuilder` method and the corresponding serialization of a Data Layout v4 class=3 message.

**Files:**
- Modify: `crates/clawhdf5-format/src/file_writer.rs` (add `with_virtual_sources`)

**Interfaces:**
- Produces: `DatasetBuilder::with_virtual_sources(mappings: Vec<VdsMapping>) -> &mut Self`

**Binary — Data Layout v4 class=3 (Virtual):**
```
version(1)=4  class(1)=3
global_heap_address(offset_size)  global_heap_index(4)
```
The global heap object holds the `serialize_vds_mappings()` output. The `global_heap_address` is the address of the global heap collection; `global_heap_index` is the 1-based object index within it. Use index=1 for the first (and only) VDS object.

- [x] **Step 1: Write the failing test**

In `crates/clawhdf5-format/src/file_writer.rs` `#[cfg(test)]` block, add:

```rust
#[test]
fn virtual_dataset_write_roundtrip() {
    use crate::data_layout::DataLayout;
    use crate::data_layout::{VdsMapping, parse_vds_mappings};
    use crate::message_type::MessageType;
    use crate::object_header::ObjectHeader;
    use crate::signature::find_signature;
    use crate::superblock::Superblock;

    // Minimal ALL-selection bytes (same as in data_layout_write tests)
    let sel: Vec<u8> = {
        let mut v = Vec::new();
        v.extend_from_slice(&3u32.to_le_bytes()); // H5S_SEL_ALL
        v.push(1); v.push(0);
        v.extend_from_slice(&[0u8; 4]);
        v
    };

    let mappings = vec![VdsMapping {
        source_file_name: "src.h5".into(),
        source_dataset_name: "/raw".into(),
        source_selection: sel.clone(),
        virtual_selection: sel.clone(),
    }];

    let mut fw = FileWriter::new();
    fw.create_dataset("virtual_ds")
      .with_virtual_sources(mappings);
    let bytes = fw.finish().unwrap();

    // Parse back
    let sig = find_signature(&bytes).unwrap();
    let sb = Superblock::parse(&bytes, sig).unwrap();
    let root_oh = ObjectHeader::parse(
        &bytes, sb.root_group_address as usize, sb.offset_size, sb.length_size,
    ).unwrap();

    // Find the dataset via group traversal, then get its DataLayout message
    use crate::group_v2::resolve_path_any;
    let ds_addr = resolve_path_any(&bytes, &sb, "virtual_ds").unwrap();
    let ds_oh = ObjectHeader::parse(
        &bytes, ds_addr as usize, sb.offset_size, sb.length_size,
    ).unwrap();
    let dl_msg = ds_oh.messages.iter()
        .find(|m| m.msg_type == MessageType::DataLayout)
        .expect("DataLayout message missing");

    let layout = DataLayout::parse(&dl_msg.data, sb.offset_size, sb.length_size).unwrap();
    assert!(
        matches!(layout, DataLayout::Virtual { .. }),
        "expected Virtual layout, got {layout:?}"
    );
}
```

- [x] **Step 2: Run test to verify it fails**

```bash
cargo test -p clawhdf5-format virtual_dataset_write_roundtrip 2>&1 | head -20
```

Expected: compile error — `with_virtual_sources` not found.

- [x] **Step 3: Add with_virtual_sources to DatasetBuilder**

Find `DatasetBuilder` in `file_writer.rs`. Add a field `virtual_sources: Option<Vec<VdsMapping>>` and the method:

```rust
use crate::data_layout::VdsMapping;

// In DatasetBuilder struct:
virtual_sources: Option<Vec<VdsMapping>>,

// In DatasetBuilder impl:
pub fn with_virtual_sources(&mut self, mappings: Vec<VdsMapping>) -> &mut Self {
    self.virtual_sources = Some(mappings);
    self
}
```

- [x] **Step 4: Serialize the virtual data layout**

In the `DatasetBuilder::build()` or equivalent finish method, add a branch for virtual datasets:

```rust
use crate::data_layout_write::serialize_vds_mappings;

// Where the DataLayout message bytes are generated:
let layout_bytes = if let Some(mappings) = &self.virtual_sources {
    // Serialize VDS mappings into a global heap object
    let heap_data = serialize_vds_mappings(mappings, length_size);
    let (heap_addr, heap_idx) = write_global_heap_object(output_buf, &heap_data);
    // Data Layout v4 class=3 (Virtual): version(1)=4, class(1)=3, addr(offset_size), idx(4)
    let mut dl = Vec::new();
    dl.push(4u8); // version
    dl.push(3u8); // class = Virtual
    write_offset_val(&mut dl, heap_addr, offset_size);
    dl.extend_from_slice(&(heap_idx as u32).to_le_bytes());
    dl
} else {
    // existing layout code (contiguous/compact/chunked)
    build_existing_layout(...)
};
```

Implement `write_global_heap_object` as a helper that appends a minimal global heap collection to the output buffer and returns `(address, object_index)`:

```rust
/// Append a single-object global heap collection to `buf` and return
/// (collection_address, object_index=1).
fn write_global_heap_object(buf: &mut Vec<u8>, data: &[u8]) -> (u64, usize) {
    let addr = buf.len() as u64;
    // Global Heap Collection header: sig(4) + version(1) + reserved(3) + collection_size(8)
    // Object: index(2) + ref_count(2) + reserved(4) + data_size(8) + data + padding
    let obj_size = data.len();
    let padded = (obj_size + 7) & !7;
    let collection_size = 16 + 16 + padded + 8; // header + one obj header + data + sentinel
    buf.extend_from_slice(b"GCOL"); // signature
    buf.push(1); // version
    buf.extend_from_slice(&[0u8; 3]); // reserved
    buf.extend_from_slice(&(collection_size as u64).to_le_bytes());
    // Object 1
    buf.extend_from_slice(&1u16.to_le_bytes()); // index
    buf.extend_from_slice(&1u16.to_le_bytes()); // ref_count
    buf.extend_from_slice(&[0u8; 4]); // reserved
    buf.extend_from_slice(&(obj_size as u64).to_le_bytes());
    buf.extend_from_slice(data);
    // Pad to 8-byte boundary
    let pad = padded - obj_size;
    buf.extend_from_slice(&vec![0u8; pad]);
    // Sentinel object (index=0)
    buf.extend_from_slice(&[0u8; 8]); // index=0 + ref_count + reserved
    buf.extend_from_slice(&0u64.to_le_bytes()); // size=0
    (addr, 1)
}
```

- [x] **Step 5: Run the test**

```bash
cargo test -p clawhdf5-format virtual_dataset_write_roundtrip 2>&1
```

Expected: PASS (or iterate on the global heap format until `parse_vds_mappings` reads back the mappings).

- [x] **Step 6: Run full suite**

```bash
cargo test -p clawhdf5-format 2>&1 | tail -10
```

Expected: all tests pass.

- [x] **Step 7: Commit**

```bash
git add crates/clawhdf5-format/src/file_writer.rs
git commit -m "feat: add DatasetBuilder::with_virtual_sources for writing VDS data layout"
```

---

### Task 4: Superblock v4 read support

**Background:** `Superblock::parse()` in `superblock.rs:178–183` returns `Err(FormatError::UnsupportedVersion(v))` for any version ≥ 4. Superblock v4 (introduced with HDF5 2.x page-buffering) shares the same 12-byte header as v2/v3 (`sig + version + offset_size + length_size + consistency_flags`) and the same four address fields, but adds a `page_size: u32` field before the trailing checksum.

**Files:**
- Modify: `crates/clawhdf5-format/src/superblock.rs`

**Interfaces:**
- Consumes/produces: `Superblock` struct — add `pub page_size: Option<u32>` field.

- [x] **Step 1: Add field to Superblock struct**

In `crates/clawhdf5-format/src/superblock.rs`, add to the `Superblock` struct:

```rust
/// Page size for page-buffer mode (v4 only). `None` for v0–v3.
pub page_size: Option<u32>,
```

Update all existing construction sites of `Superblock { ... }` in the file (parse_v0, parse_v1, parse_v2v3) to include `page_size: None`.

- [x] **Step 2: Write the failing test**

In the `#[cfg(test)]` section of `superblock.rs`, add:

```rust
#[test]
fn parse_v4_with_page_size() {
    // Superblock v4 = v2/v3 layout + page_size(4) before checksum.
    let mut buf = Vec::new();
    buf.extend_from_slice(&crate::signature::HDF5_SIGNATURE);
    buf.push(4);  // version = 4
    buf.push(8);  // offset_size
    buf.push(8);  // length_size
    buf.push(0);  // consistency_flags
    // base_address
    buf.extend_from_slice(&0u64.to_le_bytes());
    // superblock_extension_address = UNDEF
    buf.extend_from_slice(&u64::MAX.to_le_bytes());
    // eof_address
    buf.extend_from_slice(&512u64.to_le_bytes());
    // root_group_address
    buf.extend_from_slice(&96u64.to_le_bytes());
    // page_size (v4 addition before checksum)
    buf.extend_from_slice(&4096u32.to_le_bytes());
    // checksum (4 bytes; compute with jenkins_lookup3)
    let checksum = crate::checksum::jenkins_lookup3(&buf);
    buf.extend_from_slice(&checksum.to_le_bytes());

    let sb = Superblock::parse(&buf, 0).unwrap();
    assert_eq!(sb.version, 4);
    assert_eq!(sb.offset_size, 8);
    assert_eq!(sb.eof_address, 512);
    assert_eq!(sb.page_size, Some(4096));
}
```

- [x] **Step 3: Run test to verify it fails**

```bash
cargo test -p clawhdf5-format parse_v4_with_page_size 2>&1 | head -20
```

Expected: `Err(UnsupportedVersion(4))` — the test fails because v4 isn't handled.

- [x] **Step 4: Add parse_v4 branch**

In `Superblock::parse()`, change:

```rust
2 | 3 => Self::parse_v2v3(d, version),
v => Err(FormatError::UnsupportedVersion(v)),
```

to:

```rust
2 | 3 => Self::parse_v2v3(d, version),
4 => Self::parse_v4(d),
v => Err(FormatError::UnsupportedVersion(v)),
```

Add the implementation:

```rust
fn parse_v4(d: &[u8]) -> Result<Superblock, FormatError> {
    // Same as v2/v3 header, then page_size(4), then checksum(4).
    ensure_len(d, 12)?;
    let offset_size = d[9];
    let length_size = d[10];
    validate_sizes(offset_size, length_size)?;
    let consistency_flags = d[11] as u32;

    let os = offset_size as usize;
    // 4 addresses + page_size(4) + checksum(4)
    let total = 12 + 4 * os + 4 + 4;
    ensure_len(d, total)?;

    let mut pos = 12;
    let base_address = read_offset(d, pos, offset_size)?;
    pos += os;
    let superblock_extension_address = read_offset(d, pos, offset_size)?;
    pos += os;
    let eof_address = read_offset(d, pos, offset_size)?;
    pos += os;
    let root_group_address = read_offset(d, pos, offset_size)?;
    pos += os;

    let page_size = u32::from_le_bytes([d[pos], d[pos+1], d[pos+2], d[pos+3]]);
    pos += 4;

    let stored_checksum = u32::from_le_bytes([d[pos], d[pos+1], d[pos+2], d[pos+3]]);
    let computed = crate::checksum::jenkins_lookup3(&d[..pos]);
    if stored_checksum != computed {
        return Err(FormatError::ChecksumMismatch {
            expected: stored_checksum,
            computed,
        });
    }

    Ok(Superblock {
        version: 4,
        offset_size,
        length_size,
        base_address,
        eof_address,
        root_group_address,
        group_leaf_node_k: None,
        group_internal_node_k: None,
        indexed_storage_internal_node_k: None,
        free_space_address: None,
        driver_info_address: None,
        consistency_flags,
        superblock_extension_address: Some(superblock_extension_address),
        checksum: Some(stored_checksum),
        page_size: Some(page_size),
    })
}
```

- [x] **Step 5: Run the test**

```bash
cargo test -p clawhdf5-format parse_v4_with_page_size 2>&1
```

Expected: PASS (verify the checksum field name matches whatever `FormatError` uses — it may be `ChecksumMismatch { expected, computed }` or similar; find it in `error.rs` and match).

- [x] **Step 6: Run full suite**

```bash
cargo test -p clawhdf5-format 2>&1 | tail -10
```

Expected: all tests pass.

- [x] **Step 7: Commit**

```bash
git add crates/clawhdf5-format/src/superblock.rs
git commit -m "feat: parse HDF5 superblock v4 (page-buffer mode) with page_size field"
```

---

### Task 5: Superblock v4 write support

**Background:** The `FileWriter` always writes a v3 superblock (hardcoded in `file_writer.rs:1291–1306`). This task adds an optional `page_size` to `FileWriter` that, when set, emits a v4 superblock.

**Files:**
- Modify: `crates/clawhdf5-format/src/file_writer.rs` (add `page_size` field)
- Modify: `crates/clawhdf5-format/src/superblock.rs` (`Superblock::serialize` for v4)

**Interfaces:**
- Produces: `FileWriter::with_page_size(page_size: u32) -> &mut Self`

- [x] **Step 1: Write the failing test**

In the `#[cfg(test)]` block of `file_writer.rs`, add:

```rust
#[test]
fn file_writer_v4_superblock() {
    use crate::signature::find_signature;
    use crate::superblock::Superblock;

    let mut fw = FileWriter::new();
    fw.with_page_size(4096);
    fw.create_dataset("data").with_f64_data(&[1.0, 2.0]);
    let bytes = fw.finish().unwrap();

    let sig = find_signature(&bytes).unwrap();
    let sb = Superblock::parse(&bytes, sig).unwrap();
    assert_eq!(sb.version, 4, "expected superblock v4");
    assert_eq!(sb.page_size, Some(4096));
}
```

- [x] **Step 2: Run test to verify it fails**

```bash
cargo test -p clawhdf5-format file_writer_v4_superblock 2>&1 | head -20
```

Expected: compile error — `with_page_size` not found.

- [x] **Step 3: Add page_size field to FileWriter**

In `FileWriter` struct definition, add `page_size: Option<u32>`.
In `FileWriter::new()`, add `page_size: None`.
Add method:

```rust
pub fn with_page_size(&mut self, page_size: u32) -> &mut Self {
    self.page_size = Some(page_size);
    self
}
```

- [x] **Step 4: Update Superblock::serialize for v4**

In `crates/clawhdf5-format/src/superblock.rs`, the `serialize()` method currently hardcodes v2/v3 format. Update it to emit v4 when `self.version == 4` and `self.page_size.is_some()`:

```rust
pub fn serialize(&self) -> Vec<u8> {
    let mut buf = Vec::with_capacity(60);
    buf.extend_from_slice(&HDF5_SIGNATURE);
    buf.push(self.version);
    buf.push(self.offset_size);
    buf.push(self.length_size);
    buf.push(self.consistency_flags as u8);
    Self::write_offset(&mut buf, self.base_address, self.offset_size);
    let ext_addr = self.superblock_extension_address.unwrap_or(u64::MAX);
    Self::write_offset(&mut buf, ext_addr, self.offset_size);
    Self::write_offset(&mut buf, self.eof_address, self.offset_size);
    Self::write_offset(&mut buf, self.root_group_address, self.offset_size);
    if self.version >= 4 {
        let ps = self.page_size.unwrap_or(0);
        buf.extend_from_slice(&ps.to_le_bytes());
    }
    let checksum = crate::checksum::jenkins_lookup3(&buf);
    buf.extend_from_slice(&checksum.to_le_bytes());
    buf
}
```

- [x] **Step 5: Wire page_size into FileWriter::finish()**

In `file_writer.rs:finish()`, where the `Superblock` is constructed (around line 1291), change:

```rust
let sb = Superblock {
    version: if self.page_size.is_some() { 4 } else { 3 },
    // ... existing fields ...
    page_size: self.page_size,
    // ... rest of fields unchanged ...
};
```

- [x] **Step 6: Run the test**

```bash
cargo test -p clawhdf5-format file_writer_v4_superblock 2>&1
```

Expected: PASS.

- [x] **Step 7: Run full suite**

```bash
cargo test -p clawhdf5-format 2>&1 | tail -10
```

Expected: all tests pass (v3 serialize() must be byte-identical to before — add a regression test if needed).

- [x] **Step 8: Commit**

```bash
git add crates/clawhdf5-format/src/file_writer.rs \
        crates/clawhdf5-format/src/superblock.rs
git commit -m "feat: write HDF5 superblock v4 when page_size is configured"
```

---

## Verification

```bash
# Run all format tests
cargo test -p clawhdf5-format 2>&1 | tail -10

# Specifically verify new features
cargo test -p clawhdf5-format external_link_write_roundtrip 2>&1
cargo test -p clawhdf5-format data_layout_write 2>&1
cargo test -p clawhdf5-format virtual_dataset_write_roundtrip 2>&1
cargo test -p clawhdf5-format parse_v4_with_page_size 2>&1
cargo test -p clawhdf5-format file_writer_v4_superblock 2>&1

# Regression: v3 superblock still round-trips
cargo test -p clawhdf5-format write_superblock 2>&1
```