feat: assemble 1-D same-file Virtual Datasets (VDS)

A virtual layout previously returned UnsupportedVersion. Implement reading
for the common 1-D, same-file case, reverse-engineered and validated against
HDF5 2.0.

- Rewrite parse_vds_mappings to the real global-heap block format
  (version(1) · nused(length_size) · entries · checksum(4)), where each
  entry is source-file(null) · source-dataset(null) · source-selection ·
  virtual-selection. Block version 1 encodes a same-file source as a single
  0x04 marker in place of the file name; version 0 stores an explicit file
  name. The selections are H5S-serialized and self-describing in length, so
  they are decoded to find entry boundaries. The previous parser used a
  guessed layout that did not match real files.

- Extend Selection with decode_serialized() (H5S_select_serialize: ALL,
  NONE, and version-3 regular hyperslabs) and iter_linear_1d().

- Add read_virtual_data: resolve the mapping block from the global heap,
  read each same-file source dataset, and scatter its selected elements into
  the virtual buffer; unmapped regions stay at the zero fill value.
  External-file sources and N-D selections return a clean unsupported error.

Tests: real-file integration test (vds_same_file.h5: partial source slice +
fill gap), selection decoder unit tests built from the fixture bytes, and
same-file/external mapping-parser unit tests.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

CHANGELOG.md

@@ -14,6 +14,19 @@
   previously unsupported. Signed and unsigned reduced-precision integers and
   float members all read end-to-end, validated against HDF5 2.0.
 
+### New Features
+- `clawhdf5-format`: assemble **1-D, same-file Virtual Datasets (VDS)**.
+  Previously a virtual layout returned `UnsupportedVersion`. The reader now
+  decodes the global-heap mapping block (reverse-engineered against HDF5 2.0:
+  `version · nused · [source-file · source-dataset · source-selection ·
+  virtual-selection]* · checksum`, including the block-version-1 same-file
+  marker), decodes the `H5S` source/virtual dataspace **selections** (ALL,
+  NONE, and version-3 regular hyperslabs), reads each same-file source dataset,
+  and scatters its selected elements into the virtual buffer; unmapped regions
+  are left at the zero fill value. External-file sources and N-dimensional
+  selections return a clean unsupported error. The previous `parse_vds_mappings`
+  used a guessed layout that did not match real files and is replaced.
+
 ### Bug Fixes
 - `clawhdf5-format`: read **paged Fixed Array** chunk indexes. A filtered,
   fixed-dimension dataset with more than one data-block page (>1024 chunks by

crates/clawhdf5-format/src/data_layout.rs

@@ -67,74 +67,65 @@ pub enum DataLayout {
     },
 }
 
-/// Parse VDS mappings from global heap object data.
+/// Parse VDS mappings from global-heap object data.
 ///
-/// The global heap object for a VDS layout contains a serialized list of
+/// The global-heap block holding a VDS mapping list is laid out as
-/// source mappings. Each mapping has:
+/// (reverse-engineered and validated against HDF5 2.0):
-/// - Virtual selection (serialized dataspace selection, variable length)
-/// - Source file name (null-terminated string)
-/// - Source dataset name (null-terminated string)
-/// - Source selection (serialized dataspace selection, variable length)
 ///
-/// The overall format starts with:
+/// ```text
-/// - version (4 bytes LE) — currently 0
+/// version(1) · nused(length_size, LE) · entry[nused] · checksum(4)
-/// - entry count (not explicitly stored; parse until data exhausted)
+/// ```
 ///
-/// This is a best-effort parser that handles common VDS files. The exact
+/// Each entry is:
-/// binary format is not fully specified publicly and may vary by HDF5 version.
+/// - source file name — a null-terminated string in **block version 0**; in
-pub fn parse_vds_mappings(heap_data: &[u8]) -> Result<Vec<VdsMapping>, FormatError> {
+///   **block version 1** a same-file reference is encoded as a single `0x04`
-    if heap_data.len() < 4 {
+///   marker byte (the source file is the virtual file itself) in place of the
+///   name;
+/// - source dataset name (null-terminated string);
+/// - source selection (serialized `H5S` dataspace selection — self-describing
+///   in length);
+/// - virtual selection (serialized `H5S` dataspace selection).
+///
+/// The selections are decoded with [`crate::selection::Selection`] purely to
+/// learn their byte length so the entry list can be walked; the raw selection
+/// bytes are retained on each [`VdsMapping`] for the reader to interpret.
+pub fn parse_vds_mappings(
+    heap_data: &[u8],
+    length_size: u8,
+) -> Result<Vec<VdsMapping>, FormatError> {
+    use crate::selection::Selection;
+
+    let ls = length_size as usize;
+    if heap_data.len() < 1 + ls {
         return Ok(Vec::new());
     }
-    // VDS global heap object starts with version(4)
+    let version = heap_data[0];
-    let _version = u32::from_le_bytes([heap_data[0], heap_data[1], heap_data[2], heap_data[3]]);
+    let mut pos = 1;
-    let mut pos = 4;
+    let nused = read_length(heap_data, pos, length_size)?;
-    let mut mappings = Vec::new();
+    pos += ls;
 
-    while pos < heap_data.len() {
+    let mut mappings = Vec::with_capacity(nused as usize);
-        // Each entry: virtual_selection_size(4) + virtual_selection(N) +
+    for _ in 0..nused {
-        //             source_file_name(null-term) + source_dataset_name(null-term) +
+        // Source file name (with the version-1 same-file marker handled).
-        //             source_selection_size(4) + source_selection(N)
+        let source_file = if version >= 1 && heap_data.get(pos) == Some(&0x04) {
-        if pos + 4 > heap_data.len() {
+            pos += 1;
-            break;
+            String::from(".")
-        }
+        } else {
+            read_null_terminated_string(heap_data, &mut pos)?
+        };
 
-        // Virtual selection
+        // Source dataset name.
-        let vsel_size = u32::from_le_bytes([
-            heap_data[pos],
-            heap_data[pos + 1],
-            heap_data[pos + 2],
-            heap_data[pos + 3],
-        ]) as usize;
-        pos += 4;
-        if pos + vsel_size > heap_data.len() {
-            break;
-        }
-        let virtual_selection = heap_data[pos..pos + vsel_size].to_vec();
-        pos += vsel_size;
-
-        // Source file name (null-terminated)
-        let source_file = read_null_terminated_string(heap_data, &mut pos)?;
-
-        // Source dataset name (null-terminated)
         let source_dataset = read_null_terminated_string(heap_data, &mut pos)?;
 
-        // Source selection
+        // Source selection (self-describing length).
-        if pos + 4 > heap_data.len() {
+        let (_, ssel_len) = Selection::decode_serialized(&heap_data[pos..])?;
-            break;
+        let source_selection = heap_data[pos..pos + ssel_len].to_vec();
-        }
+        pos += ssel_len;
-        let ssel_size = u32::from_le_bytes([
+
-            heap_data[pos],
+        // Virtual selection.
-            heap_data[pos + 1],
+        let (_, vsel_len) = Selection::decode_serialized(&heap_data[pos..])?;
-            heap_data[pos + 2],
+        let virtual_selection = heap_data[pos..pos + vsel_len].to_vec();
-            heap_data[pos + 3],
+        pos += vsel_len;
-        ]) as usize;
-        pos += 4;
-        if pos + ssel_size > heap_data.len() {
-            break;
-        }
-        let source_selection = heap_data[pos..pos + ssel_size].to_vec();
-        pos += ssel_size;
 
         mappings.push(VdsMapping {
             source_file,
@@ -235,7 +226,7 @@ impl DataLayout {
                     index: *global_heap_index as u16,
                 },
             )?;
-            *mappings = parse_vds_mappings(&obj.data)?;
+            *mappings = parse_vds_mappings(&obj.data, length_size)?;
         }
         Ok(())
     }
@@ -774,32 +765,54 @@ mod tests {
     }
 
     #[test]
-    fn parse_vds_mappings_basic() {
+    fn parse_vds_mappings_same_file_v1() {
-        // Build a simple VDS mapping blob
+        // The exact global-heap block written by HDF5 2.0 for a same-file VDS
-        let mut blob = Vec::new();
+        // with two sources: src_a -> virtual[0:4], src_b -> virtual[4:8].
-        blob.extend_from_slice(&0u32.to_le_bytes()); // version=0
+        let blob = [
+            0x01u8, // block version 1
+            0x02, 0, 0, 0, 0, 0, 0, 0, // nused = 2 (length_size = 8)
+            // entry 0
+            0x04, // same-file marker (replaces file name)
+            0x73, 0x72, 0x63, 0x5f, 0x61, 0x00, // "src_a\0"
+            0x03, 0, 0, 0, 0x01, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // source sel = ALL
+            0x02, 0, 0, 0, 0x03, 0, 0, 0, 0x01, 0x02, 0x01, 0, 0, 0, // virtual sel: HYPER v3
+            0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x04, 0x00, // start0 stride1 count1 block4
+            // entry 1
+            0x04, 0x73, 0x72, 0x63, 0x5f, 0x62, 0x00, // "src_b\0"
+            0x03, 0, 0, 0, 0x01, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // source sel = ALL
+            0x02, 0, 0, 0, 0x03, 0, 0, 0, 0x01, 0x02, 0x01, 0, 0, 0, // virtual sel: HYPER v3
+            0x04, 0x00, 0x01, 0x00, 0x01, 0x00, 0x04, 0x00, // start4 stride1 count1 block4
+            0x68, 0xf0, 0x3e, 0xe4, // checksum (ignored)
+        ];
+        let mappings = parse_vds_mappings(&blob, 8).unwrap();
+        assert_eq!(mappings.len(), 2);
+        assert_eq!(mappings[0].source_file, ".");
+        assert_eq!(mappings[0].source_dataset, "src_a");
+        assert_eq!(mappings[1].source_file, ".");
+        assert_eq!(mappings[1].source_dataset, "src_b");
 
-        // Virtual selection (8 bytes of dummy data)
+        // Virtual selections decode to [0:4] and [4:8].
-        let vsel = vec![1, 2, 3, 4, 5, 6, 7, 8];
+        use crate::selection::Selection;
-        blob.extend_from_slice(&(vsel.len() as u32).to_le_bytes());
+        let (v0, _) = Selection::decode_serialized(&mappings[0].virtual_selection).unwrap();
-        blob.extend_from_slice(&vsel);
+        let (v1, _) = Selection::decode_serialized(&mappings[1].virtual_selection).unwrap();
+        assert_eq!(v0.iter_linear_1d(8).unwrap(), vec![0, 1, 2, 3]);
+        assert_eq!(v1.iter_linear_1d(8).unwrap(), vec![4, 5, 6, 7]);
+    }
 
-        // Source file name
+    #[test]
-        blob.extend_from_slice(b"source.h5\0");
+    fn parse_vds_mappings_external_v0() {
-
+        // Block version 0 with an explicit (external) source file name.
-        // Source dataset name
+        let blob = [
-        blob.extend_from_slice(b"/data\0");
+            0x00u8, // block version 0
-
+            0x01, 0, 0, 0, 0, 0, 0, 0, // nused = 1
-        // Source selection (4 bytes)
+            0x73, 0x72, 0x63, 0x5f, 0x65, 0x78, 0x74, 0x2e, 0x68, 0x35, 0x00, // "src_ext.h5\0"
-        let ssel = vec![10, 20, 30, 40];
+            0x64, 0x61, 0x74, 0x61, 0x00, // "data\0"
-        blob.extend_from_slice(&(ssel.len() as u32).to_le_bytes());
+            0x03, 0, 0, 0, 0x01, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // source sel = ALL
-        blob.extend_from_slice(&ssel);
+            0x03, 0, 0, 0, 0x01, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // virtual sel = ALL
-
+        ];
-        let mappings = parse_vds_mappings(&blob).unwrap();
+        let mappings = parse_vds_mappings(&blob, 8).unwrap();
         assert_eq!(mappings.len(), 1);
-        assert_eq!(mappings[0].source_file, "source.h5");
+        assert_eq!(mappings[0].source_file, "src_ext.h5");
-        assert_eq!(mappings[0].source_dataset, "/data");
+        assert_eq!(mappings[0].source_dataset, "data");
-        assert_eq!(mappings[0].virtual_selection, vsel);
-        assert_eq!(mappings[0].source_selection, ssel);
     }
 }

crates/clawhdf5-format/src/data_read.rs

@@ -128,7 +128,19 @@ pub fn read_raw_data_full(
             offset_size,
             length_size,
         ),
-        DataLayout::Virtual { .. } => Err(FormatError::UnsupportedVersion(0)),
+        DataLayout::Virtual {
+            global_heap_address,
+            global_heap_index,
+            ..
+        } => read_virtual_data(
+            file_data,
+            *global_heap_address,
+            *global_heap_index,
+            dataspace,
+            datatype,
+            offset_size,
+            length_size,
+        ),
     }
 }
 
@@ -355,10 +367,153 @@ pub fn read_raw_data_selection(
             )?;
             extract_selection_from_buffer(&full_data, dims, elem_size, selection)
         }
-        DataLayout::Virtual { .. } => Err(FormatError::UnsupportedVersion(0)),
+        DataLayout::Virtual { .. } => {
+            // Assemble the full virtual dataset, then apply the read selection.
+            let full_data = read_raw_data_full(
+                file_data,
+                layout,
+                dataspace,
+                datatype,
+                pipeline,
+                offset_size,
+                length_size,
+            )?;
+            extract_selection_from_buffer(&full_data, dims, elem_size, selection)
+        }
     }
 }
 
+/// Assemble a **Virtual Dataset (VDS)** from its source mappings.
+///
+/// Supports **1-D, same-file** virtual datasets: each mapping's source dataset
+/// is read from the same file and its selected elements are scattered into the
+/// virtual buffer at the positions given by the virtual selection. Unmapped
+/// regions are left at the zero fill value.
+///
+/// External-file sources and N-dimensional selections are reported as
+/// unsupported rather than silently producing wrong data.
+fn read_virtual_data(
+    file_data: &[u8],
+    global_heap_address: Option<u64>,
+    global_heap_index: u32,
+    dataspace: &Dataspace,
+    datatype: &Datatype,
+    offset_size: u8,
+    length_size: u8,
+) -> Result<Vec<u8>, FormatError> {
+    use crate::data_layout::parse_vds_mappings;
+    use crate::global_heap::GlobalHeapCollection;
+    use crate::selection::Selection;
+
+    let elem_size = datatype.type_size() as usize;
+    let total_elems = dataspace.num_elements() as usize;
+    let mut out = vec![0u8; total_elems.saturating_mul(elem_size)];
+
+    if dataspace.dimensions.len() > 1 {
+        return Err(FormatError::ChunkedReadError(
+            "N-dimensional virtual datasets are not supported".into(),
+        ));
+    }
+    let virtual_extent = dataspace
+        .dimensions
+        .first()
+        .copied()
+        .unwrap_or(total_elems as u64);
+
+    let addr = global_heap_address.ok_or_else(|| {
+        FormatError::ChunkedReadError("virtual dataset has no mapping global heap".into())
+    })?;
+    let coll = GlobalHeapCollection::parse(file_data, addr as usize, length_size)?;
+    let obj = coll
+        .get_object(global_heap_index as u16)
+        .ok_or(FormatError::GlobalHeapObjectNotFound {
+            collection_address: addr,
+            index: global_heap_index as u16,
+        })?;
+    let mappings = parse_vds_mappings(&obj.data, length_size)?;
+
+    for m in &mappings {
+        // Only same-file sources are reachable through the pure-byte read API.
+        if !(m.source_file.is_empty() || m.source_file == ".") {
+            return Err(FormatError::ChunkedReadError(
+                "external-file virtual dataset sources are not supported".into(),
+            ));
+        }
+
+        let (vsel, _) = Selection::decode_serialized(&m.virtual_selection)?;
+        let (ssel, _) = Selection::decode_serialized(&m.source_selection)?;
+
+        let (src_raw, src_elems) =
+            read_named_dataset_raw(file_data, &m.source_dataset, offset_size, length_size)?;
+
+        let vidx = vsel.iter_linear_1d(virtual_extent)?;
+        let sidx = ssel.iter_linear_1d(src_elems as u64)?;
+        if vidx.len() != sidx.len() {
+            return Err(FormatError::ChunkedReadError(
+                "virtual/source selection element counts differ".into(),
+            ));
+        }
+
+        for (&v, &s) in vidx.iter().zip(sidx.iter()) {
+            let (vo, so) = (v as usize * elem_size, s as usize * elem_size);
+            if vo + elem_size > out.len() || so + elem_size > src_raw.len() {
+                return Err(FormatError::ChunkedReadError(
+                    "virtual dataset selection out of bounds".into(),
+                ));
+            }
+            out[vo..vo + elem_size].copy_from_slice(&src_raw[so..so + elem_size]);
+        }
+    }
+
+    Ok(out)
+}
+
+/// Read a named dataset's raw (decoded) bytes and element count, navigating
+/// from the superblock. Used to pull VDS source datasets out of the same file.
+fn read_named_dataset_raw(
+    file_data: &[u8],
+    path: &str,
+    _offset_size: u8,
+    _length_size: u8,
+) -> Result<(Vec<u8>, usize), FormatError> {
+    use crate::filter_pipeline::FilterPipeline;
+    use crate::group_v2::resolve_path_any;
+    use crate::message_type::MessageType;
+    use crate::object_header::ObjectHeader;
+    use crate::signature::find_signature;
+    use crate::superblock::Superblock;
+
+    let sig = find_signature(file_data)?;
+    let sb = Superblock::parse(file_data, sig)?;
+    let addr = resolve_path_any(file_data, &sb, path)?;
+    let hdr = ObjectHeader::parse(file_data, addr as usize, sb.offset_size, sb.length_size)?;
+
+    let find = |t: MessageType| hdr.messages.iter().find(|m| m.msg_type == t);
+    let ds_msg = find(MessageType::Dataspace)
+        .ok_or_else(|| FormatError::ChunkedReadError("VDS source has no dataspace".into()))?;
+    let dataspace = Dataspace::parse(&ds_msg.data, sb.length_size)?;
+    let dt_msg = find(MessageType::Datatype)
+        .ok_or_else(|| FormatError::ChunkedReadError("VDS source has no datatype".into()))?;
+    let (datatype, _) = Datatype::parse(&dt_msg.data)?;
+    let dl_msg = find(MessageType::DataLayout)
+        .ok_or_else(|| FormatError::ChunkedReadError("VDS source has no data layout".into()))?;
+    let layout = DataLayout::parse(&dl_msg.data, sb.offset_size, sb.length_size)?;
+    let pipeline = find(MessageType::FilterPipeline)
+        .map(|m| FilterPipeline::parse(&m.data))
+        .transpose()?;
+
+    let raw = read_raw_data_full(
+        file_data,
+        &layout,
+        &dataspace,
+        &datatype,
+        pipeline.as_ref(),
+        sb.offset_size,
+        sb.length_size,
+    )?;
+    Ok((raw, dataspace.num_elements() as usize))
+}
+
 /// Extract selected elements from a full dataset buffer.
 fn extract_selection_from_buffer(
     full_data: &[u8],

crates/clawhdf5-format/src/selection.rs

@@ -19,6 +19,8 @@ use alloc::{vec, vec::Vec};
 
 use core::ops::Range;
 
+use crate::error::FormatError;
+
 /// A selection describing which elements of a dataset to access.
 #[derive(Debug, Clone, PartialEq)]
 pub enum Selection {
@@ -220,6 +222,169 @@ impl Selection {
             }
         }
     }
+
+    /// Decode a selection from its on-disk **`H5S_select_serialize`** form.
+    ///
+    /// Returns the selection and the number of bytes consumed (selections are
+    /// self-describing in length, so the count lets a caller walk a packed list
+    /// of selections — as the Virtual Dataset global-heap block does).
+    ///
+    /// Only the forms needed for VDS assembly are decoded: `ALL`, `NONE`, and
+    /// **regular** hyperslabs serialized at **version 3** (the encoding HDF5
+    /// 1.10+/2.0 emit). Point selections, irregular hyperslabs, and older
+    /// hyperslab versions return an error rather than mis-decoding.
+    pub fn decode_serialized(data: &[u8]) -> Result<(Selection, usize), FormatError> {
+        if data.len() < 8 {
+            return Err(FormatError::UnexpectedEof {
+                expected: 8,
+                available: data.len(),
+            });
+        }
+        let sel_type = u32::from_le_bytes([data[0], data[1], data[2], data[3]]);
+        let version = u32::from_le_bytes([data[4], data[5], data[6], data[7]]);
+
+        match sel_type {
+            // ALL / NONE: type(4) + version(4) + reserved(4) + length(4) = 16 bytes.
+            3 => Ok((Selection::All, 16)),
+            0 => Ok((Selection::None, 16)),
+            2 => decode_hyperslab_serialized(data, version),
+            1 => Err(FormatError::ChunkedReadError(
+                "VDS point selections are not supported".into(),
+            )),
+            _ => Err(FormatError::ChunkedReadError(
+                "unknown dataspace selection type".into(),
+            )),
+        }
+    }
+
+    /// Enumerate the selected element indices of a **1-D** dataspace of the
+    /// given `extent`, in selection order.
+    ///
+    /// Returns an error for selections of rank != 1 (N-dimensional VDS
+    /// assembly is not supported in this build).
+    pub fn iter_linear_1d(&self, extent: u64) -> Result<Vec<u64>, FormatError> {
+        match self {
+            Selection::All => Ok((0..extent).collect()),
+            Selection::None => Ok(Vec::new()),
+            Selection::Hyperslab {
+                start,
+                stride,
+                count,
+                block,
+            } => {
+                if start.len() != 1 {
+                    return Err(FormatError::ChunkedReadError(
+                        "only 1-D VDS hyperslab selections are supported".into(),
+                    ));
+                }
+                let (s, st, c, b) = (start[0], stride[0], count[0], block[0]);
+                let mut out = Vec::new();
+                for ci in 0..c {
+                    let base = s + ci * st;
+                    for bi in 0..b {
+                        let idx = base + bi;
+                        if idx >= extent {
+                            return Err(FormatError::ChunkedReadError(
+                                "VDS hyperslab selection exceeds dataspace extent".into(),
+                            ));
+                        }
+                        out.push(idx);
+                    }
+                }
+                Ok(out)
+            }
+            Selection::Points(pts) => {
+                let mut out = Vec::with_capacity(pts.len());
+                for p in pts {
+                    if p.len() != 1 {
+                        return Err(FormatError::ChunkedReadError(
+                            "only 1-D VDS point selections are supported".into(),
+                        ));
+                    }
+                    if p[0] >= extent {
+                        return Err(FormatError::ChunkedReadError(
+                            "VDS point selection exceeds dataspace extent".into(),
+                        ));
+                    }
+                    out.push(p[0]);
+                }
+                Ok(out)
+            }
+        }
+    }
+}
+
+/// Decode an `H5S_SEL_HYPER` selection in its serialized form. Only version-3
+/// **regular** hyperslabs are supported.
+fn decode_hyperslab_serialized(
+    data: &[u8],
+    version: u32,
+) -> Result<(Selection, usize), FormatError> {
+    if version != 3 {
+        return Err(FormatError::ChunkedReadError(
+            "only version-3 hyperslab selections are supported".into(),
+        ));
+    }
+    // type(4) ver(4) flags(1) enc_size(1) rank(4) [start,stride,count,block]*rank
+    if data.len() < 14 {
+        return Err(FormatError::UnexpectedEof {
+            expected: 14,
+            available: data.len(),
+        });
+    }
+    let flags = data[8];
+    let enc_size = data[9] as usize;
+    // Bit 0 set => regular hyperslab. Irregular hyperslabs list explicit blocks.
+    if flags & 0x01 == 0 {
+        return Err(FormatError::ChunkedReadError(
+            "irregular VDS hyperslab selections are not supported".into(),
+        ));
+    }
+    if enc_size != 2 && enc_size != 4 && enc_size != 8 {
+        return Err(FormatError::ChunkedReadError(
+            "unsupported hyperslab coordinate encoding size".into(),
+        ));
+    }
+    let rank = u32::from_le_bytes([data[10], data[11], data[12], data[13]]) as usize;
+    let mut pos = 14;
+    let read_coord = |data: &[u8], pos: usize| -> Result<u64, FormatError> {
+        if pos + enc_size > data.len() {
+            return Err(FormatError::UnexpectedEof {
+                expected: pos + enc_size,
+                available: data.len(),
+            });
+        }
+        let mut v = 0u64;
+        for (i, &b) in data[pos..pos + enc_size].iter().enumerate() {
+            v |= (b as u64) << (i * 8);
+        }
+        Ok(v)
+    };
+    let (mut start, mut stride, mut count, mut block) = (
+        Vec::with_capacity(rank),
+        Vec::with_capacity(rank),
+        Vec::with_capacity(rank),
+        Vec::with_capacity(rank),
+    );
+    for _ in 0..rank {
+        start.push(read_coord(data, pos)?);
+        pos += enc_size;
+        stride.push(read_coord(data, pos)?);
+        pos += enc_size;
+        count.push(read_coord(data, pos)?);
+        pos += enc_size;
+        block.push(read_coord(data, pos)?);
+        pos += enc_size;
+    }
+    Ok((
+        Selection::Hyperslab {
+            start,
+            stride,
+            count,
+            block,
+        },
+        pos,
+    ))
 }
 
 // ---------------------------------------------------------------------------
@@ -313,4 +478,79 @@ mod tests {
         // Chunk [9..10] should not intersect (only row 9, but selection ends at row 8)
         assert!(!sel.intersects_chunk(&[9], &[1]));
     }
+
+    #[test]
+    fn decode_all_selection_16_bytes() {
+        let bytes = [3u8, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0];
+        let (sel, consumed) = Selection::decode_serialized(&bytes).unwrap();
+        assert_eq!(sel, Selection::All);
+        assert_eq!(consumed, 16);
+        assert_eq!(sel.iter_linear_1d(4).unwrap(), vec![0, 1, 2, 3]);
+    }
+
+    #[test]
+    fn decode_regular_hyperslab_matches_vds_fixture() {
+        // Exact virtual selection for src_a in the VDS fixture:
+        // start=0 stride=1 count=1 block=4, version 3, enc_size 2, rank 1.
+        let bytes = [
+            0x02, 0, 0, 0, // type = HYPER
+            0x03, 0, 0, 0, // version 3
+            0x01, // flags = regular
+            0x02, // enc_size = 2
+            0x01, 0, 0, 0, // rank = 1
+            0x00, 0x00, // start
+            0x01, 0x00, // stride
+            0x01, 0x00, // count
+            0x04, 0x00, // block
+        ];
+        let (sel, consumed) = Selection::decode_serialized(&bytes).unwrap();
+        assert_eq!(consumed, 22);
+        assert_eq!(
+            sel,
+            Selection::Hyperslab {
+                start: vec![0],
+                stride: vec![1],
+                count: vec![1],
+                block: vec![4],
+            }
+        );
+        assert_eq!(sel.iter_linear_1d(8).unwrap(), vec![0, 1, 2, 3]);
+    }
+
+    #[test]
+    fn decode_hyperslab_start4() {
+        let bytes = [
+            0x02, 0, 0, 0, 0x03, 0, 0, 0, 0x01, 0x02, 0x01, 0, 0, 0, //
+            0x04, 0x00, 0x01, 0x00, 0x01, 0x00, 0x04, 0x00,
+        ];
+        let (sel, _) = Selection::decode_serialized(&bytes).unwrap();
+        assert_eq!(sel.iter_linear_1d(8).unwrap(), vec![4, 5, 6, 7]);
+    }
+
+    #[test]
+    fn decode_strided_hyperslab_iter() {
+        // start=1 stride=3 count=2 block=2 => 1,2, 4,5
+        let bytes = [
+            0x02, 0, 0, 0, 0x03, 0, 0, 0, 0x01, 0x02, 0x01, 0, 0, 0, //
+            0x01, 0x00, 0x03, 0x00, 0x02, 0x00, 0x02, 0x00,
+        ];
+        let (sel, _) = Selection::decode_serialized(&bytes).unwrap();
+        assert_eq!(sel.iter_linear_1d(8).unwrap(), vec![1, 2, 4, 5]);
+    }
+
+    #[test]
+    fn decode_nd_hyperslab_iter_rejected() {
+        let bytes = [
+            0x02, 0, 0, 0, 0x03, 0, 0, 0, 0x01, 0x02, 0x02, 0, 0, 0, // rank 2
+            0, 0, 1, 0, 1, 0, 2, 0, 0, 0, 1, 0, 1, 0, 2, 0,
+        ];
+        let (sel, _) = Selection::decode_serialized(&bytes).unwrap();
+        assert!(sel.iter_linear_1d(16).is_err());
+    }
+
+    #[test]
+    fn decode_irregular_hyperslab_rejected() {
+        let bytes = [0x02u8, 0, 0, 0, 0x03, 0, 0, 0, 0x00, 0x02, 0x01, 0, 0, 0];
+        assert!(Selection::decode_serialized(&bytes).is_err());
+    }
 }

crates/clawhdf5-format/tests/integration_test.rs

@@ -662,6 +662,22 @@ fn v4_fixed_array_read() {
     }
 }
 
+#[test]
+fn v4_virtual_dataset_same_file_read() {
+    // A 1-D virtual dataset assembled from two same-file sources:
+    //   virt[0:4]  <- src_a[1:5]  (partial source hyperslab) => 11,12,13,14
+    //   virt[4:8]  <- (unmapped)                              => fill 0
+    //   virt[8:12] <- src_b[0:4]  (ALL)                       => 20,21,22,23
+    let file_data = include_bytes!("fixtures/vds_same_file.h5");
+    let (raw, datatype, _) = read_chunked_dataset(file_data, "virt");
+    let values = read_as_i32(&raw, &datatype).unwrap();
+    assert_eq!(
+        values,
+        vec![11, 12, 13, 14, 0, 0, 0, 0, 20, 21, 22, 23],
+        "VDS assembly (partial source slice + fill gap) mismatch"
+    );
+}
+
 #[test]
 fn v4_paged_fixed_array_read() {
     // 1025 chunks of 16 int32s, gzip-filtered => Fixed Array index whose data