Fast contiguous and concurrent reads, VL data, nested groups and links, Python bindings #15
+18
-2
@@ -61,6 +61,22 @@
|
||||
object running past its collection. Conformance unchanged at 575 of 697
|
||||
(`crates/clawhdf5-format/tests/vl_heap_bounds.rs`).
|
||||
|
||||
- **Every reader resolves VL data the same way.** `h5rs` (`dump`, `ls`,
|
||||
`diff`, `check --data`) had its own lenient VL decoder: a heap object
|
||||
longer than the element's length was cut to it (h5py refuses it), a null
|
||||
string printed `""` where h5dump prints `NULL`, the stored element size
|
||||
was trusted, and each heap collection was kept as a copy for the whole
|
||||
run. It now resolves through `VlResolver`, so `dump` matches h5dump byte
|
||||
for byte on VL strings (`"a\0b"` as `"a"`, null as `NULL`), VL sequences
|
||||
and 4-byte-offset files, `dump --json` gives h5py's values, and
|
||||
`check --data` reports any heap object whose size is not exactly the
|
||||
element's length × base size. `clawhdf5-wasm` already resolved VL strings
|
||||
with `read_vl_strings`; it now uses `VlResolver` and refuses a VL type
|
||||
whose stored element size disagrees with the file, as `File` does
|
||||
(`crates/clawhdf5-tools/tests/h5rs_interop.rs`,
|
||||
`crates/clawhdf5-wasm/tests/vl_strings.rs`). New
|
||||
`VlResolver::element` / `string_element` resolve one element in place.
|
||||
|
||||
### Plugin filters (2026-09-26)
|
||||
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
|
||||
written by h5py with `compression="lzf"`, or with hdf5plugin's
|
||||
@@ -264,10 +280,10 @@
|
||||
printed with its address; exit 1 when there are any. libhdf5's h5check
|
||||
reads only the 1.8 format. On the conformance corpus it passes all 418
|
||||
files that both clawhdf5 and h5py read in full, and `check --data` flags
|
||||
134 of the 150 CVE and fuzzer files of the `cve_hdf5` corpus (tank,
|
||||
135 of the 150 CVE and fuzzer files of the `cve_hdf5` corpus (tank,
|
||||
2026-09-26). `--data` also follows variable-length data into its global
|
||||
heap collections and reports a damaged one at its address. It inherits
|
||||
the library's tolerance, though: 9 of the 16 it passes are files h5dump
|
||||
the library's tolerance, though: 8 of the 15 it passes are files h5dump
|
||||
1.14.6 rejects (see `docs/known-issues.md`, header checks).
|
||||
- Values over `--max-bytes` (default 1 GiB) are reported instead of read;
|
||||
a panic is caught and reported as an internal error (exit 3).
|
||||
|
||||
@@ -76,8 +76,13 @@ print the same bytes as h5dump 1.14.6 and as Debian's h5dump 1.14.5 (the
|
||||
was run in that image on 2026-09-26) — `dump_matches_h5dump` in
|
||||
`tests/h5rs_interop.rs` checks this, and `dump_shows_nul_padding_in_nested_strings`
|
||||
that null-padded strings show their NULs (`"a\000b"`) at any depth, as
|
||||
h5dump's do. Not covered by those tests: references, opaque, bitfield,
|
||||
variable-length sequences and virtual datasets. Known differences from
|
||||
h5dump's do. `dump_prints_vl_data_like_h5dump` covers variable-length
|
||||
strings (one with an embedded NUL, which prints up to the NUL; empty; null,
|
||||
which prints `NULL`), variable-length sequences, a VL compound member and
|
||||
a VL attribute, with 8- and 4-byte offsets. Not covered by those tests:
|
||||
references, opaque, bitfield, non-ASCII UTF-8 (h5dump prints each byte
|
||||
above 0x7f as a sign-extended octal escape, h5rs the character) and
|
||||
virtual datasets. Known differences from
|
||||
h5dump:
|
||||
|
||||
- Floats print at their own precision (a `float32` 0.1 prints as `0.1`),
|
||||
@@ -233,9 +238,11 @@ extension) and checks:
|
||||
(which catches corrupt compressed data and Fletcher-32 mismatches), and
|
||||
follows every variable-length element (strings and sequences, also inside
|
||||
compounds and arrays) of every dataset and attribute into its global heap
|
||||
collection: a collection that does not parse, a missing heap object, or a
|
||||
sequence longer than its heap object is a problem at the collection's
|
||||
address. Data the
|
||||
collection: a collection that does not parse or overlaps another, a missing
|
||||
heap object, or a heap object whose size is not exactly the element's
|
||||
length times its base size (libhdf5 refuses such an element) is a problem
|
||||
at the collection's address. Variable-length elements are resolved by the
|
||||
library's `VlResolver`, as `clawhdf5::File` resolves them. Data the
|
||||
tool cannot decode (a filter it does not implement, such as szip, or a
|
||||
dataset over `--max-bytes`) is a `note:`, not a problem. Every problem is
|
||||
printed with the address of the structure involved; the exit status is 0
|
||||
@@ -252,9 +259,10 @@ none at all without `--data`), and objects reachable only by external links. It
|
||||
clawhdf5's parsers, so it accepts what they accept: some header damage that
|
||||
libhdf5 refuses goes unreported. Of the 150 CVE and fuzzer files of the
|
||||
HDF Group's `cve_hdf5` corpus (`cvefiles/` and `fuzzerfiles/`),
|
||||
`check --data` passes 16, and h5dump 1.14.6 rejects 9 of those (tank,
|
||||
`check --data` passes 15, and h5dump 1.14.6 rejects 8 of those (tank,
|
||||
2026-09-26, `h5rs check --data F` and `h5dump F` per file; before the
|
||||
library's header checks it passed 28, of which h5dump rejects 21).
|
||||
library's header checks it passed 28, of which h5dump rejects 21, and 16
|
||||
and 9 before a VL type's stored element size was checked).
|
||||
|
||||
## Robustness
|
||||
|
||||
|
||||
@@ -15,11 +15,13 @@ use clawhdf5_format::btree_v2::{BTreeV2Header, collect_btree_v2_records};
|
||||
use clawhdf5_format::data_layout::DataLayout;
|
||||
use clawhdf5_format::dataspace::{Dataspace, DataspaceType};
|
||||
use clawhdf5_format::datatype::Datatype;
|
||||
use clawhdf5_format::error::FormatError;
|
||||
use clawhdf5_format::group_info::GroupInfoMessage;
|
||||
use clawhdf5_format::link_info::LinkInfoMessage;
|
||||
use clawhdf5_format::message_type::MessageType;
|
||||
use clawhdf5_format::object_header::ObjectHeader;
|
||||
use clawhdf5_format::symbol_table::SymbolTableMessage;
|
||||
use clawhdf5_format::vl_data::{VlResolver, check_element_size, parse_vl_references};
|
||||
|
||||
use crate::cli::{Args, Out};
|
||||
use crate::h5::{Error, ErrorKind, H5, Kind};
|
||||
@@ -49,6 +51,15 @@ found, 3 internal error.";
|
||||
|
||||
const MAX_CHUNKS_CHECKED: usize = 10_000_000;
|
||||
|
||||
/// A variable-length element's problem, worded as `check` reports heap
|
||||
/// problems ("global heap ...").
|
||||
fn heap_problem(e: FormatError) -> String {
|
||||
match e {
|
||||
FormatError::VlDataError(m) if m.starts_with("global heap") => m,
|
||||
e => format!("global heap: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether values of `dt` hold variable-length data (in the global heap).
|
||||
fn has_vl(dt: &Datatype, depth: u32) -> bool {
|
||||
if depth > 32 {
|
||||
@@ -104,6 +115,8 @@ struct Checker<'a> {
|
||||
btrees_seen: HashSet<u64>,
|
||||
/// Global heap collections already read (with --data).
|
||||
gcols_seen: HashSet<u64>,
|
||||
/// Resolves variable-length elements (with --data), for the whole file.
|
||||
vl: VlResolver<'a>,
|
||||
panicked: bool,
|
||||
}
|
||||
|
||||
@@ -157,6 +170,7 @@ pub fn run(args: &mut Args, out: &mut Out) -> std::io::Result<i32> {
|
||||
heaps_seen: HashSet::new(),
|
||||
btrees_seen: HashSet::new(),
|
||||
gcols_seen: HashSet::new(),
|
||||
vl: VlResolver::new(h5.data(), h5.os(), h5.ls()),
|
||||
panicked: false,
|
||||
};
|
||||
c.superblock();
|
||||
@@ -707,62 +721,48 @@ impl Checker<'_> {
|
||||
}
|
||||
match dt {
|
||||
Datatype::VariableLength {
|
||||
size,
|
||||
is_string,
|
||||
base_type,
|
||||
..
|
||||
} => {
|
||||
let os = usize::from(self.h5.os());
|
||||
let (Some(lenb), Some(addrb), Some(idxb)) =
|
||||
(b.get(..4), b.get(4..4 + os), b.get(4 + os..8 + os))
|
||||
else {
|
||||
// Resolved by the library's VlResolver, as every other
|
||||
// reader resolves them (and as libhdf5 does): a heap object
|
||||
// whose size is not the element's length × base size, a
|
||||
// collection that overlaps another, or a missing object is
|
||||
// a problem at the collection's address.
|
||||
let Ok(vl) = parse_vl_references(b, 1, self.h5.os()) else {
|
||||
return;
|
||||
};
|
||||
let le = |x: &[u8]| {
|
||||
x.iter()
|
||||
.enumerate()
|
||||
.fold(0u64, |a, (i, &v)| a | (u64::from(v) << (8 * i)))
|
||||
};
|
||||
let (len, gcol, idx) = (le(lenb), le(addrb), le(idxb));
|
||||
let undef = if os >= 8 {
|
||||
u64::MAX
|
||||
} else {
|
||||
(1u64 << (8 * os)) - 1
|
||||
};
|
||||
if len == 0 || gcol == 0 || gcol == undef || bad.contains_key(&gcol) {
|
||||
let gcol = vl[0].collection_address;
|
||||
if gcol == 0 || bad.contains_key(&gcol) {
|
||||
return;
|
||||
}
|
||||
let obj = match self.h5.heap_object(gcol, idx as u32) {
|
||||
Ok(o) => o,
|
||||
if let Err(e) = check_element_size(*size, self.h5.os()) {
|
||||
bad.insert(gcol, e.to_string());
|
||||
return;
|
||||
}
|
||||
let bs = if *is_string {
|
||||
1
|
||||
} else {
|
||||
base_type.type_size() as usize
|
||||
};
|
||||
if bs == 0 {
|
||||
return;
|
||||
}
|
||||
let obj = match self.vl.element(b, bs) {
|
||||
Ok(o) => o.unwrap_or(&[]),
|
||||
Err(e) => {
|
||||
bad.insert(e.addr.unwrap_or(gcol), e.msg);
|
||||
bad.insert(gcol, heap_problem(e));
|
||||
return;
|
||||
}
|
||||
};
|
||||
if self.gcols_seen.insert(gcol) {
|
||||
self.counts.global_heaps += 1;
|
||||
}
|
||||
let bs = if *is_string {
|
||||
1
|
||||
} else {
|
||||
u64::from(base_type.type_size())
|
||||
};
|
||||
if len
|
||||
.checked_mul(bs)
|
||||
.is_none_or(|need| need > obj.len() as u64)
|
||||
{
|
||||
bad.insert(
|
||||
gcol,
|
||||
format!(
|
||||
"global heap object {idx} holds {} bytes; the element needs {len} x {bs}",
|
||||
obj.len()
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if !*is_string && bs > 0 && has_vl(base_type, depth + 1) {
|
||||
let bs = bs as usize;
|
||||
for k in 0..len as usize {
|
||||
self.vl_element(base_type, &obj[k * bs..(k + 1) * bs], depth + 1, bad);
|
||||
if !*is_string && has_vl(base_type, depth + 1) {
|
||||
for eb in obj.chunks_exact(bs) {
|
||||
self.vl_element(base_type, eb, depth + 1, bad);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -699,6 +699,9 @@ impl Diff {
|
||||
}
|
||||
match (x, y) {
|
||||
(Value::Str(p), Value::Str(q)) => p == q,
|
||||
// h5diff compares a null VL string equal to an empty one.
|
||||
(Value::NullStr, Value::NullStr) => true,
|
||||
(Value::NullStr, Value::Str(s)) | (Value::Str(s), Value::NullStr) => s.is_empty(),
|
||||
(Value::Bytes(p), Value::Bytes(q)) | (Value::OtherRef(p), Value::OtherRef(q)) => p == q,
|
||||
(Value::Compound(p), Value::Compound(q)) => {
|
||||
p.len() == q.len()
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
use std::cell::RefCell;
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::rc::Rc;
|
||||
|
||||
use clawhdf5::File;
|
||||
use clawhdf5_format::attribute::{AttributeMessage, extract_attributes_tolerant};
|
||||
@@ -20,7 +19,6 @@ use clawhdf5_format::datatype::Datatype;
|
||||
use clawhdf5_format::error::FormatError;
|
||||
use clawhdf5_format::filter_pipeline::FilterPipeline;
|
||||
use clawhdf5_format::fractal_heap::FractalHeapHeader;
|
||||
use clawhdf5_format::global_heap::GlobalHeapCollection;
|
||||
use clawhdf5_format::group_v1;
|
||||
use clawhdf5_format::link_info::LinkInfoMessage;
|
||||
use clawhdf5_format::link_message::{LinkMessage, LinkTarget};
|
||||
@@ -191,7 +189,6 @@ pub struct H5 {
|
||||
pub path: PathBuf,
|
||||
pub file: File,
|
||||
pub max_bytes: u64,
|
||||
heaps: RefCell<HashMap<u64, std::result::Result<Rc<GlobalHeapCollection>, String>>>,
|
||||
/// Fractal heaps whose blocks were verified: `None` = sound.
|
||||
verified_heaps: RefCell<HashMap<u64, Option<Error>>>,
|
||||
}
|
||||
@@ -211,7 +208,6 @@ impl H5 {
|
||||
path: path.to_path_buf(),
|
||||
file,
|
||||
max_bytes: DEFAULT_MAX_BYTES,
|
||||
heaps: RefCell::new(HashMap::new()),
|
||||
verified_heaps: RefCell::new(HashMap::new()),
|
||||
})
|
||||
}
|
||||
@@ -433,29 +429,6 @@ impl H5 {
|
||||
r.map_or(Ok(()), Err)
|
||||
}
|
||||
|
||||
/// The global heap object `idx` of the collection at `addr` (cached per
|
||||
/// collection).
|
||||
pub fn heap_object(&self, addr: u64, idx: u32) -> Result<Vec<u8>> {
|
||||
let coll = {
|
||||
let mut cache = self.heaps.borrow_mut();
|
||||
cache
|
||||
.entry(addr)
|
||||
.or_insert_with(|| match usize::try_from(addr) {
|
||||
Ok(a) => GlobalHeapCollection::parse(self.data(), a, self.ls())
|
||||
.map(Rc::new)
|
||||
.map_err(|e| e.to_string()),
|
||||
Err(_) => Err("address out of range".into()),
|
||||
})
|
||||
.clone()
|
||||
.map_err(|e| Error::at(addr, format!("global heap: {e}")))?
|
||||
};
|
||||
let idx16 = u16::try_from(idx)
|
||||
.map_err(|_| Error::at(addr, format!("global heap object index {idx} out of range")))?;
|
||||
coll.get_object(idx16)
|
||||
.map(|o| o.data.clone())
|
||||
.ok_or_else(|| Error::at(addr, format!("global heap has no object {idx}")))
|
||||
}
|
||||
|
||||
/// The dataspace of the dataset at `path` with a virtual dataset's
|
||||
/// extent resolved from its sources (as libhdf5 reports it) instead of
|
||||
/// the stored one.
|
||||
|
||||
@@ -3,7 +3,10 @@
|
||||
//! Decoding never panics: a short buffer, an unknown byte order or a
|
||||
//! dangling heap reference becomes [`Value::Error`].
|
||||
|
||||
use std::cell::RefCell;
|
||||
|
||||
use clawhdf5_format::datatype::{Datatype, DatatypeByteOrder, ReferenceType, StringPadding};
|
||||
use clawhdf5_format::vl_data::{VlResolver, check_element_size};
|
||||
use serde_json::Value as J;
|
||||
|
||||
use crate::dtype;
|
||||
@@ -16,6 +19,9 @@ pub enum Value {
|
||||
/// its own precision.
|
||||
Float(f64, u8),
|
||||
Str(String),
|
||||
/// A null variable-length string (heap address 0): h5dump prints it as
|
||||
/// `NULL`, h5py reads it as empty.
|
||||
NullStr,
|
||||
/// Opaque, bitfield, time and oversized integers.
|
||||
Bytes(Vec<u8>),
|
||||
/// An enum member (name, when the value matches one) and its value.
|
||||
@@ -129,10 +135,10 @@ fn decode_float(dt: &Datatype, b: &[u8]) -> Value {
|
||||
}
|
||||
}
|
||||
|
||||
fn trim_string(b: &[u8], pad: Option<&StringPadding>) -> String {
|
||||
fn trim_string(b: &[u8], pad: &StringPadding) -> String {
|
||||
let cut = b.iter().position(|&c| c == 0).unwrap_or(b.len());
|
||||
let mut s = &b[..cut];
|
||||
if matches!(pad, Some(StringPadding::SpacePad)) {
|
||||
if matches!(pad, StringPadding::SpacePad) {
|
||||
while let [rest @ .., b' '] = s {
|
||||
s = rest;
|
||||
}
|
||||
@@ -140,22 +146,20 @@ fn trim_string(b: &[u8], pad: Option<&StringPadding>) -> String {
|
||||
String::from_utf8_lossy(s).into_owned()
|
||||
}
|
||||
|
||||
/// Little-endian unsigned integer of `b` (up to 8 bytes).
|
||||
fn le(b: &[u8]) -> u64 {
|
||||
b.iter()
|
||||
.take(8)
|
||||
.enumerate()
|
||||
.fold(0u64, |a, (i, &x)| a | (u64::from(x) << (8 * i)))
|
||||
}
|
||||
|
||||
/// Decodes elements of one file.
|
||||
pub struct Decoder<'a> {
|
||||
pub h5: &'a H5,
|
||||
/// Variable-length elements are resolved as the library resolves them
|
||||
/// (so as libhdf5 does), not by a decoder of our own.
|
||||
vl: RefCell<VlResolver<'a>>,
|
||||
}
|
||||
|
||||
impl<'a> Decoder<'a> {
|
||||
pub fn new(h5: &'a H5) -> Self {
|
||||
Self { h5 }
|
||||
Self {
|
||||
h5,
|
||||
vl: RefCell::new(VlResolver::new(h5.data(), h5.os(), h5.ls())),
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode element `i` of `raw`, an array of `dt` elements.
|
||||
@@ -187,7 +191,7 @@ impl<'a> Decoder<'a> {
|
||||
Datatype::Time { .. } | Datatype::BitField { .. } | Datatype::Opaque { .. } => {
|
||||
Value::Bytes(b.to_vec())
|
||||
}
|
||||
Datatype::String { padding, .. } => Value::Str(trim_string(b, Some(padding))),
|
||||
Datatype::String { padding, .. } => Value::Str(trim_string(b, padding)),
|
||||
Datatype::Compound { members, .. } => {
|
||||
let mut out = Vec::with_capacity(members.len());
|
||||
for m in members {
|
||||
@@ -246,61 +250,43 @@ impl<'a> Decoder<'a> {
|
||||
Value::Array(out)
|
||||
}
|
||||
Datatype::VariableLength {
|
||||
size,
|
||||
is_string,
|
||||
padding,
|
||||
base_type,
|
||||
..
|
||||
} => self.decode_vlen(*is_string, padding.as_ref(), base_type, b, depth),
|
||||
} => match check_element_size(*size, self.h5.os()) {
|
||||
Ok(()) => self.decode_vlen(*is_string, base_type, b, depth),
|
||||
Err(e) => Value::Error(e.to_string()),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn decode_vlen(
|
||||
&self,
|
||||
is_string: bool,
|
||||
padding: Option<&StringPadding>,
|
||||
base: &Datatype,
|
||||
b: &[u8],
|
||||
depth: u32,
|
||||
) -> Value {
|
||||
let os = usize::from(self.h5.os());
|
||||
let (Some(lenb), Some(addrb), Some(idxb)) =
|
||||
(b.get(..4), b.get(4..4 + os), b.get(4 + os..8 + os))
|
||||
else {
|
||||
return Value::Error("short VL element".into());
|
||||
};
|
||||
let len = le(lenb) as usize;
|
||||
let addr = le(addrb);
|
||||
let idx = le(idxb) as u32;
|
||||
let undef = if os >= 8 {
|
||||
u64::MAX
|
||||
} else {
|
||||
(1u64 << (8 * os)) - 1
|
||||
};
|
||||
let obj = if len == 0 || addr == 0 || addr == undef {
|
||||
Vec::new()
|
||||
} else {
|
||||
match self.h5.heap_object(addr, idx) {
|
||||
Ok(o) => o,
|
||||
Err(e) => return Value::Error(e.to_string()),
|
||||
}
|
||||
};
|
||||
/// A variable-length element, resolved by the library's
|
||||
/// [`VlResolver`]: a string ends at its first NUL, a heap object whose
|
||||
/// size is not the element's length × base size is an error, and a
|
||||
/// heap address of 0 is null — all as libhdf5 (and so h5dump and h5py)
|
||||
/// has it.
|
||||
fn decode_vlen(&self, is_string: bool, base: &Datatype, b: &[u8], depth: u32) -> Value {
|
||||
if is_string {
|
||||
let l = len.min(obj.len());
|
||||
return Value::Str(trim_string(&obj[..l], padding));
|
||||
return match self.vl.borrow_mut().string_element(b) {
|
||||
Ok(Some(s)) => Value::Str(String::from_utf8_lossy(s).into_owned()),
|
||||
Ok(None) => Value::NullStr,
|
||||
Err(e) => Value::Error(e.to_string()),
|
||||
};
|
||||
}
|
||||
let bs = base.type_size() as usize;
|
||||
if bs == 0 {
|
||||
return Value::Error("VL base type of size 0".into());
|
||||
}
|
||||
match len.checked_mul(bs) {
|
||||
Some(need) if need <= obj.len() => {}
|
||||
_ => return Value::Error("VL sequence longer than its heap object".into()),
|
||||
}
|
||||
let mut out = Vec::with_capacity(len);
|
||||
for k in 0..len {
|
||||
out.push(self.decode(base, &obj[k * bs..], depth + 1));
|
||||
}
|
||||
Value::Seq(out)
|
||||
let obj = match self.vl.borrow_mut().element(b, bs) {
|
||||
Ok(o) => o.unwrap_or(&[]),
|
||||
Err(e) => return Value::Error(e.to_string()),
|
||||
};
|
||||
Value::Seq(
|
||||
obj.chunks_exact(bs)
|
||||
.map(|e| self.decode(base, e, depth + 1))
|
||||
.collect(),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -351,6 +337,7 @@ pub fn text(v: &Value, h5paths: &dyn Fn(u64) -> Option<String>) -> String {
|
||||
Value::Int(i) => i.to_string(),
|
||||
Value::Float(f, w) => fmt_float(*f, *w),
|
||||
Value::Str(s) => format!("\"{}\"", escape(s)),
|
||||
Value::NullStr => "NULL".into(),
|
||||
Value::Bytes(b) => hex(b),
|
||||
Value::Enum(Some(n), _) => n.clone(),
|
||||
Value::Enum(None, i) => i.to_string(),
|
||||
@@ -411,6 +398,7 @@ pub fn to_json(v: &Value, h5paths: &dyn Fn(u64) -> Option<String>) -> J {
|
||||
}
|
||||
}
|
||||
Value::Str(s) => J::from(s.as_str()),
|
||||
Value::NullStr => J::from(""),
|
||||
Value::Bytes(b) | Value::OtherRef(b) => J::from(hex(b)),
|
||||
Value::Enum(_, i) => to_json(&Value::Int(*i), h5paths),
|
||||
Value::Compound(ms) => J::Array(ms.iter().map(|(_, v)| to_json(v, h5paths)).collect()),
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
"""Write the variable-length data files the h5rs VL tests run on.
|
||||
|
||||
usage: gen_vl_files.py OUTDIR
|
||||
|
||||
For 8-byte (`vl8`) and 4-byte (`vl4`) offsets, writes OUTDIR/vl8.h5 and
|
||||
OUTDIR/vl4.h5, which libhdf5 reads in full, and OUTDIR/bad8.h5 and
|
||||
OUTDIR/bad4.h5, whose `bad` and `badseq` elements 0 have a length that
|
||||
disagrees with their global heap object (libhdf5: "Expected global heap
|
||||
object size does not match"). h5py cannot write a VL string with a NUL in
|
||||
it or a null element in a contiguous dataset, so those are patched in.
|
||||
|
||||
Prints one JSON object: for each file, each dataset's values as h5py reads
|
||||
them one element at a time (strings as text, sequences as lists, a compound
|
||||
as a list of its fields), with null for an element h5py cannot read; the
|
||||
root attribute `va`; and the addresses of the `bad` elements' collections.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
|
||||
import h5py
|
||||
import numpy as np
|
||||
|
||||
out = sys.argv[1]
|
||||
S = h5py.string_dtype("utf-8")
|
||||
I4 = h5py.vlen_dtype(np.dtype("<i4"))
|
||||
|
||||
|
||||
def create(path, sizes):
|
||||
if sizes is None:
|
||||
return h5py.File(path, "w")
|
||||
fcpl = h5py.h5p.create(h5py.h5p.FILE_CREATE)
|
||||
fcpl.set_sizes(*sizes)
|
||||
return h5py.File(h5py.h5f.create(path.encode(), h5py.h5f.ACC_TRUNC, fcpl=fcpl))
|
||||
|
||||
|
||||
def element(length, addr, index, os_):
|
||||
return struct.pack("<I", length) + addr.to_bytes(os_, "little") + struct.pack("<I", index)
|
||||
|
||||
|
||||
def good(path, sizes):
|
||||
os_ = 8 if sizes is None else sizes[0]
|
||||
with create(path, sizes) as f:
|
||||
f.create_dataset(
|
||||
"d", data=np.array(["aXb", "", "ok", "zz", "hello"], dtype=object), dtype=S
|
||||
)
|
||||
u = f.create_dataset("u", shape=(4,), dtype=S, chunks=(1,))
|
||||
u[1] = "w"
|
||||
s = f.create_dataset("seq", shape=(3,), dtype=I4)
|
||||
s[0] = [1, 2, 3]
|
||||
s[1] = []
|
||||
s[2] = [-5]
|
||||
s = f.create_dataset("sequ", shape=(3,), dtype=I4, chunks=(1,))
|
||||
s[0] = [7, 8]
|
||||
ct = np.dtype([("id", "<i4"), ("name", S)])
|
||||
arr = np.zeros(3, dtype=ct)
|
||||
arr["id"] = [1, 2, 3]
|
||||
arr["name"] = ["one", "", "three"]
|
||||
f.create_dataset("cmp", data=arr)
|
||||
f.attrs.create("va", np.array(["p", "", "q"], dtype=object), dtype=S)
|
||||
off = f["d"].id.get_offset()
|
||||
b = bytearray(open(path, "rb").read())
|
||||
i = b.index(b"aXb")
|
||||
b[i + 1] = 0 # "a\0b"
|
||||
es = 8 + os_
|
||||
b[off + 2 * es : off + 3 * es] = element(2, 0, 1, os_) # "ok" -> null
|
||||
open(path, "wb").write(bytes(b))
|
||||
|
||||
|
||||
def bad(path, sizes):
|
||||
os_ = 8 if sizes is None else sizes[0]
|
||||
with create(path, sizes) as f:
|
||||
f.create_dataset("bad", data=np.array(["cdefgh", "ok"], dtype=object), dtype=S)
|
||||
s = f.create_dataset("badseq", shape=(2,), dtype=I4)
|
||||
s[0] = [1, 2, 3]
|
||||
s[1] = [4]
|
||||
off, soff = f["bad"].id.get_offset(), f["badseq"].id.get_offset()
|
||||
b = bytearray(open(path, "rb").read())
|
||||
gcol = int.from_bytes(b[off + 4 : off + 4 + os_], "little")
|
||||
struct.pack_into("<I", b, off, 3) # "cdefgh": length 6 -> 3
|
||||
struct.pack_into("<I", b, soff, 2) # [1, 2, 3]: length 3 -> 2
|
||||
open(path, "wb").write(bytes(b))
|
||||
return gcol
|
||||
|
||||
|
||||
def value(v):
|
||||
if isinstance(v, bytes):
|
||||
return v.decode()
|
||||
if isinstance(v, str):
|
||||
return v
|
||||
if isinstance(v, np.void):
|
||||
return [value(x) for x in v]
|
||||
if isinstance(v, np.ndarray):
|
||||
return [value(x) for x in v]
|
||||
return v.item() if hasattr(v, "item") else v
|
||||
|
||||
|
||||
def read(ds):
|
||||
got = []
|
||||
for i in range(ds.shape[0]):
|
||||
try:
|
||||
got.append(value(ds[i]))
|
||||
except OSError:
|
||||
got.append(None)
|
||||
return got
|
||||
|
||||
|
||||
result = {}
|
||||
for tag, sizes in (("8", None), ("4", (4, 4))):
|
||||
g, x = os.path.join(out, f"vl{tag}.h5"), os.path.join(out, f"bad{tag}.h5")
|
||||
good(g, sizes)
|
||||
gcol = bad(x, sizes)
|
||||
with h5py.File(g, "r") as f:
|
||||
result[f"vl{tag}"] = {n: read(f[n]) for n in ("d", "u", "seq", "sequ", "cmp")}
|
||||
result[f"vl{tag}"]["va"] = [value(s) for s in f.attrs["va"]]
|
||||
with h5py.File(x, "r") as f:
|
||||
result[f"bad{tag}"] = {n: read(f[n]) for n in ("bad", "badseq")}
|
||||
result[f"bad{tag}"]["gcol"] = gcol
|
||||
json.dump(result, sys.stdout)
|
||||
@@ -773,3 +773,125 @@ fn every_subcommand_rejects_a_non_hdf5_file_cleanly() {
|
||||
assert_eq!(code(&h5rs(&["ls"])), 2);
|
||||
assert_eq!(code(&h5rs(&["--help"])), 0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// variable-length data
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Runs `tests/gen_vl_files.py`: VL strings (with an embedded NUL, empty
|
||||
/// and null elements), VL sequences, a VL compound member and a VL
|
||||
/// attribute, with 8- and 4-byte offsets, plus files whose heap objects
|
||||
/// disagree with their elements' lengths.
|
||||
fn generate_vl() -> Option<Files> {
|
||||
if missing(python_available(), "python3 with h5py") {
|
||||
return None;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let script = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/gen_vl_files.py");
|
||||
let out = Command::new(python())
|
||||
.arg(&script)
|
||||
.arg(dir.path())
|
||||
.output()
|
||||
.expect("run gen_vl_files.py");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"gen_vl_files.py failed:\n{}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let values = serde_json::from_slice(&out.stdout).expect("gen_vl_files.py output");
|
||||
Some(Files { dir, values })
|
||||
}
|
||||
|
||||
/// `dump` resolves VL elements through the library's `VlResolver`, as
|
||||
/// libhdf5 does: "a\0b" prints as "a", a null string as NULL (it printed
|
||||
/// ""), and with 4-byte offsets too; the output is h5dump's byte for byte.
|
||||
#[test]
|
||||
fn dump_prints_vl_data_like_h5dump() {
|
||||
let Some(f) = generate_vl() else { return };
|
||||
for name in ["vl8.h5", "vl4.h5"] {
|
||||
let p = f.p(name);
|
||||
let ours = stdout(&h5rs(&["dump", &p]));
|
||||
assert!(
|
||||
ours.contains(r#"(0): "a", "", NULL, "zz", "hello""#),
|
||||
"{name}:\n{ours}"
|
||||
);
|
||||
assert!(ours.contains(r#"(0): NULL, "w", NULL, NULL"#), "{name}");
|
||||
assert!(ours.contains("(0): (1, 2, 3), (), (-5)"), "{name}");
|
||||
if missing(tool_available("h5dump"), "h5dump") {
|
||||
continue;
|
||||
}
|
||||
let reference = run("h5dump", &[&p]);
|
||||
assert!(reference.status.success(), "{name}: {reference:?}");
|
||||
assert_eq!(ours, stdout(&reference).replacen(&p, name, 1), "{name}");
|
||||
}
|
||||
}
|
||||
|
||||
/// `dump --json` gives the values h5py reads, element by element; and an
|
||||
/// element whose heap object is not its length × base size is an error, as
|
||||
/// in h5py, not a truncated value (it printed "cde" and (1, 2)).
|
||||
#[test]
|
||||
fn dump_json_vl_values_match_h5py() {
|
||||
let Some(f) = generate_vl() else { return };
|
||||
for tag in ["8", "4"] {
|
||||
let (good, bad) = (format!("vl{tag}"), format!("bad{tag}"));
|
||||
let o = h5rs(&["dump", "--json", &f.p(&format!("{good}.h5"))]);
|
||||
assert!(o.status.success(), "{good}: {o:?}");
|
||||
let doc: serde_json::Value = serde_json::from_slice(&o.stdout).unwrap();
|
||||
let want = &f.values[&good];
|
||||
for d in doc["datasets"].as_object().unwrap().values() {
|
||||
let path = d["alias"][0].as_str().unwrap();
|
||||
assert_eq!(d["value"], want[&path[1..]], "{good}: {path}");
|
||||
}
|
||||
let attrs = &doc["groups"][doc["root"].as_str().unwrap()]["attributes"];
|
||||
assert_eq!(attrs[0]["name"], "va");
|
||||
assert_eq!(attrs[0]["value"], want["va"], "{good}: va");
|
||||
|
||||
let o = h5rs(&["dump", "--json", &f.p(&format!("{bad}.h5"))]);
|
||||
let doc: serde_json::Value = serde_json::from_slice(&o.stdout).unwrap();
|
||||
let want = &f.values[&bad];
|
||||
for d in doc["datasets"].as_object().unwrap().values() {
|
||||
let path = d["alias"][0].as_str().unwrap();
|
||||
let got = d["value"].as_array().unwrap();
|
||||
let want = want[&path[1..]].as_array().unwrap();
|
||||
assert_eq!(got.len(), want.len(), "{bad}: {path}");
|
||||
for (g, w) in got.iter().zip(want) {
|
||||
if w.is_null() {
|
||||
// h5py cannot read it: neither can we.
|
||||
let e = g["error"]
|
||||
.as_str()
|
||||
.unwrap_or_else(|| panic!("{bad}: {path}: {g}"));
|
||||
assert!(e.contains("holds"), "{bad}: {path}: {e}");
|
||||
} else {
|
||||
assert_eq!(g, w, "{bad}: {path}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `check --data` holds VL elements to libhdf5's rule: a heap object whose
|
||||
/// size is not exactly the element's length × base size is a problem (it
|
||||
/// only caught objects shorter than the element).
|
||||
#[test]
|
||||
fn check_data_flags_mis_sized_vl_heap_objects() {
|
||||
let Some(f) = generate_vl() else { return };
|
||||
for tag in ["8", "4"] {
|
||||
let o = h5rs(&["check", "--data", &f.p(&format!("vl{tag}.h5"))]);
|
||||
let s = stdout(&o);
|
||||
assert_eq!(code(&o), 0, "vl{tag}: {s}");
|
||||
assert!(
|
||||
s.contains("global heap collections read: 1"),
|
||||
"vl{tag}: {s}"
|
||||
);
|
||||
|
||||
let o = h5rs(&["check", "--data", &f.p(&format!("bad{tag}.h5"))]);
|
||||
let s = stdout(&o);
|
||||
assert_eq!(code(&o), 1, "bad{tag}: {s}");
|
||||
let at = f.values[format!("bad{tag}")]["gcol"].as_u64().unwrap();
|
||||
for (path, what) in [("/bad", "6 bytes"), ("/badseq", "12 bytes")] {
|
||||
let want = format!("problem: {at:#x} {path}: variable-length data: global heap object");
|
||||
assert!(s.contains(&want), "bad{tag}: no {want:?} in\n{s}");
|
||||
assert!(s.contains(what), "bad{tag}: {s}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
use clawhdf5::{AttrValue, File, Selection};
|
||||
use clawhdf5_format::data_read;
|
||||
use clawhdf5_format::datatype::{Datatype, DatatypeByteOrder};
|
||||
use clawhdf5_format::vl_data::{VlResolver, check_element_size};
|
||||
|
||||
/// Errors are reported to JavaScript as messages.
|
||||
pub type Result<T> = std::result::Result<T, String>;
|
||||
@@ -221,6 +222,12 @@ impl Reader {
|
||||
None => (Selection::All, shape.clone()),
|
||||
Some(h) => hyperslab_selection(h, &shape)?,
|
||||
};
|
||||
// A VL type whose stored element size is not the one the file's
|
||||
// offset size implies is refused before its data is read, as
|
||||
// `File::read_string` refuses it.
|
||||
if let Datatype::VariableLength { size, .. } = array_base(&dt) {
|
||||
check_element_size(*size, self.file.superblock().offset_size).map_err(err)?;
|
||||
}
|
||||
let raw = ds.read_selection(&selection).map_err(err)?;
|
||||
let data = self.decode(&raw, &dt)?;
|
||||
out_shape.extend(element_shape(&dt));
|
||||
@@ -270,23 +277,14 @@ impl Reader {
|
||||
Datatype::VariableLength {
|
||||
is_string: true, ..
|
||||
} if !is_array => {
|
||||
let size = dt.type_size() as usize;
|
||||
if size == 0 || !raw.len().is_multiple_of(size) {
|
||||
return Err(format!(
|
||||
"{} bytes is not a whole number of {size}-byte string references",
|
||||
raw.len()
|
||||
));
|
||||
}
|
||||
// The library's resolver, as File::read_string uses: a
|
||||
// string ends at its first NUL and a heap object of the
|
||||
// wrong size is an error, as in libhdf5 and h5py.
|
||||
let sb = self.file.superblock();
|
||||
Data::Strings(
|
||||
clawhdf5_format::vl_data::read_vl_strings(
|
||||
self.file.as_bytes(),
|
||||
raw,
|
||||
(raw.len() / size) as u64,
|
||||
sb.offset_size,
|
||||
sb.length_size,
|
||||
)
|
||||
.map_err(err)?,
|
||||
VlResolver::new(self.file.as_bytes(), sb.offset_size, sb.length_size)
|
||||
.strings(raw)
|
||||
.map_err(err)?,
|
||||
)
|
||||
}
|
||||
Datatype::Enumeration { .. } if !is_array => {
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
//! The wasm reader resolves VL strings with the library's `VlResolver`, so
|
||||
//! it returns what `File::read_string` and h5py return: a string ends at
|
||||
//! its first NUL, a null element is empty, a heap object of the wrong size
|
||||
//! is an error, and a VL datatype whose stored element size disagrees with
|
||||
//! the file's offset size is refused. Checked with 8- and 4-byte offsets.
|
||||
//!
|
||||
//! Skipped when python3 with h5py is missing, unless
|
||||
//! `CLAWHDF5_REQUIRE_INTEROP=1`. `CLAWHDF5_PYTHON` names the interpreter.
|
||||
|
||||
use std::process::Command;
|
||||
|
||||
use clawhdf5::File;
|
||||
use clawhdf5_wasm::core::{Data, Reader};
|
||||
|
||||
fn python() -> String {
|
||||
std::env::var("CLAWHDF5_PYTHON").unwrap_or_else(|_| "python3".to_string())
|
||||
}
|
||||
|
||||
fn h5py_available() -> bool {
|
||||
let ok = Command::new(python())
|
||||
.args(["-c", "import h5py, numpy"])
|
||||
.output()
|
||||
.map(|o| o.status.success())
|
||||
.unwrap_or(false);
|
||||
if !ok {
|
||||
assert!(
|
||||
std::env::var("CLAWHDF5_REQUIRE_INTEROP").as_deref() != Ok("1"),
|
||||
"CLAWHDF5_REQUIRE_INTEROP=1 but python with h5py is not available"
|
||||
);
|
||||
eprintln!("SKIP: python with h5py not available");
|
||||
}
|
||||
ok
|
||||
}
|
||||
|
||||
/// For each offset size: `vl{8,4}.h5` with dataset `d` = "a\0b", "", null,
|
||||
/// "zz" (patched: h5py writes neither a NUL nor a null element);
|
||||
/// `bad{8,4}.h5` whose element 0 claims 3 bytes of a 6-byte heap object;
|
||||
/// and `size{8,4}.h5` whose VL datatype message stores a 24-byte element.
|
||||
/// Prints h5py's reading of each element as hex, or "error".
|
||||
const SCRIPT: &str = r#"
|
||||
import struct, sys, h5py, numpy as np
|
||||
out = sys.argv[1]
|
||||
S = h5py.string_dtype('utf-8')
|
||||
def create(path, os_):
|
||||
if os_ == 8:
|
||||
return h5py.File(path, 'w', libver='earliest')
|
||||
# The earliest format, so the patched object header has no checksum.
|
||||
fcpl = h5py.h5p.create(h5py.h5p.FILE_CREATE); fcpl.set_sizes(4, 4)
|
||||
fapl = h5py.h5p.create(h5py.h5p.FILE_ACCESS)
|
||||
fapl.set_libver_bounds(h5py.h5f.LIBVER_EARLIEST, h5py.h5f.LIBVER_V18)
|
||||
return h5py.File(h5py.h5f.create(path.encode(), h5py.h5f.ACC_TRUNC, fcpl=fcpl, fapl=fapl))
|
||||
def elem(length, addr, index, os_):
|
||||
return struct.pack('<I', length) + addr.to_bytes(os_, 'little') + struct.pack('<I', index)
|
||||
def make(path, os_, values):
|
||||
with create(path, os_) as f:
|
||||
f.create_dataset('d', data=np.array(values, dtype=object), dtype=S)
|
||||
return f['d'].id.get_offset()
|
||||
for os_ in (8, 4):
|
||||
es = 8 + os_
|
||||
p = '%s/vl%d.h5' % (out, os_)
|
||||
off = make(p, os_, ['aXb', '', 'ok', 'zz'])
|
||||
b = bytearray(open(p, 'rb').read())
|
||||
b[b.index(b'aXb') + 1] = 0
|
||||
b[off + 2 * es:off + 3 * es] = elem(2, 0, 1, os_)
|
||||
open(p, 'wb').write(bytes(b))
|
||||
p = '%s/bad%d.h5' % (out, os_)
|
||||
off = make(p, os_, ['cdefgh', 'ok'])
|
||||
b = bytearray(open(p, 'rb').read())
|
||||
struct.pack_into('<I', b, off, 3)
|
||||
open(p, 'wb').write(bytes(b))
|
||||
p = '%s/size%d.h5' % (out, os_)
|
||||
make(p, os_, ['x', 'yy'])
|
||||
b = bytearray(open(p, 'rb').read())
|
||||
# datatype message: version 1, class 9 (VL); string, null-terminated, UTF-8
|
||||
pat = bytes([0x19, 0x01, 0x01, 0x00]) + struct.pack('<I', es)
|
||||
assert b.count(pat) == 1, b.count(pat)
|
||||
i = b.index(pat)
|
||||
struct.pack_into('<I', b, i + 4, 24)
|
||||
open(p, 'wb').write(bytes(b))
|
||||
for name in ('vl', 'bad', 'size'):
|
||||
with h5py.File('%s/%s%d.h5' % (out, name, os_), 'r') as f:
|
||||
got = []
|
||||
for i in range(f['d'].shape[0]):
|
||||
try:
|
||||
got.append(f['d'][i].hex())
|
||||
except OSError:
|
||||
got.append('error')
|
||||
print('%s%d\t%s' % (name, os_, ','.join(got)))
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn vl_strings_read_like_file_and_h5py() {
|
||||
if !h5py_available() {
|
||||
return;
|
||||
}
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let out = Command::new(python())
|
||||
.args(["-c", SCRIPT])
|
||||
.arg(dir.path())
|
||||
.output()
|
||||
.expect("run python");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let h5py: std::collections::HashMap<String, String> = String::from_utf8(out.stdout)
|
||||
.unwrap()
|
||||
.lines()
|
||||
.map(|l| {
|
||||
let (k, v) = l.split_once('\t').unwrap();
|
||||
(k.to_string(), v.to_string())
|
||||
})
|
||||
.collect();
|
||||
let read = |name: &str| {
|
||||
let bytes = std::fs::read(dir.path().join(format!("{name}.h5"))).unwrap();
|
||||
let wasm = Reader::open(bytes).unwrap().read("/d", None);
|
||||
let file = File::open(dir.path().join(format!("{name}.h5")))
|
||||
.unwrap()
|
||||
.dataset("d")
|
||||
.unwrap()
|
||||
.read_string();
|
||||
(wasm, file)
|
||||
};
|
||||
for os in [8, 4] {
|
||||
// h5py: "a\0b" is "a"; the null element (address 0) is empty.
|
||||
assert_eq!(h5py[&format!("vl{os}")], "61,,,7a7a");
|
||||
let (wasm, file) = read(&format!("vl{os}"));
|
||||
let Data::Strings(wasm) = wasm.unwrap().data else {
|
||||
panic!("vl{os}: not strings")
|
||||
};
|
||||
assert_eq!(wasm, ["a", "", "", "zz"], "vl{os}");
|
||||
assert_eq!(wasm, file.unwrap(), "vl{os}");
|
||||
|
||||
// h5py refuses the mis-sized element; so do both readers.
|
||||
assert_eq!(h5py[&format!("bad{os}")], "error,6f6b");
|
||||
let (wasm, file) = read(&format!("bad{os}"));
|
||||
assert!(wasm.unwrap_err().contains("holds 6 bytes"), "bad{os}");
|
||||
assert!(file.is_err(), "bad{os}");
|
||||
|
||||
// libhdf5 ignores the stored element size and reads the values;
|
||||
// File refuses the datatype rather than guess its layout, and the
|
||||
// wasm reader now does the same (it read with the stored size).
|
||||
assert_eq!(h5py[&format!("size{os}")], "78,7979");
|
||||
let (wasm, file) = read(&format!("size{os}"));
|
||||
let e = wasm.unwrap_err();
|
||||
assert!(e.contains("stores 24-byte elements"), "size{os}: {e}");
|
||||
assert!(file.is_err(), "size{os}");
|
||||
}
|
||||
}
|
||||
@@ -207,9 +207,10 @@ fill-value item that did is fixed).
|
||||
- (`cve-2024-32616` `/group1/dset3` and `cve-2025-2309`'s `Comp_OBJREF`
|
||||
attribute are h5py/numpy type-mapping failures, not libhdf5 refusals.)
|
||||
- `h5rs check` validates with the library's parsers, so it inherits what
|
||||
they accept: of the 150 CVE and fuzzer files, `check --data` passes 16,
|
||||
and h5dump 1.14.6 rejects 9 of those (tank, 2026-09-26; 28 and 21
|
||||
before these checks).
|
||||
they accept: of the 150 CVE and fuzzer files, `check --data` passes 15,
|
||||
and h5dump 1.14.6 rejects 8 of those (tank, 2026-09-26; 28 and 21
|
||||
before these checks, 16 and 9 before a VL type's stored element size
|
||||
was checked, which flags `cve-2024-32608`).
|
||||
- **Writer:**
|
||||
- Nested groups beyond one level: path-like names are now refused, not
|
||||
created.
|
||||
@@ -537,8 +538,9 @@ which is what libhdf5 itself writes.
|
||||
followed (no file system).
|
||||
- Variable-length string datasets are read by decoding `read_selection`'s
|
||||
bytes with `clawhdf5_format::vl_data` in the wasm crate; `File` itself still
|
||||
cannot (see the audit gaps above). (`File` can since 2026-09-26; the wasm
|
||||
crate still decodes them itself.)
|
||||
cannot (see the audit gaps above). (`File` can since 2026-09-26. Since
|
||||
2026-09-26 the wasm crate resolves them with the same `VlResolver` as
|
||||
`File` and `h5rs`, so all three return h5py's values.)
|
||||
|
||||
## The Node.js package (`packages/clawhdf5-node`) does not work
|
||||
|
||||
|
||||
Reference in New Issue
Block a user