wasm: restartable NeedBytes storage for lazy reads (range-read M4, core)

LazyStorage holds the blocks of a remote file fetched so far. An
operation runs as passes over it: a read that misses records the
missing blocks and fails, the pass's result is dropped whatever it is
(a parser may have caught the error and carried on), and the caller
fetches the reported ranges and re-runs the pass. No block is evicted
while an operation is in flight, so every pass that does not finish
asks for at least one new block and the operation ends. Blocks sit in
an LRU with a byte budget, trimmed between operations, bulk (raw data)
blocks first.

Reader::open_storage opens a file through any Storage, and variable-
length strings resolve through the file's storage instead of
File::as_bytes, which panics for a file not in memory.

tests/lazy.rs compares, file by file, what the viewer can show (kinds,
listings, attributes, info, whole reads and hyperslabs) read lazily
with the facade's range-storage path and the in-memory reader: files
built here at 512 B to 1 MiB blocks, the h5py/netCDF4 fixture, and with
CLAWHDF5_WASM_CORPUS the conformance corpus (656 files agree). A
listing plus a small read of a 48 MB file fetches 3 ranges.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

crates/clawhdf5-wasm/src/core.rs

@@ -6,9 +6,12 @@
 //! with no typed-array mapping (compound, reference, opaque, ...) is refused
 //! with a message naming it, never returned as reinterpreted bytes.
 
+use std::sync::Arc;
+
 use clawhdf5::{AttrValue, File, Selection};
 use clawhdf5_format::data_read;
 use clawhdf5_format::datatype::{Datatype, DatatypeByteOrder};
+use clawhdf5_format::storage::Storage;
 use clawhdf5_format::vl_data::{VlResolver, check_element_size};
 
 /// Errors are reported to JavaScript as messages.
@@ -119,7 +122,9 @@ pub struct Attr {
     pub value: AttrValue,
 }
 
-/// An open file, held in memory.
+/// An open file: held in memory ([`Reader::open`]) or read through a
+/// [`Storage`] ([`Reader::open_storage`], such as a
+/// [`LazyStorage`](crate::lazy::LazyStorage)).
 pub struct Reader {
     file: File,
 }
@@ -132,6 +137,14 @@ impl Reader {
         })
     }
 
+    /// Open a file read through `storage` (the file's bytes from offset 0,
+    /// user block included, as [`File::open_storage`] takes them).
+    pub fn open_storage(storage: Arc<dyn Storage + Send + Sync>) -> Result<Self> {
+        Ok(Self {
+            file: File::open_storage(storage).map_err(err)?,
+        })
+    }
+
     /// Whether `path` names a group or a dataset.
     pub fn kind(&self, path: &str) -> Result<Kind> {
         match self.file.dataset(path) {
@@ -281,11 +294,14 @@ impl Reader {
                 // string ends at its first NUL and a heap object of the
                 // wrong size is an error, as in libhdf5 and h5py.
                 let sb = self.file.superblock();
-                Data::Strings(
-                    VlResolver::new(self.file.as_bytes(), sb.offset_size, sb.length_size)
-                        .strings(raw)
-                        .map_err(err)?,
-                )
+                let strings = match self.file.contiguous_bytes() {
+                    Some(bytes) => {
+                        VlResolver::new(bytes, sb.offset_size, sb.length_size).strings(raw)
+                    }
+                    None => VlResolver::new_in(self.file.storage(), sb.offset_size, sb.length_size)
+                        .strings(raw),
+                };
+                Data::Strings(strings.map_err(err)?)
             }
             Datatype::Enumeration { .. } if !is_array => {
                 Data::Strings(data_read::read_enum_names(raw, dt).map_err(err)?)