format, wasm: Storage::hint, fetched by the lazy reader with a pass's misses
A parser often learns where the next structures are (a node's children, a structure's body once its prefix gives its length) before it reads them one at a time. Over openUrl's restartable reader a structure only reached after a miss costs a pass, and a round trip, of its own. - clawhdf5-format: `Storage::hint(offset, len)`, "about to be read by this operation". Default: nothing (every backend that reads when asked); `&T`, `Box`, `Arc` and the facade's `FileData` forward it (shifted past a user block, clamped to the file). - clawhdf5-wasm `LazyStorage` records hinted blocks it lacks. A pass that misses nothing ignores them (a hint never adds a round trip); a pass that misses also asks for them, in file order, while the pass stays within what is left of the operation's `maxFetch` budget (a hint never makes a call fail). At most 1 MiB (or a block) per hint and 65536 blocks per pass are recorded, whatever a file makes a parser hint. `Operation::attempt` follows hints; the plain `LazyStorage::attempt` does not. `run_blocking` and the browser's driver use the former. `LazyStats::hinted_blocks` counts them. No parser hints yet: results, passes and requests are unchanged. Tests: hinted blocks come with a miss and never alone (cached ones skipped, a plain attempt ignores them), and stay within the fetch budget, a hint past the end or longer than the file harmless. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -89,6 +89,21 @@ pub trait Storage {
|
||||
fn as_contiguous(&self) -> Option<&[u8]> {
|
||||
None
|
||||
}
|
||||
|
||||
/// A hint that `[offset, offset + len)` is about to be read by the
|
||||
/// same operation: a parser that has just learnt where the next
|
||||
/// structures are (a node's children, a structure's body) says so
|
||||
/// before it reads them one at a time. Nothing is read and nothing
|
||||
/// fails. The default ignores it, as does every backend that reads
|
||||
/// when asked; the browser's restartable reader, which fetches over
|
||||
/// the network between attempts, fetches hinted bytes along with the
|
||||
/// bytes an attempt actually missed, so structures a parser only
|
||||
/// reaches after a miss arrive in the same round trip. Results never
|
||||
/// depend on hints.
|
||||
#[inline]
|
||||
fn hint(&self, offset: u64, len: usize) {
|
||||
let _ = (offset, len);
|
||||
}
|
||||
}
|
||||
|
||||
impl Storage for [u8] {
|
||||
@@ -148,6 +163,11 @@ impl<T: Storage + ?Sized> Storage for &T {
|
||||
fn as_contiguous(&self) -> Option<&[u8]> {
|
||||
(**self).as_contiguous()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn hint(&self, offset: u64, len: usize) {
|
||||
(**self).hint(offset, len)
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: Storage + ?Sized> Storage for Box<T> {
|
||||
@@ -170,6 +190,11 @@ impl<T: Storage + ?Sized> Storage for Box<T> {
|
||||
fn as_contiguous(&self) -> Option<&[u8]> {
|
||||
(**self).as_contiguous()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn hint(&self, offset: u64, len: usize) {
|
||||
(**self).hint(offset, len)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
@@ -193,6 +218,11 @@ impl<T: Storage + ?Sized> Storage for std::sync::Arc<T> {
|
||||
fn as_contiguous(&self) -> Option<&[u8]> {
|
||||
(**self).as_contiguous()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn hint(&self, offset: u64, len: usize) {
|
||||
(**self).hint(offset, len)
|
||||
}
|
||||
}
|
||||
|
||||
/// `storage.len()` as the `usize` the parsers' end-of-file errors report
|
||||
|
||||
Reference in New Issue
Block a user