clawhdf5-remote, h5rs: URLs' credentials are never shown
Every RemoteError message and HttpStorage's Debug output held the URL as given, with any user:password@ and the query string — for a presigned S3/GCS/Azure URL, its signature or token. An application logging the error leaked the credential. - New clawhdf5_remote::redact_url: no userinfo, no fragment, query values replaced by REDACTED (plain key names kept). - HttpStorage formats every message with the redacted URL, and scrubs the URL's secret parts from errors of the HTTP client (whose texts can echo the URI); Debug shows the redacted URL. storage_for_url's and the object store URL errors are redacted too. HttpStorage::url() still returns the URL as given, documented as not for logging. - h5rs prints FILE arguments that are URLs redacted: in errors and in dump/stat/check/diff output. - The test server can force a status and send a wrong Content-Range. Tests: 404, 403 (at open and on a read), wrong Content-Range (at open and on a read), no range support, encoded body, ETag change, timeout, connection closed and bad scheme errors, Display and Debug, contain none of the secrets; h5rs likewise for every subcommand. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -45,7 +45,7 @@ use clawhdf5::File;
|
||||
use clawhdf5_format::storage::Storage;
|
||||
|
||||
pub use cache::{BlockCache, CacheConfig, CacheStats};
|
||||
pub use error::{Error, RemoteError};
|
||||
pub use error::{Error, RemoteError, redact_url};
|
||||
#[cfg(feature = "http")]
|
||||
pub use http::{HttpOptions, HttpStats, HttpStorage};
|
||||
#[cfg(feature = "object-store")]
|
||||
@@ -93,13 +93,13 @@ pub fn storage_for_url(url: &str, options: &Options) -> Result<Arc<RemoteStorage
|
||||
let scheme = url
|
||||
.split_once("://")
|
||||
.map(|(s, _)| s.to_ascii_lowercase())
|
||||
.ok_or_else(|| RemoteError::InvalidUrl(format!("{url}: no scheme")))?;
|
||||
.ok_or_else(|| RemoteError::InvalidUrl(format!("{}: no scheme", redact_url(url))))?;
|
||||
match scheme.as_str() {
|
||||
"http" | "https" => http_storage(url, options),
|
||||
"s3" | "s3a" | "gs" | "az" | "azure" | "abfs" | "abfss" | "adl" => {
|
||||
cloud_storage(url, &scheme, options)
|
||||
}
|
||||
_ => Err(RemoteError::UnsupportedScheme(url.to_string()).into()),
|
||||
_ => Err(RemoteError::UnsupportedScheme(redact_url(url)).into()),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -116,7 +116,8 @@ fn http_storage(url: &str, options: &Options) -> Result<Arc<RemoteStorage>, Erro
|
||||
#[cfg(not(feature = "http"))]
|
||||
fn http_storage(url: &str, _options: &Options) -> Result<Arc<RemoteStorage>, Error> {
|
||||
Err(RemoteError::UnsupportedScheme(format!(
|
||||
"{url}: http(s):// needs the `http` feature of clawhdf5-remote"
|
||||
"{}: http(s):// needs the `http` feature of clawhdf5-remote",
|
||||
redact_url(url)
|
||||
))
|
||||
.into())
|
||||
}
|
||||
@@ -136,7 +137,8 @@ fn cloud_storage(url: &str, scheme: &str, _options: &Options) -> Result<Arc<Remo
|
||||
_ => "azure",
|
||||
};
|
||||
Err(RemoteError::UnsupportedScheme(format!(
|
||||
"{url}: {scheme}:// needs the `{feature}` feature of clawhdf5-remote"
|
||||
"{}: {scheme}:// needs the `{feature}` feature of clawhdf5-remote",
|
||||
redact_url(url)
|
||||
))
|
||||
.into())
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user