Fix/getobjectlength (#920)

* fix getobject content length resp

* Fix regression in exception handling for non-existent key with enhanced compression predicate and metadata improvements (#915)

* Initial plan

* Fix GetObject regression by excluding error responses from compression

The issue was that CompressionLayer was attempting to compress error responses,
which could cause Content-Length header mismatches. By excluding 4xx and 5xx
responses from compression, we ensure error responses (like NoSuchKey) are sent
correctly without body truncation.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Enhance NoSuchKey fix with improved compression predicate and comprehensive tests

- Enhanced ShouldCompress predicate with size-based exclusion (< 256 bytes)
- Added detailed documentation explaining the compression logic
- Added debug logging for better observability
- Created comprehensive test suite with 4 test cases:
  - test_get_deleted_object_returns_nosuchkey
  - test_head_deleted_object_returns_nosuchkey
  - test_get_nonexistent_object_returns_nosuchkey
  - test_multiple_gets_deleted_object
- Added extensive inline documentation and comments
- Created docs/fix-nosuchkey-regression.md with full analysis

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add compression best practices documentation

Added comprehensive guide covering:
- Best practices for HTTP response compression
- Common pitfalls and solutions
- Performance considerations and trade-offs
- Testing guidelines and examples
- Monitoring and alerting recommendations
- Migration guide for existing services

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fmt

* fmt

* Fix/objectdelete (#917)

* fix getobject content length resp

* fix delete object

---------

Co-authored-by: houseme <housemecn@gmail.com>

* Add comprehensive analysis of NoSuchKey fix and related improvements

Created detailed documentation analyzing:
- HTTP compression layer fix (primary issue)
- Content-length calculation fix from PR #917
- Delete object metadata fixes from PR #917
- How all components work together
- Complete scenario walkthrough
- Performance impact analysis
- Testing strategy and deployment checklist

This ties together all the changes in the PR branch including the merged
improvements from PR #917.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* replace `once_cell` to `std`

* fmt

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: weisd <im@weisd.in>

* fmt

---------

Co-authored-by: weisd <weishidavip@163.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: weisd <im@weisd.in>

rustfs/src/server/http.rs

@@ -43,7 +43,7 @@ use tokio_rustls::TlsAcceptor;
 use tonic::{Request, Status, metadata::MetadataValue};
 use tower::ServiceBuilder;
 use tower_http::catch_panic::CatchPanicLayer;
-use tower_http::compression::CompressionLayer;
+use tower_http::compression::{CompressionLayer, predicate::Predicate};
 use tower_http::cors::{AllowOrigin, Any, CorsLayer};
 use tower_http::request_id::{MakeRequestUuid, PropagateRequestIdLayer, SetRequestIdLayer};
 use tower_http::trace::TraceLayer;
@@ -108,6 +108,60 @@ fn get_cors_allowed_origins() -> String {
         .unwrap_or(rustfs_config::DEFAULT_CONSOLE_CORS_ALLOWED_ORIGINS.to_string())
 }
 
+/// Predicate to determine if a response should be compressed.
+///
+/// This predicate implements intelligent compression selection to avoid issues
+/// with error responses and small payloads. It excludes:
+/// - Client error responses (4xx status codes) - typically small XML/JSON error messages
+/// - Server error responses (5xx status codes) - ensures error details are preserved
+/// - Very small responses (< 256 bytes) - compression overhead outweighs benefits
+///
+/// # Rationale
+/// The CompressionLayer can cause Content-Length header mismatches with error responses,
+/// particularly when the s3s library generates XML error responses (~119 bytes for NoSuchKey).
+/// By excluding these responses from compression, we ensure:
+/// 1. Error responses are sent with accurate Content-Length headers
+/// 2. Clients receive complete error bodies without truncation
+/// 3. Small responses avoid compression overhead
+///
+/// # Performance
+/// This predicate is evaluated per-response and has O(1) complexity.
+#[derive(Clone, Copy, Debug)]
+struct ShouldCompress;
+
+impl Predicate for ShouldCompress {
+    fn should_compress<B>(&self, response: &Response<B>) -> bool
+    where
+        B: http_body::Body,
+    {
+        let status = response.status();
+
+        // Never compress error responses (4xx and 5xx status codes)
+        // This prevents Content-Length mismatch issues with error responses
+        if status.is_client_error() || status.is_server_error() {
+            debug!("Skipping compression for error response: status={}", status.as_u16());
+            return false;
+        }
+
+        // Check Content-Length header to avoid compressing very small responses
+        // Responses smaller than 256 bytes typically don't benefit from compression
+        // and may actually increase in size due to compression overhead
+        if let Some(content_length) = response.headers().get(http::header::CONTENT_LENGTH) {
+            if let Ok(length_str) = content_length.to_str() {
+                if let Ok(length) = length_str.parse::<u64>() {
+                    if length < 256 {
+                        debug!("Skipping compression for small response: size={} bytes", length);
+                        return false;
+                    }
+                }
+            }
+        }
+
+        // Compress successful responses with sufficient size
+        true
+    }
+}
+
 pub async fn start_http_server(
     opt: &config::Opt,
     worker_state_manager: ServiceStateManager,
@@ -507,8 +561,8 @@ fn process_connection(
             )
             .layer(PropagateRequestIdLayer::x_request_id())
             .layer(cors_layer)
-            // Compress responses
-            .layer(CompressionLayer::new())
+            // Compress responses, but exclude error responses to avoid Content-Length mismatch issues
+            .layer(CompressionLayer::new().compress_when(ShouldCompress))
             .option_layer(if is_console { Some(RedirectLayer) } else { None })
             .service(service);