Fix/getobjectlength (#920)

* fix getobject content length resp

* Fix regression in exception handling for non-existent key with enhanced compression predicate and metadata improvements (#915)

* Initial plan

* Fix GetObject regression by excluding error responses from compression

The issue was that CompressionLayer was attempting to compress error responses,
which could cause Content-Length header mismatches. By excluding 4xx and 5xx
responses from compression, we ensure error responses (like NoSuchKey) are sent
correctly without body truncation.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Enhance NoSuchKey fix with improved compression predicate and comprehensive tests

- Enhanced ShouldCompress predicate with size-based exclusion (< 256 bytes)
- Added detailed documentation explaining the compression logic
- Added debug logging for better observability
- Created comprehensive test suite with 4 test cases:
  - test_get_deleted_object_returns_nosuchkey
  - test_head_deleted_object_returns_nosuchkey
  - test_get_nonexistent_object_returns_nosuchkey
  - test_multiple_gets_deleted_object
- Added extensive inline documentation and comments
- Created docs/fix-nosuchkey-regression.md with full analysis

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add compression best practices documentation

Added comprehensive guide covering:
- Best practices for HTTP response compression
- Common pitfalls and solutions
- Performance considerations and trade-offs
- Testing guidelines and examples
- Monitoring and alerting recommendations
- Migration guide for existing services

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fmt

* fmt

* Fix/objectdelete (#917)

* fix getobject content length resp

* fix delete object

---------

Co-authored-by: houseme <housemecn@gmail.com>

* Add comprehensive analysis of NoSuchKey fix and related improvements

Created detailed documentation analyzing:
- HTTP compression layer fix (primary issue)
- Content-length calculation fix from PR #917
- Delete object metadata fixes from PR #917
- How all components work together
- Complete scenario walkthrough
- Performance impact analysis
- Testing strategy and deployment checklist

This ties together all the changes in the PR branch including the merged
improvements from PR #917.

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* replace `once_cell` to `std`

* fmt

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: weisd <im@weisd.in>

* fmt

---------

Co-authored-by: weisd <weishidavip@163.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: weisd <im@weisd.in>
This commit is contained in:
houseme
2025-11-24 18:56:34 +08:00
committed by GitHub
parent fce4e64da4
commit 069194f553
14 changed files with 1229 additions and 66 deletions
@@ -12,14 +12,14 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use once_cell::sync::Lazy;
use std::sync::Arc;
use std::sync::LazyLock;
use std::sync::atomic::{AtomicU32, AtomicUsize, Ordering};
use tokio::sync::Notify;
/// Optimized notification pool to reduce memory overhead and thundering herd effects
/// Increased pool size for better performance under high concurrency
static NOTIFY_POOL: Lazy<Vec<Arc<Notify>>> = Lazy::new(|| (0..128).map(|_| Arc::new(Notify::new())).collect());
static NOTIFY_POOL: LazyLock<Vec<Arc<Notify>>> = LazyLock::new(|| (0..128).map(|_| Arc::new(Notify::new())).collect());
/// Optimized notification system for object locks
#[derive(Debug)]
+7 -7
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use once_cell::unsync::OnceCell;
use serde::{Deserialize, Serialize};
use smartstring::SmartString;
use std::hash::{Hash, Hasher};
use std::sync::Arc;
use std::sync::OnceLock;
use std::time::{Duration, SystemTime};
use crate::fast_lock::guard::FastLockGuard;
@@ -72,10 +72,10 @@ pub struct OptimizedObjectKey {
/// Version - optional for latest version semantics
pub version: Option<SmartString<smartstring::LazyCompact>>,
/// Cached hash to avoid recomputation
hash_cache: OnceCell<u64>,
hash_cache: OnceLock<u64>,
}
// Manual implementations to handle OnceCell properly
// Manual implementations to handle OnceLock properly
impl PartialEq for OptimizedObjectKey {
fn eq(&self, other: &Self) -> bool {
self.bucket == other.bucket && self.object == other.object && self.version == other.version
@@ -116,7 +116,7 @@ impl OptimizedObjectKey {
bucket: bucket.into(),
object: object.into(),
version: None,
hash_cache: OnceCell::new(),
hash_cache: OnceLock::new(),
}
}
@@ -129,7 +129,7 @@ impl OptimizedObjectKey {
bucket: bucket.into(),
object: object.into(),
version: Some(version.into()),
hash_cache: OnceCell::new(),
hash_cache: OnceLock::new(),
}
}
@@ -145,7 +145,7 @@ impl OptimizedObjectKey {
/// Reset hash cache if key is modified
pub fn invalidate_cache(&mut self) {
self.hash_cache = OnceCell::new();
self.hash_cache = OnceLock::new();
}
/// Convert from regular ObjectKey
@@ -154,7 +154,7 @@ impl OptimizedObjectKey {
bucket: SmartString::from(key.bucket.as_ref()),
object: SmartString::from(key.object.as_ref()),
version: key.version.as_ref().map(|v| SmartString::from(v.as_ref())),
hash_cache: OnceCell::new(),
hash_cache: OnceLock::new(),
}
}