Skip to content

fix: disable boto3 checksum calculation for Tencent COS compatibility (#15927)#15930

Open
zaviermeekz-cpu wants to merge 1 commit into
infiniflow:mainfrom
zaviermeekz-cpu:fix/cos-upload-checksum
Open

fix: disable boto3 checksum calculation for Tencent COS compatibility (#15927)#15930
zaviermeekz-cpu wants to merge 1 commit into
infiniflow:mainfrom
zaviermeekz-cpu:fix/cos-upload-checksum

Conversation

@zaviermeekz-cpu

@zaviermeekz-cpu zaviermeekz-cpu commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

After upgrading to v0.25.6, uploading PPT/PPTX files (and any file >8MB) fails with MissingContentLength when using Tencent Cloud COS (S3‑compatible). The cause is the new default checksum behavior in boto3 >=1.36.0, which is not fully supported by COS.

This PR explicitly sets:

  • request_checksum_calculation = 'WHEN_REQUIRED'
  • response_checksum_validation = 'WHEN_REQUIRED'

These options restore compatibility with Tencent COS without affecting other S3 endpoints.

Type of change

  • Bug Fix

How has this been tested?

Manually tested uploading a PPT file >8MB to Tencent COS – the upload succeeds. No regression observed with AWS S3.

Related Issue

Closes #15927

@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. 🐞 bug Something isn't working, pull request that fix bug. labels Jun 11, 2026
@coderabbitai

coderabbitai Bot commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 72173bc7-ffc7-44f0-b4d6-bcffb9b91fa3

📥 Commits

Reviewing files that changed from the base of the PR and between 32559d2 and 817e51f.

📒 Files selected for processing (1)
  • rag/utils/s3_conn.py
📝 Walkthrough

Walkthrough

Updated RAGFlowS3.__open__ to merge boto3 Config settings that disable checksum features (request_checksum_calculation and response_checksum_validation), fixing MissingContentLength errors during large-file uploads to Tencent Cloud COS after boto3 upgrade.

Changes

Tencent COS Upload Compatibility

Layer / File(s) Summary
S3 client checksum configuration
rag/utils/s3_conn.py		 S3 client configuration merges a botocore.config.Config that adjusts checksum behavior to avoid chunked encoding incompatibility with Tencent COS, resolving failures on files larger than 8 MB.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Poem

🐰 A checksum dance with COS so grand,
Where Tencent's storage claims the land—
Boto3's leap caused uploads to fail,
But merging configs now tips the scale.
Large files soar, no length to mourn! 📤✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically identifies the bug fix addressing boto3 checksum calculation issues for Tencent COS compatibility, directly matching the main change.
Description check ✅ Passed The description follows the template, clearly explains the problem, identifies it as a bug fix, includes testing details, and references the related issue.
Linked Issues check ✅ Passed The PR successfully addresses issue #15927 by configuring boto3 checksum settings to restore Tencent COS compatibility for large file uploads without affecting other S3 endpoints.
Out of Scope Changes check ✅ Passed All changes in rag/utils/s3_conn.py are directly focused on fixing the checksum configuration issue for Tencent COS compatibility, with no out-of-scope modifications.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@yingfeng yingfeng added the ci Continue Integration label Jun 11, 2026
@yingfeng yingfeng marked this pull request as draft June 11, 2026 03:08
@yingfeng yingfeng marked this pull request as ready for review June 11, 2026 03:08
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.16%. Comparing base (32559d2) to head (817e51f).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main   #15930   +/-   ##
=======================================
  Coverage   93.16%   93.16%           
=======================================
  Files          10       10           
  Lines         717      717           
  Branches      118      118           
=======================================
  Hits          668      668           
  Misses         29       29           
  Partials       20       20

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

🐞 bug Something isn't working, pull request that fix bug. ci Continue Integration size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Files larger than 8MB fail to upload with MissingContentLength error when using Tencent Cloud COS (S3-compatible) after upgrade to v0.25.6

2 participants

@zaviermeekz-cpu @yingfeng