Tuesday, May 27, 2025

Fix: Large file upload tests in a Rails app start failing after bumping rack to 3.0.16

Problem: After bumping the Rack gem in a Rails app from 3.0.14 to 3.0.16, a couple of tests which tested an HTTP POST of a large-ish (~30 MB) PDF file to a controller started failing. Other similar tests, which did not upload a large file, continued passing.

The failure symptom was the POST returning an HTTP 400 Bad Request error. The error was returned before the request made it to any of our application code (as evidenced by none of our breakpoints being hit, or debug statements appearing in the output log).  

For our particular test, the specific error message was: Minitest::Assertion: Expected response to be a <413: payload_too_large>, but was a <400: Bad Request>

Fortunately, there were no other gem dependencies that came with the bump of Rack to 3.0.16. So we knew the problem must be caused by some change in Rack 3.0.15 or 3.0.16. 

Solution: The single item in the rack 3.0.16 release notes did turn out to be the pertinent change:

Security: CVE-2025-46727 Unbounded parameter parsing in Rack::QueryParser can lead to memory exhaustion.

As that linked advisory mentions, a new check is added in rack 3.0.16 that turns away "extremely large" application/x-www-form-urlencoded bodies. From this nist.gov page on CVE-2025-46727, links were available to the fix's specific code changes. From the first linked change, we can see that a "BYTESIZE_LIMIT" of 4194304 (a little over 4 MB) is now imposed on application/x-www-form-urlencoded request bodies. 

Per the mozilla.org page on HTTP POSTapplication/x-www-form-urlencoded is only one possible form Content-Type HTTP header value -- and not the one that is supposed to be used with binary data, such as PDF file uploads! Instead, multipart/form-data should be used.

Our original unit test code was actually not specifying any Content-Type HTTP header value in our test POST request. Changing the test's POST request to additionally send the HTTP header Content-Type: multipart/form-data fixed the problem, and got our test passing once again.

Aside: When I spent a minute during the investigation to ask ChatGPT about this problem -- a POST request failure introduced as of Rack 3.0.16, possibly due to a large request payload -- ChatGPT obligingly hallucinated up a plausible-sounding (but 100% fictional) response about Rack introducing a default limit in POST request payload size, and a corresponding plausible-sounding (but 100% fictional) response about a configuration line to be added to config/applicaiton.rb to override the supposed new behavior. 😂🙄

No comments:

Post a Comment

Non-spammers: Thanks for visiting! Please go ahead and leave a comment; I read them all!

Attention SPAMMERS: I review all comments before they get posted, and I REPORT 100% of spam comments to Google as spam! Why not avoid getting your account banned as quickly -- and save us both a little time -- by skipping this comment form and moving on to the next one on your list? Thanks, and I hope you have a great day!