Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Prevent automatic charset=utf-8 addition to Content-Type headers #631

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

AffanShaikhsurab
Copy link
Contributor

@AffanShaikhsurab AffanShaikhsurab commented Mar 2, 2025

PR Description

This change prevents the HTTP package from automatically appending 'charset=utf-8' to Content-Type headers by:

  1. Using low-level http.Request instead of high-level client methods
  2. Setting request.bodyBytes directly with UTF-8 encoded content
  3. Preserving exact Content-Type headers as specified by users

This ensures users have full control over Content-Type headers sent to servers, addressing issues where some APIs reject requests with charset parameters.

Related Issues

Checklist

  • I have gone through the contributing guide
  • I have updated my branch and synced it with project main branch before making this PR
  • I am using the latest Flutter stable branch (run flutter upgrade and verify)
  • I have run the tests (flutter test) and all tests are passing

Added/updated tests?

We encourage you to add relevant test cases.

  • Yes
  • No, and this is why: This fix resolves an urgent issue with Content-Type headers. Proper testing requires mocking HTTP client responses which will be implemented in a separate pr to focus on comprehensive test coverage for HTTP service functionality. We can add proper testing in this pr also after proper discussion

OS on which you have developed and tested the feature?

  • Windows
  • macOS
  • Linux

This change prevents the HTTP package from automatically appending 'charset=utf-8' to Content-Type headers by:

Using low-level http.Request instead of high-level client methods
Setting request.bodyBytes directly with UTF-8 encoded content
Preserving exact Content-Type headers as specified by users
This ensures users have full control over Content-Type headers sent to servers, addressing issues where some APIs reject requests with charset parameters.
- Add support for multiple character encodings using charset package
- Implement mapping for all IANA standard charsets to their Dart equivalents
- Fix mismatch between Content-Type header charset and actual body encoding
- Support UTF-8, UTF-16, UTF-32, ISO-8859 variants, Windows code pages, and Asian encodings
- Provide appropriate fallbacks for less common encodings
- Add proper content type detection and encoding for request bodies
- Improve error handling for unsupported encodings
final normalizedName = charsetName.toLowerCase().trim();

// Direct codec mappings - built-in encodings first
switch (normalizedName) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the source of this codec map and names?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don’t need this much but I have just added it we can remove them also

Copy link
Member

@ashitaprasad ashitaprasad Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AffanShaikhsurab Where in the link that you provided are the strings iso646-us & iso_8859-1:1987 that was written below in the case matching?
Kindly do not beat around the bush and tell exactly how you got all the string cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashitaprasad I initially thought that the ISO encoding was present in the list of supported encodings, but upon further inspection, I realized it was not included in the updated code. I have now ensured that only the encoding types available in the dart:convert package and Flutter’s Charset are used.

I have a question: should we first encode it as UTF-8 and then pass it as text/csv in the content headers , is this the expected behaviour ?

Apologies for the earlier oversight—I was working on multiple issues simultaneously, which led to an incomplete implementation. I will ensure that I take on fewer tasks at a time and implement them more carefully moving forward.

@AffanShaikhsurab
Copy link
Contributor Author

and for the iso646-us & iso_8859 encoding i got it from the list of encodings from there :: https://api.dart.dev/stable/latest/dart-convert/Latin1Codec-class.html for iso_8859 and for iso646-us here is the reference https://api.dart.dev/stable/latest/dart-convert/AsciiCodec-class.html as the iso646-us is same as the assci i got this reference from
http://aspell.net/charsets/iso646.html#:~:text=Back%20then%2C%20the%20socialist%20countries,used%20in%20the%20core%20Internet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to override Content-Type header
2 participants