Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inconsistent newline handling in email module #40414

Open
iko mannequin opened this issue Jun 18, 2004 · 5 comments
Open

Inconsistent newline handling in email module #40414

iko mannequin opened this issue Jun 18, 2004 · 5 comments
Labels
stdlib Python modules in the Lib dir topic-email type-bug An unexpected behavior, bug, or error

Comments

@iko
Copy link
Mannequin

iko mannequin commented Jun 18, 2004

BPO 975330
Nosy @warsaw, @devdanzin, @bitdancer
Files
  • Charset.diff: Patch to change email.Charset as described
  • crlf-body-encode.patch
  • base_encode_tests.patch: Base encode patch tests
  • crlf_base64_body_encode.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2004-06-18.12:50:50.000>
labels = ['type-bug', 'expert-email']
title = 'Inconsistent newline handling in email module'
updated_at = <Date 2012-05-28.18:40:32.448>
user = 'https://bugs.python.org/iko'

    bugs.python.org fields:

    activity = <Date 2012-05-28.18:40:32.448>
actor = 'r.david.murray'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['email']
creation = <Date 2004-06-18.12:50:50.000>
creator = 'iko'
dependencies = []
files = ['1309', '21541', '22473', '25748']
hgrepos = []
issue_num = 975330
keywords = []
message_count = 5.0
messages = ['21213', '82058', '133028', '139098', '161798']
nosy_count = 5.0
nosy_names = ['barry', 'iko', 'ajaksu2', 'r.david.murray', 'davbo']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue975330'
versions = ['Python 3.3']
    @iko
    Copy link
    Mannequin Author

    iko mannequin commented Jun 18, 2004

    text/* parts of email messages must use \r\n as the
    newline separator. For unencoded messages. smtplib and
    friends take care of the translation from \n to \r\n in
    the SMTP processing.

    Parts which are unencoded (i.e. 7bit character sets)
    MUST use \n line endings, or smtplib with translate to
    \r\r\n.

    Parts that get encoded using quoted-printable can use
    either, because the qp-encoder assumes input data is
    text and reencodes with \n.

    However, parts which get encoded using base64 are NOT
    translated, and so must use \r\n line endings.

    This means you have to guess whether your text is going
    to get encoded or not (admittedly, usually not that
    hard), and translate the line endings appropriately
    before generating a Message instance.

    I think the fix would be for Charset.encode_body() to
    alway force the encoder to text mode
    (i.e.binary=False), since it seems unlikely to have a
    Charset for something which is not text.

    @iko iko mannequin assigned warsaw Jun 18, 2004
    @iko iko mannequin added the stdlib Python modules in the Lib dir label Jun 18, 2004
    @iko iko mannequin assigned warsaw Jun 18, 2004
    @iko iko mannequin added the stdlib Python modules in the Lib dir label Jun 18, 2004
    @devdanzin
    Copy link
    Mannequin

    devdanzin mannequin commented Feb 14, 2009

    Email sprint candidate.

    @devdanzin devdanzin mannequin added easy type-bug An unexpected behavior, bug, or error labels Feb 14, 2009
    @warsaw warsaw assigned bitdancer and unassigned warsaw May 5, 2010
    @bitdancer
    Copy link
    Member

    Well, it's two years later, but I did look at this during the sprints at PyCon, though I didn't get as far as posting it then (I only just now rediscovered the patch on my laptop).

    Python3 no longer has a "binary" flag on base64mime.encode, so here is a proposed patch for Python3. I'm not sure if this should be backported or not, but I'm leaning that way. Theoretically it should be only an improvement, but I can easily imagine unix-only programs unknowingly depending on the previous non-translation of newlines. Still, since email is about intermachine communication and this clearly makes it more RFC compliant, the change is a legitimate bug fix and the chance of breakage is relatively small.

    Tests are still needed.

    @davbo
    Copy link
    Mannequin

    davbo mannequin commented Jun 25, 2011

    Added some tests against the patch provided by R. David Murray. See attached patch.

    Tests pass against default.

    @bitdancer
    Copy link
    Member

    I almost applied this patch, but my gut is having second thoughts about it. I don't think this is the correct solution. The correct solution would be to delay the encoding of the body part until the message generation phase, and use the requested linesep at that point. That is, in 3.2 and 3.3 I've changed the paradigm from "always use \n and convert the final string at need" to "specify the linesep when flattening the message".

    I'm uploading the completed patch here so I don't lose it, but I don't think I'm going to use it in this form.

    @bitdancer bitdancer added topic-email and removed easy stdlib Python modules in the Lib dir labels May 28, 2012
    @bitdancer bitdancer removed their assignment May 28, 2012
    @bitdancer bitdancer added topic-email and removed easy stdlib Python modules in the Lib dir labels May 28, 2012
    @bitdancer bitdancer removed their assignment May 28, 2012
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 9, 2022
    @iritkatriel iritkatriel added the stdlib Python modules in the Lib dir label Nov 23, 2023
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir topic-email type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @warsaw @bitdancer @iritkatriel