Skip to content
Snippets Groups Projects
  1. Sep 13, 2023
  2. Sep 12, 2023
  3. Sep 11, 2023
  4. Sep 10, 2023
  5. Sep 09, 2023
  6. Sep 08, 2023
  7. Sep 07, 2023
  8. Sep 06, 2023
  9. Sep 05, 2023
  10. Sep 04, 2023
  11. Sep 03, 2023
  12. Sep 02, 2023
  13. Sep 01, 2023
    • Xavier Morel's avatar
      [FIX] base: encoding guessing of html module descriptions · 10ad2506
      Xavier Morel authored
      
      I missed a critical issue in #133708: various users had discovered
      they could already fix description issues by adding an XML declaration
      to their document which is very cool (though technically not really
      valid).
      
      What is a lot less cool is that lxml gets *extremely* unhappy when
      asked to parse *strings* with an encoding declaration, raising a
      ValueError, so the purported fix breaks on any module which does that,
      which seems to include a lot of OCA modules.
      
      Gate the encoding guessing by bailing if the document has an XML
      declaration, in which case we just assume the author knows what
      they're doing and we leave them alone. For extra safety, check the
      encoding declaration in ascii and utf16. Could also have checked for
      BOMs, but lxml seems to not care about them overly much (in fact it
      seems to prefer them decoded which is odd).
      
      closes odoo/odoo#133918
      
      Reported-by: @rezak400
      X-original-commit: 51d37560
      Signed-off-by: default avatarXavier Morel (xmo) <xmo@odoo.com>
      10ad2506
    • Odoo's Mergebot's avatar
      [MERGE] base, (test_)mail, various: be defensive in email formatting · 9aeae36e
      Odoo's Mergebot authored
      
      PURPOSE
      
      Be defensive when dealing with email fields, notably when having multi-emails
      or email field containing an already-formatted email.
      
      RATIONALE
      
      Two main use case of corner case usage of 'email' fields are tested in this PR
      and their support is improved
      
        * formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
          field;
        * multi emails: `email1@domain.com, email2@domain.com` stored into a single
          'email' field;
      
      IMPLICATION
      
      Email field is generally managed as "containing a valid email". This means
      it is sometimes used as it in 'formataddr' as well as to perform searches or
      identification checks. Example of issue: partner 'Raoul' has a formatted email
      like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to
      
        from: "Raoul" <"Raoul" <raoul@raoul.fr>>
        -> which is incorrect (but often dynamically corrected by email servers);
      
      Email field holding multi-emails are not normalized, as current normalize
      is done only if the field holds a single email. It means
      
        * no easy finding based on 'email_normalized', e.g. various tools like
          '_mail_find_partner_from_emails' or 'find_or_create' do not find partners
          based on this email;
        * no exclusion list management;
        * issue with formatting, like
      
        to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
        -> which is incorrect (but often dynamically corrected by email servers);
      
      USAGE: OUTGOING EMAILS
      
      Those use cases currently generate faulty outgoing emails. This is valid for
      recipients ('email_cc', 'email_to') as well as author ('email_from').
      
      For formatted emails: `email_to` is formatted again based on name and email
      which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.
      
      Note that multi emails without formatting may work as it leads to email_to
      `"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
      servers correctly send multiple emails. It depends on their fault
      tolerance.
      
      USAGE: FIND BASED ON EMAIL (NORMALIZED)
      
      When searching for partners (e.g. using '_mail_find_partner_from_emails' or
      'find_or_create') normalized version of input is used.
      
      In case of multi emails sanitize is 'False', as normalization expects a single
      email in the field. Therefore no partner is found. In processes that do a
      "search or create" (e.g. using a template on a record) this leads to creating
      a new partner (or several partners in case of multi emails) each time.
      
      USAGE: OTHER FLOWS
      
      Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
      of outgoing emails and are impacted by formatted email / multi email usage.
      Those include notably
      
        * mass_mailing: '_message_get_default_recipients' should be defensive to
          give correct values when creating mailing emails;
        * mass_mailing: faulty emails is based on normalize and multi-emails are
          considered as faulty and ignored;
        * after post hook: '_message_post_after_hook' tries to link messages without
          author (but email_from) with newly-created partners, when partners are
          created from chatter. It is therefore impacted by those corner cases;
        * marketing_automation: built on top of mass_mailing and suffers from the
          same issues;
      
      USAGE: UNICODE
      
      Unicode in emails should be supported. 'formataddr' and IrMailServer notably
      received fixes to support unicode. Some check performed on email addresses
      fail when unicode is involved, which leads to some emails not being sent
      while they could.
      
      USAGE: WRONG EMAIL FORMATTING
      
      With input 'name email@domain.com' (missing chevrons allowing to clearly spot
      the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
      whole input is considered as being the email.
      
      To improve the heuristic we can add a fallback by recalling 'getadresses'
      on the input with spaces replaced by commas when it found only an email and
      no name. The new email will be split into sub pairs allowing to find the real
      email and various name parts, allowing to make a new name / email pair.
      
      Emails should not contain spaces thus this is coherent with email formation.
      This fallback actually comes from a specific code done in '_parse_partner_name'
      of Partner model. Supporting it directly at tools level make the behavior
      coherent for all models.
      
      SPECIFICATIONS
      
      This PR contains first tests to cover current support of those use cases. Then
      we gradually improve support of multi-emails and formatted emails in various
      layers of mail code, from mail.mail to mass_mailing or email formatting.
      
      See individual commit for more explanation, each of them solving a specific
      issue / use case.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      closes odoo/odoo#74474
      
      Related: odoo/enterprise#41828
      Signed-off-by: default avatarThibault Delavallee (tde) <tde@openerp.com>
      9aeae36e
    • Thibault Delavallée's avatar
      [FIX] mail: correctly parse formatted email in JS · ecb549c9
      Thibault Delavallée authored
      Formatted emails using our own formataddr contains quotes to correctly
      separate name from emails, like'"Name" <email@domain.com'. However
      JS parse function does not correctly handle quotes, assuming everything
      being left of opening chevron is the name.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      Part-of: odoo/odoo#74474
      ecb549c9
    • Thibault Delavallée's avatar
      [FIX] crm: temporarily disable test · fed39b3e
      Thibault Delavallée authored
      It seems the commented test make PLS test crash. Not sure why as tests unit
      tests should not be dependent but PLS is a complicated matter, flushing
      stuff could have an impact on DB state notably.
      
      Until we figure out what happens test that makes the PLS tests crash is
      commented. Crashing assert in PLS is improved to better know which one
      is crashing.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      Part-of: odoo/odoo#74474
      fed39b3e
    • Thibault Delavallée's avatar
      [FIX] tools, base, mail: add a fallback when parsing wrongly-formatted emails · b2f3f0f3
      Thibault Delavallée authored
      With input 'name email@domain.com' (missing chevrons allowing to clearly spot
      the email part) 'getaddresses' returns ('', 'name email@domain.com) i.e. the
      whole input is considered as being the email.
      
      To improve the heuristic we can add a fallback by recalling 'getadresses'
      on the input with spaces replaced by commas when it found only an email and
      no name. The new email will be split into sub pairs allowing to find the real
      email and various name parts, allowing to make a new name / email pair.
      
      Emails should not contain spaces thus this is coherent with email formation.
      This fallback actually comes from a specific code done in '_parse_partner_name'
      of Partner model. Supporting it directly at tools level make the behavior
      coherent for all models.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      Part-of: odoo/odoo#74474
      b2f3f0f3
    • Thibault Delavallée's avatar
      [IMP] tools, base, mail: better support non-ascii / IDNA when normalizing · a70327da
      Thibault Delavallée authored
      PURPOSE
      
      Be defensive when dealing with email fields, notably when having multi-emails
      or email field containing an already-formatted email.
      
      SPECIFICATIONS
      
      As of rfc5322 section 3.4.1 local-part is case-sensitive. However most main
      providers do consider the local-part as case insensitive. With the introduction
      of smtp-utf8 within odoo, this assumption is certain to fall short for
      international emails. We now consider that
      
        * if local part is ascii: normalize still 'lower' ;
        * else: use as it, SMTP-UF8 is made for non-ascii local parts;
      
      Concerning domain part of the address, as of v14 international domain (IDNA)
      are handled fine. The domain is always lowercase, lowering it is fine as it
      is probably an error. With the introduction of IDNA, there is an encoding
      that allow non-ascii characters to be encoded to ascii ones, using 'idna.encode'.
      
      Also remove usage of 'email_re' in mailing email check. It is too restrictive
      compared to real formatting we support (or try to). Valid outgoing emails
      were directly canceled, notably when containing unicode.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      Part-of: odoo/odoo#74474
      a70327da
    • Thibault Delavallée's avatar
      [IMP] various: support multi-emails in mailings · 516ccbc9
      Thibault Delavallée authored
      PURPOSE
      
      Be defensive when dealing with email fields, notably when having multi-emails
      or email field containing an already-formatted email.
      
      SPECIFICATIONS: MAIL COMPOSER IN MAILING
      
      When using the composer with a mailing, it currently skips recipients whose
      email is a multi-email due to the strict usage of 'email_normalize'.
      
      We can improve multi-email support by effectively checking for the first
      email found, using the "less strict" mode of normalize. It means more emails
      are detected as valid, and therefore sent.
      
      Due to lower support of multi-emails when sending emails, this even allows
      to send multiple emails as all emails are mailed.
      
      SPECIFICATIONS: DEFAULT RECIPIENTS
      
      Mailings are generally done using default recipients, aka using a model method
      that returns the people to mail: customers ('partner_id'), customer emails
      ('email_from'), specific implementation, ...
      
      This is implementation using '_message_get_default_recipients' that returns
      'partner_ids', 'email_to' and 'email_cc' that are then used in the mail
      composer to generate final recipients.
      
      In this commit we better handle the content of email fields to avoid issues
      with multi-emails. For that purpose we correctly split the content of those
      fields. We now have several 'email_to' for records having multi-emails instead
      of a single badly-formatted 'email_to'.
      
      Task-2612945 (Mail: Defensive email formatting)
      
      Part-of: odoo/odoo#74474
      516ccbc9
Loading