Before the commit, the _get_seen_list() function in the mass_mailing module was
not able to correctly identify all the duplicate email addresses in a given mass
mailing. This was because the function chose and used only one way to find an
email address for each record in the mailing list, even though there are many
ways to find an email address for a record.

For example, a crm.lead record might have an email address in its partner_id
field, but it might also have an email address in its email_normalized field.
This can vary from record to record.

To fix this issue, the _get_seen_list() function was updated to only look at the
email address to which emails have already been sent, rather than trying to
fetch it from the record itself. This ensures that all duplicate emails are
correctly identified and that no duplicate emails are sent in the mass mailing.

Task-3234378

closes odoo/odoo#118220

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>

PURPOSE

Be defensive when dealing with email fields, notably when having multi-emails
or email field containing an already-formatted email.

RATIONALE

Add tests related to not standard usage of email field. Two main use cases
are tested here

  * formatted emails: `"Full Name" <email@domain.com>` stored into the 'email'
    field;
  * multi emails: `email1@domain.com, email2@domain.com` stored into a single
    'email' field;

Additional tests: tests with unicode / ascii / case / wrong formatting are also
added to check the support in normalize and format methods.

IMPLICATION

Email field is generally managed as "containing a valid email". This means
it is sometimes used as it in 'formataddr' as well as to perform searches or
identification checks. Example of issue: partner 'Raoul' has a formatted email
like "Raoul" <raoul@raoul.fr>. Using 'formataddr' in email_from leads to

  from: "Raoul" <"Raoul" <raoul@raoul.fr>>
  -> which is incorrect (but often dynamically corrected by email servers);

Email field holding multi-emails are not normalized, as current normalize
is done only if the field holds a single email. It means

  * no easy finding based on 'email_normalized', e.g. various tools like
    '_mail_find_partner_from_emails' or 'find_or_create' do not find partners
    based on this email;
  * no exclusion list management;
  * issue with formatting, like

  to: "Raoul" <raoul@raoul.fr,raoul.other@raoul.fr>
  -> which is incorrect (but often dynamically corrected by email servers);

USAGE: OUTGOING EMAILS

Those use cases currently generate faulty outgoing emails. This is valid for
recipients ('email_cc', 'email_to') as well as author ('email_from').

For formatted emails: `email_to` is formatted again based on name and email
which leads to sending emails to `"Full Name" <"Other"<email@domain.com>>`.

Note that multi emails without formatting may work as it leads to email_to
`"Full name" <email1@domain.com,email2@domain.com>`. Some outgoing email
servers correctly send multiple emails. It depends on their fault
tolerance.

USAGE: FIND BASED ON EMAIL (NORMALIZED)

When searching for partners (e.g. using '_mail_find_partner_from_emails' or
'find_or_create') normalized version of input is used.

In case of multi emails sanitize is 'False', as normalization expects a single
email in the field. Therefore no partner is found. In processes that do a
"search or create" (e.g. using a template on a record) this leads to creating
a new partner (or several partners in case of multi emails) each time.

USAGE: OTHER FLOWS

Other flows are build on top of '_mail_find_partner_from_emails' / 'create'
of outgoing emails and are impacted by formatted email / multi email usage.
Those include notably

  * mass_mailing: '_message_get_default_recipients' should be defensive to
    give correct values when creating mailing emails;
  * mass_mailing: faulty emails is based on normalize and multi-emails are
    considered as faulty and ignored;
  * after post hook: '_message_post_after_hook' tries to link messages without
    author (but email_from) with newly-created partners, when partners are
    created from chatter. It is therefore impacted by those corner cases;
  * marketing_automation: built on top of mass_mailing and suffers from the
    same issues;

USAGE: UNICODE

Unicode in emails should be supported. 'formataddr' and IrMailServer notably
received fixes to support unicode. Some check performed on email addresses
fail when unicode is involved, which leads to some emails not being sent
while they could.

SPECIFICATIONS

Add tests related to those corner cases. Also add tests for computation of
`email_formatted` field of Partner model. It currently generates wrong email
values for the same corner cases (multi emails, formatted emails).

Tests are also added for the computation of `email_normalized` field used
notably for blacklists. It is not computed currently when being in multi
email mode which prevents from any blacklist mechanism as well as make
email finding harder. `_mail_find_partner_from_emails` tool method is also
tested with multi email as it uses the same heuristic as normalized email
field.

Tests are also added for mass mailing, when having to mail documents that
have a partner with formatted emails / multi-emails, or that have an email
field with formatted emails / multi-emails.

Also restore a test removed at odoo/odoo@afcb7349083c669dbda18b9b1955eabbe14ea675 while it should have been
updated to state that email addresses containing non-ascii characters are
supported.

Add some tests for tools methods used in various email processing flows.
Unicode tests are also added.

In future commits we will try to make email usage a bit more defensive to
try to lessen issues with that kind of use cases.

Task-2612945 (Mail: Defensive email formatting)

Part-of: odoo/odoo#74474

Add a model and a test allowing to test the seen list using raw SQL based
on partner_id field. Test indicates a not stored partner_id field currently
crashes beyond redemption.

Task-2852943

X-original-commit: odoo/odoo@cff53e078db2f3c9dc03a873f94146a7808e6989
Part-of: odoo/odoo#91533

USE CASE

Send a mailing to some recipients. Set reply-to of this mailing to an alias
creating records on a model inheriting from utm.mixin. UTM informations on
mailing are not propagated to the newly-created records.

SPECIFICATIONS

When an user answers to a mail of a mailing with an alias creating records
on an UTM enabled model, propagate UTM from the mailing to the new record.

In order to set UTM informations during record creation, "message_new" has
been overridden in mass_mailing. That way it is automatically available on
all models inheriting from utm.mixin.

A test simulating this scenario has been added. It checks if UTMs info are
correctlwy set on the created record by comparing them to the mailing's UTM
info.

Task Id 2210334

closes odoo/odoo#48602

Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>

URPOSE

Prepare code cleaning and optimization in mail, mass_mailing and SMS by
cleaning models for readability and code complexity and footprint reduction.

SPECIFICATIONS

Add tool methods and asserts for link tracker tests.

Add tool methods and assets for mass mailing tests, notably about mailing
traces and their link with emails.

Merge existing and duplicated tests and rewrite them a bit.

Rename and improve test_mass_mailing models.

LINKS

Prepares task ID 2238597 (notification and trace models cleaning)
Task ID 1906925 (mass mailing tests cleaning)
PR #50169

Related: odoo/enterprise#10180
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>

Purpose of this commit is to add the skeleton of a module holding models and
tests for mass mailing. It is build on top of test_mail in order to use
data, models and parameters already defined in it. Moreover having a
separate module allow to define test models that will not bloat production
databases.

It will soon hold unit tests for blacklist mechanism. Future commits will
gradually move tests from mass mailing module and improve them.

This commit is related to task ID 33224 (original blacklist implementation
done for v12) and its PR #25966 as well as task ID 1889703 (tests and fixes)
and its PR #27330. Done with collaboration of @dbeguin.