[MERGE][IMP] mail: limit cache invalidation

This merge goes through various cache invalidation done in mail application
in order to effectively limit it to records and/or fields to invalidate.
Purpose is to avoid having too much cache invalidation that causes to
fetch data again, therefore adding computation and queries to various mail
features. See sub commits for more details about the changes.

All performance tests are improved of a few queries as there is less data
to fetch again from database. On overall community runbot this leads to a
gain of about 16K queries when installing all 205 modules. This merge is
related to task ID 1834147. Closes #24061 .