-
- Downloads
[MERGE] base: Remove customer/supplier fields on res.partner
1/ Improve res.partner search for customers/suppliers ========================================= Purpose ------- This commit allows to call `name_search` with context key `res_partner_search_mode` that can take currently two values: "customer" or "supplier". When ordering search results, order partners by the number of SO they have when the value is "customer" and by the number of PO if the value is "supplier". Top suppliers/customers are displayed above in a PO/SO partner dropdown. We have thought about different implementations before selecting this one: - Keep the 2 boolean flags and automatically set them if a SO or PO is created for a customer/supplier (helps a bit but doesn't work well for the first SO/PO). As the onboarding is also a priority, we didn't implement it. - Instead of filtering on the bool flags, sort the name_search() result according to the number of orders they already have (e.g. if you search for "Foo" on a PO, a supplier "SuperFoo" with 32 purchase orders will appear before supplier "HorribleFoo" that has no PO). Problem is to do this in an efficient manner for large databases - possibly we could store a "relevance index" based on the number of recent orders. Storing those values is an issue too, as it doesn't work well for multi company database. Indeed, storing values that depend on the context is certainly not a good idea. - Same as previous alternative but with no stored columns, rather a JOIN in name_search. This is the solution that have been kept. The only concern was about the perfomances. We tested it on our production base, on a sales order, as we have way much more customers than suppliers. After a deep analysis on the querry and the bitmap generated by PostgreSQL, we can conclude that the count(*) works rather efficiently and that the request is not significantly heavier than without the context key. Performance Analysis -------------------- The following results have been obtained on our production database. ### Before this commit ### The original query is the following one: ``` EXPLAIN ANALYSE SELECT res_partner.id FROM "res_partner" WHERE ("res_partner"."customer" = True) AND ("res_partner"."active" = True) AND ((("res_partner"."type" != 'private') OR "res_partner"."type" IS NULL) OR "res_partner"."type" IS NULL ) AND (res_partner.email ilike '%eezee-i%' OR res_partner.display_name ilike '%eezee-i%' OR res_partner.ref ilike '%eezee-i%' OR res_partner.vat ilike '%eezeei%') -- don't panic, trust postgres bitmap ORDER BY res_partner.display_name ilike '%eezee-i%' desc, res_partner.display_name limit 8; ``` Here is the EXPLAIN ANALYSE returned by PostgreSQL: ``` Limit (cost=1625.50..1625.52 rows=8 width=21) (actual time=13.397..13.404 rows=8 loops=1) -> Sort (cost=1625.50..1626.32 rows=326 width=21) (actual time=13.395..13.396 rows=8 loops=1) Sort Key: (((display_name)::text ~~* '%eezee-i%'::text)) DESC, display_name Sort Method: top-N heapsort Memory: 26kB -> Bitmap Heap Scan on res_partner (cost=1282.79..1618.98 rows=326 width=21) (actual time=13.025..13.350 rows=56 loops=1) Recheck Cond: (((email)::text ~~* '%eezee-i%'::text) OR ((display_name)::text ~~* '%eezee-i%'::text) OR ((ref)::text ~~* '%eezee-i%'::text) OR ((vat)::text ~~* '%eezeei%'::text)) Rows Removed by Index Recheck: 1 Filter: (customer AND active AND (((type)::text <> 'private'::text) OR (type IS NULL) OR (type IS NULL))) Rows Removed by Filter: 1 Heap Blocks: exact=58 -> BitmapOr (cost=1282.79..1282.79 rows=328 width=0) (actual time=12.983..12.983 rows=0 loops=1) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=5.022..5.022 rows=54 loops=1) Index Cond: ((email)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=4.128..4.128 rows=56 loops=1) Index Cond: ((display_name)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..321.00 rows=1 width=0) (actual time=2.240..2.240 rows=0 loops=1) Index Cond: ((ref)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..317.03 rows=4 width=0) (actual time=1.585..1.585 rows=0 loops=1) Index Cond: ((vat)::text ~~* '%eezeei%'::text) Planning time: 0.766 ms Execution time: 13.489 ms ``` ### After this commit ### Without the context key, the query looks like this: ``` SELECT res_partner.id FROM "res_partner" WHERE ("res_partner"."active" = True) AND ((("res_partner"."type" != 'private') OR "res_partner"."type" IS NULL) OR "res_partner"."type" IS NULL ) AND (res_partner.email ilike '%eezee-i%' OR res_partner.display_name ilike '%eezee-i%' OR res_partner.ref ilike '%eezee-i%' OR res_partner.vat ilike '%eezeei%') -- don't panic, trust postgres bitmap GROUP BY res_partner.id ORDER BY COUNT(*) DESC, res_partner.display_name ilike '%eezee-i%' desc, res_partner.display_name limit 8; ``` And it quite clear when looking at the EXPLAIN ANALYSE that the request is quite the same, except the aggregation that is made with a quicksort method. ``` Limit (cost=1645.00..1645.02 rows=8 width=29) (actual time=13.200..13.206 rows=8 loops=1) -> Sort (cost=1645.00..1645.82 rows=328 width=29) (actual time=13.197..13.198 rows=8 loops=1) Sort Key: (count(*)) DESC, (((display_name)::text ~~* '%eezee-i%'::text)) DESC, display_name Sort Method: top-N heapsort Memory: 26kB -> GroupAggregate (cost=1631.88..1638.44 rows=328 width=29) (actual time=13.090..13.160 rows=56 loops=1) Group Key: id -> Sort (cost=1631.88..1632.70 rows=328 width=20) (actual time=13.080..13.084 rows=56 loops=1) Sort Key: id Sort Method: quicksort Memory: 29kB -> Bitmap Heap Scan on res_partner (cost=1282.79..1618.17 rows=328 width=20) (actual time=12.793..13.055 rows=56 loops=1) Recheck Cond: (((email)::text ~~* '%eezee-i%'::text) OR ((display_name)::text ~~* '%eezee-i%'::text) OR ((ref)::text ~~* '%eezee-i%'::text) OR ((vat)::text ~~* '%eezeei%'::text)) Rows Removed by Index Recheck: 1 Filter: (active AND (((type)::text <> 'private'::text) OR (type IS NULL) OR (type IS NULL))) Rows Removed by Filter: 1 Heap Blocks: exact=58 -> BitmapOr (cost=1282.79..1282.79 rows=328 width=0) (actual time=12.755..12.755 rows=0 loops=1) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=4.823..4.823 rows=54 loops=1) Index Cond: ((email)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=3.941..3.941 rows=56 loops=1) Index Cond: ((display_name)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..321.00 rows=1 width=0) (actual time=2.186..2.186 rows=0 loops=1) Index Cond: ((ref)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..317.03 rows=4 width=0) (actual time=1.798..1.798 rows=0 loops=1) Index Cond: ((vat)::text ~~* '%eezeei%'::text) Planning time: 0.761 ms Execution time: 13.317 ms ``` With the context key, the query looks like this: ``` EXPLAIN ANALYSE SELECT res_partner.id FROM "res_partner" LEFT JOIN sale_order ON res_partner.id = sale_order.partner_id WHERE ("res_partner"."active" = True) AND ((("res_partner"."type" != 'private') OR "res_partner"."type" IS NULL) OR "res_partner"."type" IS NULL ) AND (res_partner.email ilike '%eezee-i%' OR res_partner.display_name ilike '%eezee-i%' OR res_partner.ref ilike '%eezee-i%' OR res_partner.vat ilike '%eezeei%') -- don't panic, trust postgres bitmap GROUP BY res_partner.id ORDER BY COUNT(*) DESC, res_partner.display_name ilike '%eezee-i%' desc, res_partner.display_name limit 8; ``` The only difference is that a nested loop is made for the left join, as postgreSQL has correctly identified the dicriminating table. The correct result is obtained within a similar duration. ``` Limit (cost=2700.68..2700.70 rows=8 width=29) (actual time=12.561..12.568 rows=8 loops=1) -> Sort (cost=2700.68..2701.50 rows=328 width=29) (actual time=12.559..12.560 rows=8 loops=1) Sort Key: (count(*)) DESC, (((res_partner.display_name)::text ~~* '%eezee-i%'::text)) DESC, res_partner.display_name Sort Method: top-N heapsort Memory: 26kB -> GroupAggregate (cost=2687.56..2694.12 rows=328 width=29) (actual time=12.449..12.527 rows=56 loops=1) Group Key: res_partner.id -> Sort (cost=2687.56..2688.38 rows=328 width=20) (actual time=12.408..12.425 rows=305 loops=1) Sort Key: res_partner.id Sort Method: quicksort Memory: 42kB -> Nested Loop Left Join (cost=1283.21..2673.86 rows=328 width=20) (actual time=11.629..12.340 rows=305 loops=1) -> Bitmap Heap Scan on res_partner (cost=1282.79..1618.17 rows=328 width=20) (actual time=11.596..11.862 rows=56 loops=1) Recheck Cond: (((email)::text ~~* '%eezee-i%'::text) OR ((display_name)::text ~~* '%eezee-i%'::text) OR ((ref)::text ~~* '%eezee-i%'::text) OR ((vat)::text ~~* '%eezeei%'::text)) Rows Removed by Index Recheck: 1 Filter: (active AND (((type)::text <> 'private'::text) OR (type IS NULL) OR (type IS NULL))) Rows Removed by Filter: 1 Heap Blocks: exact=58 -> BitmapOr (cost=1282.79..1282.79 rows=328 width=0) (actual time=11.560..11.560 rows=0 loops=1) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=4.724..4.724 rows=54 loops=1) Index Cond: ((email)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..322.22 rows=162 width=0) (actual time=3.825..3.825 rows=56 loops=1) Index Cond: ((display_name)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..321.00 rows=1 width=0) (actual time=1.752..1.752 rows=0 loops=1) Index Cond: ((ref)::text ~~* '%eezee-i%'::text) -> Bitmap Index Scan on res_partner_name_tgm_idx_gin (cost=0.00..317.03 rows=4 width=0) (actual time=1.254..1.254 rows=0 loops=1) Index Cond: ((vat)::text ~~* '%eezeei%'::text) -> Index Only Scan using sale_order_partner_id_index on sale_order (cost=0.42..2.74 rows=48 width=4) (actual time=0.006..0.007 rows=5 loops=56) Index Cond: (partner_id = res_partner.id) Heap Fetches: 12 Planning time: 1.056 ms Execution time: 13.791 ms ``` 2/ Remove customer and supplier fields from res.partner =========================================== Purpose ------- Fields `customer` and `supplier` on `res.partner` are mostly used in domains of many2x fields. Those domains can confuse end users because they don't see the partner they are looking for; and it's not obvious why. Some identified problems: 1. It can lead to duplicated partners: the user does not find the partner, so he creates a new one. 2. The user imports supplier contacts in the Contacts app, so they don't get the `supplier` flag. Then the user wants to make a purchase order, and cannot find the new suppliers in the list 3. A user removes the customer flag on a prospect, because they don't think it's a customer yet - except now they can't make a quote for that customer... Specification ------------- Remove the two mentioned fields. Since fields `customer` and `supplier` have been removed, all partners are now shown in many2one dropdowns. But in some cases, not all partners are relevant or some are more likely to be relevant than others. e.g. when creating a PO, top suppliers have a higher priority than other partners. So, adapt the places where those fields were used with the new mechanism to display the searched the partners, according to the number purchase/sales orders they made. TaskID: 2031147 closes odoo/odoo#34524 Signed-off-by:Yannick Tivisse (yti) <yti@odoo.com> Co-authored-by:
Yannick Tivisse <yti@odoo.com>
No related branches found
No related tags found
Showing
- addons/account/models/account_move.py 0 additions, 3 deletionsaddons/account/models/account_move.py
- addons/account/models/reconciliation_widget.py 1 addition, 6 deletionsaddons/account/models/reconciliation_widget.py
- addons/account/report/account_invoice_report_view.xml 2 additions, 2 deletionsaddons/account/report/account_invoice_report_view.xml
- addons/account/static/src/js/reconciliation/reconciliation_model.js 2 additions, 8 deletions...ount/static/src/js/reconciliation/reconciliation_model.js
- addons/account/static/src/js/reconciliation/reconciliation_renderer.js 1 addition, 1 deletion...t/static/src/js/reconciliation/reconciliation_renderer.js
- addons/account/static/tests/reconciliation_tests.js 8 additions, 10 deletionsaddons/account/static/tests/reconciliation_tests.js
- addons/account/tests/account_test_no_chart.py 0 additions, 1 deletionaddons/account/tests/account_test_no_chart.py
- addons/account/views/account_move_views.xml 1 addition, 4 deletionsaddons/account/views/account_move_views.xml
- addons/account/views/account_payment_view.xml 1 addition, 1 deletionaddons/account/views/account_payment_view.xml
- addons/account/views/account_view.xml 1 addition, 2 deletionsaddons/account/views/account_view.xml
- addons/account/views/partner_view.xml 3 additions, 3 deletionsaddons/account/views/partner_view.xml
- addons/auth_signup/models/res_users.py 0 additions, 1 deletionaddons/auth_signup/models/res_users.py
- addons/base_automation/data/base_automation_demo.xml 1 addition, 1 deletionaddons/base_automation/data/base_automation_demo.xml
- addons/base_automation/tests/test_base_automation.py 4 additions, 4 deletionsaddons/base_automation/tests/test_base_automation.py
- addons/base_automation/tests/test_models.py 1 addition, 1 deletionaddons/base_automation/tests/test_models.py
- addons/crm/views/crm_lead_views.xml 4 additions, 4 deletionsaddons/crm/views/crm_lead_views.xml
- addons/crm/views/res_partner_views.xml 0 additions, 1 deletionaddons/crm/views/res_partner_views.xml
- addons/crm/wizard/crm_lead_to_opportunity_views.xml 1 addition, 1 deletionaddons/crm/wizard/crm_lead_to_opportunity_views.xml
- addons/delivery/views/partner_view.xml 1 addition, 1 deletionaddons/delivery/views/partner_view.xml
- addons/fleet/models/fleet_vehicle_cost.py 4 additions, 4 deletionsaddons/fleet/models/fleet_vehicle_cost.py
Loading
Please register or sign in to comment