Skip to content
Snippets Groups Projects
Commit 9e675254 authored by Denis Vermylen's avatar Denis Vermylen Committed by Olivier Dony
Browse files

[FIX] server.py: <threaded> avoid registry lock upon shutdown


A deadlock can occur between threads when concurrent requests
acquire the registry lock and conflicting database-level locks
in different orders. The database won't be able to detect and
break the deadlock because it involves an external, Python-level
lock. This situation is more likely to occur during module
installations [1].

If the server is started with the `limit_time_real` option,
it should be able to abort the deadlocked requests after the
timeout, and restart. However that could not work because
the recovery initiated by `reload()` is blocked at the end
of the `stop()` method, as it cannot acquire the registry
lock either, necessary for `Registry.delete_all()`.

Since that deletion step is in fact not necessary, it can
be skipped, avoiding the deadlock entirely.

Indeed there's no real reason anymore to delete the DB's
registry upon shutdown. This was introduced for 7.0 by
b5daffc1, in order to perform
other cleanups (including cron agent threads). These other
cleanups are not necessary anymore, and when the stop()
method of the ThreadedServer completes, the next step is
either a restart of the whole process (via execve() through
_reexec()), or a full process exit. Keeping the registry in
memory for a few cycles until this happens makes no difference.

When such a deadlock occurs, it's always possible to manually
kill the server with 2 `kill` commands, or 1 `kill -9`.

~~~~~~~~~~~~~~~~~~~~~~
[1] Reproduction info:

The following deadlock was observed in Odoo threaded server mode:

1. incoming request spawns a new thread A
   A starts a transaction and does a "SELECT ... FROM res_users ..."
   getting an ACCESS SHARE lock on the table
2. incoming request spawns a new thread B
   B is a request that calls `button_immediate_install`, that will
   install new modules and alter the res_users table.
3. B takes and holds the registry lock and executes "ALTER TABLE
   res_users ...", that waits to get the ACCESS EXCLUSIVE lock on the
   table until A's transaction releases the ACCESS SHARE lock.
4. A continues code execution and reaches a .sudo() call, it tries to
   create a new environment. The creation of the new environment
   requires to wait for the registry's lock to be release but it's held
   by B.

-> A waits for B's registry lock to be released
-> B waits for A's ACCESS SHARE lock to be released
-> Deadlock that can't be broken except by force-killing the server

closes odoo/odoo#40339

Signed-off-by: default avatarOlivier Dony (odo) <odo@openerp.com>
parent 896127a6
Branches
Tags
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment