We have our own html2plaintext, already used in lot of use cases instead of
just a few for the html2txt library.

Notably for emails: most emails going through Odoo stack use our simple
html2plaintext to format the body alternative. When no body alternative
is given to ``build_email`` an alternative is built using the library to
remove. Using our own parser allows to have the same results compared to
using ``MailMail.send()``. Difference lies in spaces and new lines as well
as markdown. Our html2plaintext is a bit simple and does not try to generate
Markdown but generates a simple plaintext version.

This also helps solving some issues with depending on that library.

Task-2702034

closes odoo/odoo#82330

X-original-commit: odoo/odoo@9b5f86e8fa5bef341ea7392b86987092d6f6933a
Related: odoo/enterprise#23292
Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>

- Up to odoo 10.0, feedparser dependency was optionally used in the cli of vendored html2text.py
  (see https://github.com/odoo/odoo/blob/10.0/addons/mail/models/html2text.py#L437)
- Since 11.0 (67c17cb3), vendored html2text.py has been removed in favor of maintained package
- Turns out the feedparser part in html2text was dead code for a long time anyway
  (see https://github.com/Alir3z4/html2text/issues/220

)
- So we can safely drop this dependency

closes odoo/odoo#76273

X-original-commit: 271b9468
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

In Debian and Ubuntu distros, the python reportlab package provides
multiple binary packages among which python3-reportlab [0] [1] and
python3-renderpm [2] [3]. The latter is only recommended, but is mandatory for
odoo because it's needed for rendering barcode with the `asString` method.

[0] https://packages.debian.org/buster/python3-reportlab
[1] https://packages.ubuntu.com/focal/python3-reportlab
[2] https://packages.debian.org/buster/python3-renderpm
[3] https://packages.ubuntu.com/focal/python3-renderpm



closes odoo/odoo#67410

X-original-commit: 73e99264
Signed-off-by: Xavier Dollé (xdo) <xdo@odoo.com>
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Zeep replaced suds but Debian and Rpm packaging were not updated.
Ofxparse is required but did not appear in Debian nor Rpm packaging.

closes odoo/odoo#66784

X-original-commit: 39738f2b
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

When using the windows installer in French language, the `Hôte` label
used to configure postgresql server does not display correctly.

The LangString documentation does not specify how to use the special
characters but after some tests, specifying a BOM for the nsi file seems
to be the way to go.

closes odoo/odoo#65365

X-original-commit: 3d0871d1
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

As Fedora 32 was the current release when Odoo 14.0 was released, this
should be the supported version.

Also, a few  old libs were still in mentioned in the packaging files.
They flew under the radar because they never broke the packaging.
This is not the case anymore, those libs disappeared from the Fedora
repos.

It seems that pyparsing is not used anymore since 5a1c06a1 and thus can
be safely removed from `requirements.txt` too.

pychart is not used anymore since 3425752e.

While at it, remove mix of tabs and spaces in package.dffedora, also add
missing packages to avoid installation at test time.

Now that I started down the slippery slope, also removed some `-dev`
packages in package.dfsrc as wheel's are available.

Finally, the rpm install script now detects the python ABI version in
order to avoid update this file at each ABI change in Fedora.

Fixes #63719

closes odoo/odoo#65224

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

The createrepo Debian package is not available in Ubuntu anymore, this
can cause problems on Ubuntu based build systems.

In order to solve that once and for all, with this commit the rpm repo is
generated from a Docker container.

Fixes #63419

closes odoo/odoo#63580

X-original-commit: 8f5c5e58
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Only updates outdated requirements which actively cause issues:

* freezegun broken in 3.8 (removal of time.clock)
* xlrd broken in 3.8 (removal of time.clock)
* also monkeypatches xlrd.xlsx for 3.9 (removal of
  Element.getiterator, breaks because of defusedxml)
* jinja triggers DeprecationWarning in 3.8
* pillow triggers warning in 3.9
* lxml, greenlet don't compile in 3.9
* reportlab doesn't work in 3.9

New versions try to match those of Debian Bullseye.

Also adds a script to more easily compare dependency versions between
the requirements files and what's in various distributions (currently
supports checking against debian and ubuntu).

Furthermore updates warnings filtering:

* removes xlrd (mischeck was monkeypatched as noted above)
* removes setuptools (was for older versions, one would hope this
  isn't an issue anymore)
* adds babel: python-babel/babel#684 fixes the deprecation warning but
  is not part of any release yet
* ignores error related to `random.sample` on a set, this is a
  diagnostics bug because recordsets implement both Sequence and Set,
  and the stdlib checks for Set first (bpo-42470)

See #59980
Closes #61103

closes odoo/odoo#62380

Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>

The package.py script is moving the addons in odoo/addons which is not
more necessary for the Debian package as it's made at the Debian
packaging level. So package.py will not do that anymore.

closes odoo/odoo#60421

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

In some undeterministic conditions, the Windows virtual machine can take
more than 60 sec to start and have the sshd service ready.

With this commit, the packaging script tries to connect to the VM up to
30 times, giving more than 10 minutes for the boot.

As soon as the connexion is ready, the packaging starts. This means
that the script will probably start sooner in most of the cases.

closes odoo/odoo#58271

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Up to Odoo 13.0, the windows setup is 32 bits and embeds an old
postgresql 9.5 32 bits version too.

With this commit, the postgresql server is no longer embedded in the
resulting setup file. Instead, if the user choose to install the
postgresql server, the 12.4 version is downloaded from the official site
(version 10.14 in the case of a 32 bits windows).

Also, the Python target achitecture is choosen based on the Windows
system on which Odoo is installed.

This imply changes on the KVM images used to build the installer:

    - The Windows system on the KVM as to be 64 bits.
    - A 32 bits WinPython version must be in `c:/odoobuild/WinPy32`
    - A 64 bits WinPyhton version must be in `c:/odoobuild/WinPy64`

A little bit of cleaning also comes with this commit to get rid of the
old `openerp` references.

closes odoo/odoo#57155

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

When the Odoo installation is tested, if the test fails an error is
raised but the Docker container is not stopped.

As a result, a ghost container stays alive with a Postgres server
running. Worse, if another package is built on the same host, the
ghost container prevent the other build to succeed.

With this commit, a specific Exception is raised and the container is
stopped in any case.

closes odoo/odoo#56270

X-original-commit: fbe2092d
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

As this file was only used as a one shot for the windows pacakging, it
can be removed from the code.

closes odoo/odoo#55307

X-original-commit: 19b44653fef85a851239c656b1fdf4798ab83f34
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

When building the package list of the APT repository, dpkg-scanpackages
reports the following and only output the latest package information in
the Packages file:

```
dpkg-scanpackages: warning: Packages with multiple instances but no --multiversion specified:
dpkg-scanpackages: warning: ./odoo_13.0.20191002_all.deb
dpkg-scanpackages: warning: ./odoo_13.0.20191003_all.deb
[...]
dpkg-scanpackages: warning: ./odoo_13.0.20200630_all.deb
```

Having only the latest version listed in the Packages file makes it
impossible to tell apt to use a specific version through apt-pinning.
It also makes it impossible to revert to a previous version if a
regression is caused by an update.

For these reasons, tell dpkg-scanpackages to include all versions of the
odoo package in the Packages file it generates.

closes odoo/odoo#54302

X-original-commit: f41c00a8
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Since 5952928b, the python3-mock debian package is removed from the
Dockerfile used to build the Debian package. This prevent the Odoo
Debian package to be built.

The python3-mock debian package has an inderect dependency on
python3-setuptools which is necessary to build the package.

With this commit, the dependency is explicitely set in the Dockerfile.

closes odoo/odoo#49278

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Before this commit, a lot of leftover import shims existed in the
codebase for py2-py3 compatibility, these are no longer needed since
Odoo 13.0+ doesn't support Python 2 anymore and is (finally) in EOL.

With this commit, these shims are dropped, making the code cleaner,
easier to read and with one less dependency.

Queue -> queue -> py2-py3 compatibility
xmlrpclib -> xmlrpc.client -> py2-py3 compatibility
ConfigParser -> configparser -> py2-py3 compatibility
itertools.izip_longest -> itertools.zip_longest -> py2-py3 compatibility
urllib -> urllib.request -> py2-py3 compatibility
__builtins__ -> builtins -> py2-py3 compatibility
_winreg -> winreg -> py2-py3 compatibility

mock -> unittest.mock -> merged into CPython

The debian/fedora packages and requirements.txt have been updated accordingly

closes odoo/odoo#44601

Related: odoo/enterprise#8141
Signed-off-by: Xavier Morel (xmo) <xmo@odoo.com>

Python module vatnumber doesn't seem maintained anymore. Therefore, we should:
    - call directly stdnum (which is maintained and mostly used everywhere in vatnumber)

Also improve stdnum import, vat fix method and vat expected formats

task-1915371

closes odoo/odoo#36978

Signed-off-by: Quentin De Paoli (qdp) <qdp@openerp.com>

At 795c7b0a the external dependencies was changed from trying
to import 'ldap' to checking than 'pyldap' package was installed.
The problem is that pyldap is a unmaintained library that should no
longer be used, as explained on the package page:
https://pypi.org/project/pyldap/


"The pyldap fork was merged back into python-ldap, and released as
 python-ldap 3.0.0."

Having pyldap version >= 3.0 installs python-ldap automatically and
will not cause any issue.

The Debian control file package name is adapted to use the latest.

The "ldap" externalm dependency defined in __manifest__.py will cause
pkg_resources.get_distribution() to fail in both case ("python-lap" or
"pyldap"), but the "import" fallback will succeed. For that reason, the
log warning is turned into a log info.

closes odoo/odoo#43769

Note: This library should be replaced by the pure python "ldap3" library.
X-original-commit: 1afd0ccf
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

When building the MS Windows package, the PYTHON_VERSION variable is not
used and the WinPython path is hard coded in the Makefile and NSI file.
This prevent the usage of a newer version of Python.

With this commit, the PYTHON_VERSION is used to compute the WinPython
Python directory. That way, this directory can be derived from
package.py command line argument --vm-winxp-python-version.

Also the less windows binaries are not packaged anymore.

closes odoo/odoo#39821

X-original-commit: ac90d858
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

closes odoo/odoo#39199

X-original-commit: 25d6e9a6
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Packaging oversight, the displayed version in the installer window was still
12.0.

closes odoo/odoo#38486

X-original-commit: c5bd1a4f
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Actually, docker is used to test Linux packages once they are built.
The packaging process is done on the host system where package.py is
run.

Consequences:
    * difficult to reproduce an exact same build on another host
    * changing a Debian/Ubuntu target version means some upgrade steps
      on the build host (mix that with the fact that the host also
      could build old Odoo versions)
    * Fedora rpm package is built on an Ubuntu system (which is not a
      real problem but it could be)

With this commit, the package.py utility is refactored to use Docker
containers to build the Gnu/Linux packages.

Also, the Debian package is adapted for Debian Buster, the RPM package
is built based on Fedora 30.

Some minor changes are also made in the windows packaging system.

closes odoo/odoo#37766

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Use polib library that handles this correctly
The complexity of the parser is moved to the library

The use of polib as introduced in commit 632fa044.
The nightly packaging system uses Docker to test the packages and they need to
be updated when a new library dependency is introduced.

closes odoo/odoo#34117

Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>

Recommended by GitHub's repository alerts.

We normally stick as close as possible to the version we depend
on in the official DEB packages. This in turn depends on the version of
Debian stable at the time of release - for 11.0 that would be Debian 9
(stretch) and thus Jinja 2.8 (with security backports).

However Jinja2 before 2.10.1 suffers from a few issues that could lead
to crashes of Odoo processes.

It seems it's worth an exception to our rule for pip users, similarly to
previous bump up at d2605bcc.

closes odoo/odoo#32601

Signed-off-by: Christophe Simonis <chs@odoo.com>

Recommended by GitHub's repository alerts.

We normally stick as close as possible to the version we depend
on in the official DEB packages. This in turn depends on the version of
Debian stable at the time of release - for 11.0 that would be Debian 9
(stretch) and thus Jinja 2.8 (with security backports).

However Jinja2 before 2.10.1 suffers from a few issues that could lead
to crashes of Odoo processes.

It seems it's worth an exception to our rule for pip users, similarly to
previous bump up at d2605bcc.

closes odoo/odoo#32601

Signed-off-by: Christophe Simonis <chs@odoo.com>

- Added new library zeep for soap request.
- Remove old suds library.

Task ID: 41696 Closes #26934

Fedora 29 switched to python 3.7. It seems that there is no official
way to build a RPM package that could works with 3.6 and 3.7.
Building two different packages for each version could open the pandora
box of packaging.

With this commit an ugly hack is used: if python3.7 is found, the odoo
directory is symlinked into its site-packages dir.

fixes #30579

closes odoo/odoo#30618

After the Odoo package is built, the package.py script uses a Docker
image to test the package. After that python3-qrcode was added as a
dependency for the package in 2ff49c5f , it revealed some issues.

A missing cli parameter and a forgotten '&&' in the Dockerfile prevented
the installation of the depending packages.

Also, the script shebang was wrong by launching python2.

Finally, the fact that python3-xlwt is missing in Debian stretch was
highlighted. With this commit, the python3-xlwt is explicitly removed
from the dependencies and the documentation is updated accordingly.

closes odoo/odoo#28807

PKG-INFO is generated during the setup phase and is not needed anymore.
Also this file contains outdated informations.

py2exe is not used anymore to build the windows installer.

closes odoo/odoo#28494

Fixes #27420

closes odoo/odoo#27590

Commit cf853a78 removed all yml tests
and the yml import engine from Odoo forever, however PyYAML remains a
dependency even though it's not used anymore.

This commit removes any reference to this lib that could be found.

closes odoo/odoo#27563

Since 2f7c03d9 the admin user could have any uid and the uid 1 is
reserved for the system user. As the uid was hardcoded as 1 when testing
packages, the tests were failing.

With this commit, the uid of the admin user is searched before testing
the packages.

Closes #26603

Since bf3193ac libsass is needed but is named libsass0 in Debian.
With this commit, the docker source image used for tests will build again.

Since 565277ac sassc is used as a
fallback when the python libsass is not available.

With this commit, the saasc package dependency is added to the rpm to
avoid installing libsass with pip.

Since 285ead28, `base` should be loaded
as a server wide module to enable rpc routes.

When the virtual machine used to build MS Win package takes too much
time (to build or to test the build), a warning is logged and the VM is killed.

Before this commit, the PID of the process was passed as an argument to
the warning method, causing a traceback.

When building the Windows installer, a virtual machine is used with
prepackaged python modules.

With this commit, the packaging script will try to install on the
virtual machine the python packages specified in requirements.txt.
Each package is installed individually, that way, if an install fails,
the install of the other packages continues.
At the end of the process, successfully packages are listed so that they
will appear in the build log files.

Also, the Makefile was cleaned in this commit (removal of py2exe stuffs).

A dependency to libsass was added in debian package and in the docker
file. The Odoo nightly builds of the deb package was failing because
this lib is named libsass0 in Debian.

Finally, this dependency is not required because python3-libsass already depends on libsass0.

Include libsass instead of node-less

Thank you to @d-fence for testing this.