freepoteries-doc/content/mailman.rst

mailman
#######

:date: 2021-06-21 13:00
:modified: 2021-06-21 13:00
:tags: mail mailing-list
:category: documentation
:slug: mailman
:authors: Ms.Leave-A-Mix


We have a mailman instance on the mail server that runs on Alpine Linux. Here's
a how-to install this mailman server.


I did this after a failed upgrade caused to Pip packages that I installed in the
same repository that alpine python packages. That caused troubles during the
upgrade from python3.8 to python3.9.

I backuped and removed /usr/lib/python3.{7,8}.


I'll barely follow the mailman documentation on "how to install mailman on a
virtual env".


--------
cleaning
--------

I had a huge problem with my previous mailman installation, that I did with Pip.
Pip mixed PyPi packages with my alpine-repo python packages (those that begins
with py3... and some others).

Here's how I did to clean my server :

I listed all installed packages and their content.

:code:`apk list -I|awk -F'[\}\{]' '{print $2}'|xargs -I'{}' apk info -L "{}" > /tmp/installed_packages_content`

Then I checked all the pacckages that had at least one file installed in the
:code:`/usr/lib/python3.9/site-packages` directory.

:code:`grep -E '((site-packages.*__init__.py)|(:$))' ./installed_packages_content |awk '/:$/{package=$0}/^usr/{usr=$0;if(prev!=package){print package;print $usr};prev=package}'|rev|cut -d'/ ' -f2-|rev`


I quickly realized that many of the directories in
:code:`/usr/lib/python3.9/site-packages` where created by pip. So I simply backuped this
folder, and then removed it. Then I looped on the list of alpine packages I got
to reinstall them.

:code:`apk list -I|awk -F'[\}\{]' '{print $2}'|xargs apk fix`

I saw that the py3-* packages and some others (certbot, certbot-nginx and
py-augeas) where causing errors. So I just reinstalled them.

:code:`apk del py3-* py-augeas certbot certbot-nginx`

:code:`grep "^py3" /tmp/installed_packages|xargs apk add`

:code:`apk add certbot certbot-nginx py-augeas`


------------
installation
------------

mailman
-------

https://docs.mailman3.org/en/latest/install/virtualenv.html

As I already had a mailman instance, I just had to do the following to create a
virtual environment.

.. code-block:: bash

   cd /opt/mailman
   chown -R mailman:mailman .
   su mailman
   python3.9 -m venv venv
   source ./venv/bin/activate
   echo 'source /opt/mailman/venv/bin/activate' >> /opt/mailman/.bashrc

Check if the version of pip you use is really the one of the virtual env :

.. code-block:: bash

   (venv) {13:24} mail:/opt/mailman$ which pip
   /opt/mailman/venv/bin/pip


:code:`pip install wheel mailman`

-- An error occured because "pg_config" did not exist. As I won't need psycopg2, I
-- just avoid this and install pymysql instead (I run a mysql database for my mail
-- server).

:code:`pip install pymysql`

I'll modify the mailman init to point to the one in the virtual environment.

.. code-block:: bash

   vim /etc/init.d/mailman


   #!/sbin/openrc-run
   # Copyright 1999-2004 Gentoo Foundation
   # Distributed under the terms of the GNU General Public License v2
   NAME="mailman"
   CONF=/etc/mailman.cfg
   PIDFILE=`grep -o "^pid_file:.*" "$CONF"|cut -d' ' -f2`
   BIN="/opt/mailman/venv/bin/mailman"
   USER="mailman"
   ENV_="MAILMAN_CONFIG_FILE=$CONF"

   extra_commands="info conf"

   depend() {
         need mta
         use logger
   }

   start_pre() {
         sleep 0
   }

   start() {
         ebegin "Starting mailman"
         mkdir -p -m 0750 /var/run/mailman
         chown mailman: /var/run/mailman
         mkdir -p -m 0750 /var/log/mailman
         chown mailman: /var/log/mailman
         mkdir -p -m 0750 /var/lock/mailman
         chown mailman: /var/lock/mailman

         start-stop-daemon -S -p $PIDFILE -u $USER -e $ENV_ \
                  -x $BIN start --
         eend 0
   }

   stop() {
         ebegin "Stopping mailman"
         start-stop-daemon -K -p $PIDFILE -e $ENV_ \
                  -x $BIN stop --

         eend 0
   }

   status() {
         ebegin "Mailman status"
         eval $ENV_
         $BIN status
         eend 0
   }

   info() {
         ebegin "Mailman info"
         eval $ENV_
         $BIN info

         eend 0
   }

   conf() {
         ebegin "Mailman configuration"
         eval $ENV_
         $BIN conf

         eend 0
   }

   svc_restart() {
         ebegin "Restarting mailman"
         start-stop-daemon -K -p $PIDFILE -u $USER -e $ENV_ \
                  -x $BIN restart --

         eend 0
   }


Modify mailman.cfg according to new configuration (path.fhs.bin_dir)

.. code-block:: ini

   [mailman]
   layout: fhs
   site_owner: root@freepoteries.fr
   noreply_address: no-reply

   [mta]
   incoming: mailman.mta.postfix.LMTP
   outcoming: mailman.mta.deliver.deliver
   lmtp_host: 127.0.0.1
   lmtp_port: 8025
   smtp_host: 127.0.0.1
   smtp_port: 25

   [webservice]
   hostname: 0.0.0.0
   port: 8081
   use_https: no
   admin_user: root
   admin_pass: password
   api_version: 3.1

   [database]
   class: mailman.database.mysql.MySQLDatabase
   url: mysql+pymysql://mailman:password@maria.freepoteries.fr/mailman?charset=utf8&use_unicode=1
   debug: yes


   [paths.fhs]
   # Filesystem Hiearchy Standard 2.3
   # http://www.pathname.com/fhs/pub/fhs-2.3.html

   # BEGIN(MODIFY THIS)
   bin_dir: /opt/mailman/venv/bin
   # END

   var_dir: /var/mail/mailman
   queue_dir: /var/spool/mailman
   log_dir: /var/log/mailman
   lock_dir: /var/lock/mailman
   etc_dir: /etc
   ext_dir: /etc/mailman.d
   pid_file: /var/run/mailman/master.pid

   [language.master]
   description: English (USA)
   charset: utf-8
   enabled: yes

   [language.fr]
   description: Français
   charset: utf-8
   enabled: yes



I also removed /etc/mailman/conf.d/mailman.cfg as it was unused.


Start the service and test a mailing list (i.e. test@lists.freepoteries.fr).

:code:`rc-service mailman start`

Check if the logs are not crying.

:code:`tail -n1000 /var/log/mailman/mailman.log`


Freeze the requirements.

:code:`pip freeze > /opt/mailman/requirements.txt`

Add the crontabs.

:code:`EDITOR=vim crontab -e -u mailman`

.. code-block:: crontab

   0       2       *       *       *       /opt/mailman/venv/bin/mailman digests --periodic
   0       2       *       *       *       /opt/mailman/venv/bin/mailman notify


Git commit all those changes before continuing.


mailman-web
-----------

Let's take care of the web interface.

You can check the documentation in the mailman-web repo/docs/index.rst .

You should be in the same virtualenv as before.


mailman-web needs the cryptography, as specified in the cryptography manual :

   As If you are on Alpine or just want to compile it yourself then cryptography
   requires a C compiler, a Rust compiler, headers for Python (if you’re not
   using pypy), and headers for the OpenSSL and libffi libraries available on
   your system.

The rust package is the compilator, and the cargo is the rust package manager.

:code:`apk add rust cargo`

:code:`pip install mailman-web mailman-hyperkitty`

Freeze requirements and git the files.

:code:`pip freeze > /opt/mailman/requirements.txt`


Migrate mailman new version.

:code:`mailman-web migrate`

:code:`mailman-web collectstatic`

:code:`mailman-web compress`

:code:`mailman-web compilemessages`


uwsgi
.....

"env pythonpath" was not taken into account by django, I changed the option by
settings "env = PYTHONPATH=/etc/mailman3".

TODO: Archivage