Running Mailpile from a docker container

#1
#2

Cool :slight_smile: While I won’t use it myself, I think this is very welcome for some users. Good job, thank you for contributing :smiley:

#3

What would be an easier way to install Mailpile for a group of people, for example employees of a company or students of a school? Or do you mean that you wouldn’t use Mailpile at all in such a case? What could be any alternative?

#4

Strictly personal speaking: I don’t “do” Docker (nor do I use an OS that supports it). From a personal attitude, I’d rather help others install Mailpile themselves or automate it with Ansible than running it as a webmail service for a group of users.

Then again, there is a clear need for something like your Docker scripts for others. Mailpile has a superiour interface to Squirrelmail, Roundcube and many other webmail scripts. Also, the functionality. I have yet to stumble upon a webmail that has a search that is as fast als Mailpile’s (hundreds of thousand of messages can be searched within a second!).

Thank you very much for your scripts :smiley: I most certain see the benefits and would recommend it to others if searching for a easy way to kick a Multipile instance online.

#5

This is neat, thanks for doing this!

As a Docker newbie, I have two (sets of) questions which I think your README could perhaps be updated to answer better:

  1. Does per-user Mailpile data live somewhere outside the container? Does it persist even if the container is deleted?

  2. How do I update the Mailpile inside the container, if there is a new release or a security bug is fixed? Is this or can this be automatic?

#6

The README only says “The user data are on the directory accounts/”, which I admit is a bit cryptic. But to answer your questions:

  1. Yes, the user data lives on the directory accounts/, which is shared between the host and the container (it is mounted on /home/ inside the container). If the container is deleted the user data is still there.

  2. Normally you do just ds make, which rebuilds the container. However Mailpile is installed by apt install mailpile mailpile-multi, so if nothing is changed on these packages nothing will be updated.

#7

I realized that my second answer is not correct. This is because Docker uses cache for building the images, so ds make will use the cached image and will not build a new one that contains the latest versions of the packages.

This can be solved by running ds remove first, which will delete the image as well, or by running first ds build --no-cache, which will rebuild the image completely from scratch, ignoring any caches.

To make this even easier I have added the command ds remake, which will run these two commands: ds build --no-cache and then ds make.

#8

Cool! Thanks for your responses and your updates. Those have been my main concerns about dockerization of Mailpile, so this is definitely important progress.

Are you using Mailpile yourself? That’s the other concern; the reason the Mailpile project doesn’t officially provide Docker support is because we haven’t had anyone who both used both on a regular basis and was willing to be responsible for keeping things in sync.

#9

Besides @BjarniRunar’s questions and concerns, I have looked a bit further than the first glance I had with my previous reply.

Two follow-up questions:

  1. From a utter personal attitude, I’d always recommend hardening the webserver config. I have zero experience with Docker, but from what I gather, the Apache httpd vhost is generated here. Especially when publicly exposed or when running a multi-user instance, it would be best to only allow TLSv1.2, use solid PFS ciphers and add some headers that benefit security. Are you open to that? And if so, would you prefer I submit them via a pull request? I am AFK the rest of the week, but could do that once I return from my trip.
  2. Is there a benefit from using Apache over, eg, nginx? Apache seems to be a tidbit more resource intensive, hence the question. Coming purely from personal curiousity.

These questions are not - in any way - meant as critique. I really appreciate your effort; I am merely hoping I can contribute something - anything at all.

#10

I am just testing Mailpile with my gmail account (maybe later I will switch to my own mail server, but I am not there yet). My intention was to use it regularly, but for the time being I am too much familiar with the automatic categorization that gmail web-ui does to the incoming messages (Primary, Social, Promotions, Updates, Forums, etc.) that I find a bit overwhelming to switch to a client that has all the messages in one place.

Maybe with time I may find ways to overcome this obstacle, but for the time being I have decided to keep using the gmail web-ui as usual, and to use Mailpile when I need encrypted or signed communication. This reminds me that I need to add some scripts for easy management of the private key (import to the GNUPGHOME used by Mailpile an existing private key, export it, etc.)

Keeping the Docker scripts up to date seems an easy task to me. Maybe you have noticed that they are quite lean and there is no complex magic in them. The only time when they should need any update is when we decide to switch to another version of Debian (which may come with another version of Apache, for example), or when the Mailpile scripts itself (mailpile-apache2) introduce some major changes on their interface. In general, it should be trivial to keep things in sync and up to date.

Import of existing GPG identities
#11

I am not an expert on Apache configuration, and certainly not an expert on its security options. It seems that you know them better than me. I would certainly appreciate a pull request, if this improves the security and it does not break anything.

However, the relevant configuration is on these two places:

  • https://gitlab.com/docker-scripts/wsproxy/blob/master/misc/apache2.conf
  • https://gitlab.com/docker-scripts/mailpile/blob/master/scripts/apache2.sh

I admit that it is not so easy to find the relevant configurations, unless you have an overall picture of what is going on.

Initially I used Apache because I was a bit more familiar with it, although I was aware that nginx had a better performance (due to using threads for each request, instead of different processes). However I think that Apache later closed that gap, and now it also uses by default threads for each request, instead of processes. However I am not so sure on this. Maybe nginx still has a better performance (and an easier configuration). My attitude on this is: Since it is working, why fix it?

However, if someone is quite familiar with nginx and would like to give a try replacing Apache with nginx, I am open to it.

I also appreciate your interest and I would appreciate any contribution that you may make, even if it is small.

#12

Thank you for your elaborate reply! Some PRs are now on my todo.txt :slight_smile: Expect 'em somewhere next week!

#13

Just to quickly chime in here: Mailpile’s multi-user support (such as it is, it’s not great) is currently Apache-only. We don’t have a maipile-nginx package yet, although it would be fantastic if we did.

Regarding hardening things: the most important thing is that you always use TLS for connecting to your Mailpile. If you’re doing that, you’re 95% of the way there. The rest may not be worth complicating things for. IMO, obviously.

Regarding moving from GMail to a single inbox - you can configure Mailpile to auto-tag things (reusing the built-in Bayes classifier we use for spam, see: https://www.mailpile.is/blog/2014-01-12_A_Plan_For_Spam.html), and after a bit of training it can do a decent job moving notifications and promotions and such things into separate “inboxes”. But yes, there is definitely a learning curve, and you will have to train it before it does a good job.

#14

From what I have seen (in the context of using Mailpile in the Docker container), the package mailpile-apache does not do much, except for generating a decent apache2 configuration, which I have to override anyway. So, I think that it should not be very difficult to replace apache2 by nginx, if someone is familiar with nginx configuration (rewriting, forwarding, etc.) However I don’t see why nginx should be better than apache2, especially since this is not supposed to be a site with a high traffic (just a couple of users, or maybe dozens).

I agree. I usually trust the default apache2 configurations made by debian/ubuntu packages, although I don’t understand all the details. Going with the defaults make easy the upgrade as well, when a new version of debian/ubuntu is out.

I will have to look into this. But I didn’t have to train my Gmail inbox. Maybe it is possible to have a default configuration that works well out of the box. Maybe it is just a set of filtering rules, instead of machine learning. Or maybe it is a combination of filtering rules and ML rules. The advantage of Gmail is that they look at the emails of too many people, so ML and training makes sense and works well. With a small input ML may not work so well, unless it is possible to share somehow the “experiences” between different installations.

#15

The mailpile-apache package handles mapping users to ports (you can’t have two Mailpiles on the same port), launching Mailpile on demand if it’s down (and launching it with flags telling it to shut down if it’s been idle for a while), and provides admin tools for adding/removing users. The launcher has some bugs, and the docs need work, but if you’ve thrown all that out the window then odds are high that you’ve missed something important. :man_shrugging:

I agree Apache is more than fast enough for a normal Mailpile setup. The main reason for wanting nginx, is because it’s trendy and there’s user demand (edit: and people may already be using it for other things, so coexisting would be nice).

#16

Oh, one more thing… if your Docker is intended to be multi-user then it would be really, really, really great if you would take the time to get it working with mailpile-apache. It is not good for our ecosystem if everyone is doing things in a different way, it makes it impossible to share knowledge and it takes much longer to iron out the bugs in what we do have.

#17

Docs: https://github.com/mailpile/Mailpile/tree/master/shared-data/multipile

#18

I certainly do use it. But all this is not apache specific, but multi-user specific. There is only one apache config file that relates the all package to apache. I think that the package name mailpile-apache is misleading and not correct. A better name might be mailpile-multiuser. This package may support (in the future) both an apache config and an nginx config, and leave it up to the users to choose which server to use.

It does work with mailpile-apache. The only thing that I did not find suitable is that with the stock configuration the user mailpile accounts have to be accessed like this: https://mailpile.example.org/mailpile/username/.... I have fixed it to be accessed like https://mailpile.example.org/username/...:

https://gitlab.com/docker-scripts/mailpile/blob/master/scripts/apache2.sh

#19

P.s. Your community/forum filters are classifying me as spammer and are not allowing me to post any links. This is the same everywhere. I hate discourse for this, they are ruining my life.

#20

Why is the extra /mailpile/ a problem? You’ve just added complexity and created a difference between your setup, and the documented one, to save a few bytes in an URL. :grin:

The mailpile-apache package is called mailpile-apache because it integrates with Apache. And it certainly does contain logic that is Apache specific, in particular some of the scripts that run on package install. When we have support for more webservers, either we’ll rename or each one will get its own package. Time will tell!

Regarding the links and spam, sorry about that. I’ve approved your posts… hopefully Discourse will learn that you’re a good guy and stop? I’m not a Discourse expert…