Improve system documentation
[cacert-infradocs.git] / docs / systems / email.rst
1 .. index::
2 single: Systems; Email
3
4 =====
5 Email
6 =====
7
8 Purpose
9 =======
10
11 This system handles email for @cacert.org addresses. It also provides users of
12 @cacert.org with IMAPs and POP3s access to their accounts.
13
14 The database on this container is used by :doc:`webmail` too.
15
16 Administration
17 ==============
18
19 System Administration
20 ---------------------
21
22 * Primary: :ref:`people_jselzer`
23 * Secondary: :ref:`people_jandd`
24
25 Contact
26 -------
27
28 * email-admin@cacert.org
29
30 Additional People
31 -----------------
32
33 :ref:`people_mario` has :program:`sudo` access on that machine too.
34
35 Basics
36 ======
37
38 Physical Location
39 -----------------
40
41 This system is located in an :term:`LXC` container on physical machine
42 :doc:`infra02`.
43
44 Logical Location
45 ----------------
46
47 :IP Internet: :ip:v4:`213.154.225.228`
48 :IP Intranet: :ip:v4:`172.16.2.19`
49 :IP Internal: :ip:v4:`10.0.0.19`
50 :IPv6: :ip:v6:`2001:7b8:616:162:2::19`
51 :MAC address: :mac:`00:ff:8f:e0:4a:90` (eth0)
52
53 .. seealso::
54
55 See :doc:`../network`
56
57 .. index::
58 single: Monitoring; Email
59
60 Monitoring
61 ----------
62
63 :internal checks: :monitor:`email.infra.cacert.org`
64
65 DNS
66 ---
67
68 .. index::
69 single: DNS records; Email
70
71 ======================= ======== ============================================
72 Name Type Content
73 ======================= ======== ============================================
74 email.cacert.org. IN A 213.154.225.228
75 email.cacert.org. IN SSHFP 1 1 BF391FD72656A275524D1D25A624C6045B44AE90
76 email.cacert.org. IN SSHFP 2 1 73B0D8ACB492A7187016DD3C5FC1519B309A550F
77 email.intra.cacert.org. IN A 172.16.2.19
78 ======================= ======== ============================================
79
80 A DKIM record for cacert.org ist setup but no DKIM signing is active currently.
81
82 .. todo:: setup DKIM properly, see :bug:`696` for an older discussion
83
84 .. todo:: setup SPF records when the system is ready, see :bug:`492` for an
85 older discussion
86
87 .. seealso::
88
89 See :wiki:`SystemAdministration/Procedures/DNSChanges`
90
91 Operating System
92 ----------------
93
94 .. index::
95 single: Debian GNU/Linux; Lenny
96 single: Debian GNU/Linux; 5.0.10
97
98 * Debian GNU/Linux 5.0.10
99
100 Applicable Documentation
101 ------------------------
102
103 This is it :-)
104
105 Services
106 ========
107
108 Listening services
109 ------------------
110
111 +----------+---------+----------------+----------------------------------------+
112 | Port | Service | Origin | Purpose |
113 +==========+=========+================+========================================+
114 | 22/tcp | ssh | ANY | admin console access |
115 +----------+---------+----------------+----------------------------------------+
116 | 25/tcp | smtp | ANY | mail receiver for cacert.org |
117 +----------+---------+----------------+----------------------------------------+
118 | 110/tcp | pop3 | ANY | POP3 access for cacert.org mail |
119 | | | | addresses |
120 +----------+---------+----------------+----------------------------------------+
121 | 143/tcp | imap | ANY | IMAP access for cacert.org mail |
122 | | | | addresses |
123 +----------+---------+----------------+----------------------------------------+
124 | 465/tcp | smtps | ANY | SMTPS for cacert.org mail addresses |
125 +----------+---------+----------------+----------------------------------------+
126 | 587/tcp | smtp | ANY | mail submission for cacert.org mail |
127 | | | | addresses |
128 +----------+---------+----------------+----------------------------------------+
129 | 993/tcp | imaps | ANY | IMAPS access for cacert.org mail |
130 | | | | addresses |
131 +----------+---------+----------------+----------------------------------------+
132 | 995/tcp | pop3s | ANY | POP3S access for cacert.org mail |
133 | | | | addresses |
134 +----------+---------+----------------+----------------------------------------+
135 | 2000/tcp | sieve | ANY | Manage sieve access for cacert.org |
136 | | | | mail addresses |
137 +----------+---------+----------------+----------------------------------------+
138 | 2001/tcp | sieve | :doc:`webmail` | Manage sieve access for cacert.org |
139 | | | | mail addresses without TLS, accessible |
140 | | | | from ``172.16.2.20`` only |
141 +----------+---------+----------------+----------------------------------------+
142 | 3306/tcp | mysql | local | MySQL database server |
143 +----------+---------+----------------+----------------------------------------+
144 | 4433/tcp | http | local | Apache httpd with phpmyadmin |
145 +----------+---------+----------------+----------------------------------------+
146 | 5666/tcp | nrpe | monitor | remote monitoring service |
147 +----------+---------+----------------+----------------------------------------+
148
149 .. topic:: PHPMyAdmin access
150
151 Administrators can use ssh to forward the Apache httpd HTTPS port to their
152 own machine:
153
154 .. code-block:: bash
155
156 ssh -L 4433:localhost:4433 -l username email.cacert.org
157
158 and access PHPMyAdmin at https://localhost:4433/
159
160 Running services
161 ----------------
162
163 .. index::
164 single: apache httpd
165 single: cron
166 single: dovecot
167 single: mysql
168 single: nrpe
169 single: openssh
170 single: postfix
171 single: pysieved
172 single: rsyslog
173 single: xinetd
174
175 +--------------------+---------------------+----------------------------------------+
176 | Service | Usage | Start mechanism |
177 +====================+=====================+========================================+
178 | Apache httpd | Webserver for | init script |
179 | | phpmyadmin | :file:`/etc/init.d/apache2` |
180 +--------------------+---------------------+----------------------------------------+
181 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
182 +--------------------+---------------------+----------------------------------------+
183 | dovecot | IMAP(s) and POP3(s) | init script |
184 | | daemon | :file:`/etc/init.d/dovecot` |
185 +--------------------+---------------------+----------------------------------------+
186 | MySQL | MySQL database | init script |
187 | | server for email | :file:`/etc/init.d/mysql` |
188 | | services | |
189 +--------------------+---------------------+----------------------------------------+
190 | Nagios NRPE server | remote monitoring | init script |
191 | | service queried by | :file:`/etc/init.d/nagios-nrpe-server` |
192 | | :doc:`monitor` | |
193 +--------------------+---------------------+----------------------------------------+
194 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
195 | | remote | |
196 | | administration | |
197 +--------------------+---------------------+----------------------------------------+
198 | Postfix | SMTP server for | init script |
199 | | cacert.org | :file:`/etc/init.d/postfix` |
200 +--------------------+---------------------+----------------------------------------+
201 | rsyslog | syslog daemon | init script |
202 | | | :file:`/etc/init.d/syslog` |
203 +--------------------+---------------------+----------------------------------------+
204 | xinetd | socket listener | init script |
205 | | for pysieved | :file:`/etc/init.d/xinetd` |
206 +--------------------+---------------------+----------------------------------------+
207
208 Databases
209 ---------
210
211 +-------+----------------+----------------------------------+
212 | RDBMS | Name | Used for |
213 +=======+================+==================================+
214 | MySQL | cacertusers | database for dovecot and postfix |
215 +-------+----------------+----------------------------------+
216 | MySQL | postfixpolicyd | empty database |
217 +-------+----------------+----------------------------------+
218 | MySQL | roundcubemail | roundcube on :doc:`webmail` |
219 +-------+----------------+----------------------------------+
220
221 .. todo:: check whether the empty postfixpolicyd database is required
222
223 Connected Systems
224 -----------------
225
226 * :doc:`monitor`
227 * :doc:`webmail`
228 * all @cacert.org address owners have access to POP3 (STARTTLS and POP3S), IMAP
229 (STARTTLS and IMAPS), SMTPS, SMTP submission (STARTTLS) and manage sieve
230
231 Outbound network connections
232 ----------------------------
233
234 * DNS (53) resolving nameservers 172.16.2.2 and 172.16.2.3
235 * :doc:`proxyout` as HTTP proxy for APT
236 * :doc:`issue` for OTRS mail
237 * :doc:`lists` for mailing lists
238 * arbitrary Internet SMTP servers for outgoing mail
239
240 Security
241 ========
242
243 .. sshkeys::
244 :RSA: SHA256:yLaPPrmoOQI5G3hoa0iFoxf6wPdLBJCnizLsu+6SHfE MD5:a1:d2:17:53:6b:0f:b6:a4:14:13:46:f7:04:ef:4a:23
245 :DSA: SHA256:zY4YEmiCYrbDXK1FHum9Qw8cKAInnizrbODF8o2ofEU MD5:f4:eb:0a:36:40:1c:55:6b:75:a2:26:34:ea:18:7e:91
246
247 .. warning::
248
249 The system is too old to support ECDSA or ED25519 keys.
250
251 Non-distribution packages and modifications
252 -------------------------------------------
253
254 Tlslite in :file:`/usr/local/lib/tlslite-0.3.8/` has been patched to handle
255 GeneratorExit exceptions. The original tlslite 0.3.8 is stored in
256 :file:`/usr/local/lib/tlslite-0.3.8-orig/`.
257
258 Pysieved in :file:`/usr/local/lib/pysieved.neale/` seems to be a git clone from
259 2009 originating from http://woozle.org/~neale/repos/pysieved at commit
260 ``d9b67036387a9a7aca954a17ff6fec44a8d309e0`` with no local modifications.
261
262 :file:`/usr/local/lib/pysieved` is a symbolic link to
263 :file:`/usr/local/lib/pysieved.neale/`.
264
265 .. todo:: use pysieved, python-tlslite and dovecot-sieve from distribution
266 packages after OS upgrade
267
268
269 Risk assessments on critical packages
270 -------------------------------------
271
272 The whole system is outdated, it needs to be replaced as soon as possible.
273
274 Critical Configuration items
275 ============================
276
277 Keys and X.509 certificates
278 ---------------------------
279
280 Server certificate for SMTP communication from the Internet and PHPMyAdmin.
281
282 .. sslcert:: email.cacert.org
283 :altnames: DNS:email.cacert.org
284 :certfile: /etc/ssl/certs/ssl-cert-email-cacert.pem
285 :keyfile: /etc/ssl/private/ssl-cert-email-cacert.key
286 :serial: 1381FA
287 :expiration: Mar 16 11:23:55 2020 GMT
288 :sha1fp: 3A:EC:11:D0:78:6C:99:34:F2:45:A5:DF:08:90:94:1F:67:2C:6F:47
289 :issuer: CA Cert Signing Authority
290
291 Server certificate for community email services (SMTPS, SMTP submission in
292 Postfix and IMAP with STARTTLS, IMAPS, POP3 with STARTTLS, POP3S and pysieved)
293
294 .. sslcert:: community.cacert.org
295 :certfile: /etc/ssl/certs/ssl-cert-community-cacert.pem
296 :keyfile: /etc/ssl/private/ssl-cert-community-cacert.key
297 :serial: 1381F8
298 :secondary:
299
300 * :file:`/etc/postfix/dh_1024.pem` and :file:`/etc/postfix/dh_512.pem`
301 Diffie-Hellman parameter files for Postfix
302
303 .. note::
304
305 Postfix uses the email.cacert.org certificate for client authentication if
306 requested by a target server.
307
308 .. todo::
309 check whether it makes sense to use a separate certificate for that
310 purpose
311
312 .. seealso::
313
314 * :wiki:`SystemAdministration/CertificateList`
315
316 .. index::
317 pair: Apache httpd; configuration
318
319 Apache httpd configuration
320 --------------------------
321
322 :file:`/etc/apache2/sites-available/adminssl` configures a VirtualHost that
323 allows dedicated users to access a PHPMyAdmin instance. The allowed users are
324 authenticated by client certificates and are authorized by an entry in
325 :file:`/etc/apache2/phpmyadmin.passwd`.
326
327 .. note::
328
329 to authorize a user you need the subject distinguished name of the user's
330 client certificate which can be extracted with::
331
332 openssl x509 -noout -subject -in certificate.crt
333
334 A line with the subject distinguished name and the fake password
335 ``xxj31ZMTZzkVA`` separated by colon have to be added to
336 :file:`/etc/apache2/phpmyadmin.passwd`::
337
338 /CN=Example User/emailAddress=example@cacert.org:xxj31ZMTZzkVA
339
340 .. seealso::
341
342 FakeBasicAuth option of the `SSLOptions
343 <https://httpd.apache.org/docs/2.2/mod/mod_ssl.html#ssloptions>`_
344 directive in the mod_ssl reference documentation.
345
346 .. index::
347 pair: MySQL; configuration
348
349 MySQL configuration
350 -------------------
351
352 MySQL configuration is stored in the :file:`/etc/mysql/` directory.
353
354 .. index::
355 pair: MySQL; NSS
356 single: libnss-mysql
357
358 .. _nss:
359
360 NSS configuration
361 -----------------
362
363 The libc name service switch is configured to use MySQL lookups for passwd,
364 group and shadow via :file:`/etc/nsswitch.conf`. The queries are configured in
365 :file:`/etc/libnss-mysql.cfg` and the root user for reading shadow information
366 is configured in :file:`/etc/libnss-mysql-root.cfg`.
367
368 .. index::
369 pair: PHPMyAdmin; configuration
370
371 PHPMyAdmin configuration
372 ------------------------
373
374 PHPMyAdmin configuration is stored in the :file:`/etc/phpmyadmin/` directory.
375
376 .. index::
377 pair: dovecot; configuration
378
379 Dovecot configuration
380 ---------------------
381
382 Dovecot configuration is stored in the :file:`/etc/dovecot/` directory. The
383 database settings are stored in
384 :file:`dovecot-sql-masterpassword-webmail.conf`.
385
386 .. index::
387 pair: dovecot; authentication
388
389 .. topic:: Dovecot authentication
390
391 :file:`/etc/dovecot/dovecot.conf` refers to PAM mail. PAM mail is defined
392 :file:`/etc/pam.d/mail`. System users are defined by NSS which is a
393 combination of :file:`/etc/passwd` (for root and non-imap/pop users) and
394 :file:`/etc/libnss-mysql*` (see `nss`_).
395
396 There is a special master password so that webmail can do the authentication
397 for dovecot using certificates. This is defined in
398 :file:`/etc/dovecot/dovecot-sql-masterpassword-webmail.conf`. This special
399 password is restricted to the IP address of Community.
400
401 .. index::
402 pair: Postfix; configuration
403
404 Postfix configuration
405 ---------------------
406
407 Postfix configuration is stored in the :file:`/etc/postfix/` directory. The
408 following files are special for this setup:
409
410 +----------------+-------------------------------------------------------------+
411 | File | Used for |
412 +================+=============================================================+
413 | arbitration | rewrite recipients matching specific regular expressions to |
414 | | support+deletedaccounts@cacert.org and |
415 | | support@issue.cacert.org |
416 +----------------+-------------------------------------------------------------+
417 | cacert-inc-bcc | used as recipient_bcc_maps for specific functional mail |
418 | | addresses |
419 +----------------+-------------------------------------------------------------+
420 | main.cf | the main configuration file |
421 +----------------+-------------------------------------------------------------+
422 | master.cf | adds configuration for the community SMTPS and SMTP |
423 | | submission transports |
424 +----------------+-------------------------------------------------------------+
425 | mysql-\*.cf | configuration of several MySQL queries for alias mapping, |
426 | | Postfix operates on views for the user table |
427 +----------------+-------------------------------------------------------------+
428 | transport | forward email for lists.cacert.org to :doc:`lists` and for |
429 | | issue.cacert.org to :doc:`issue` |
430 +----------------+-------------------------------------------------------------+
431
432 .. todo:: consider to send all outgoing mail via :doc:`emailout`
433
434 .. todo:: remove unused transports from :file:`master.cf`
435
436 .. index::
437 pair: pysieved; configuration
438
439 PySieved configuration
440 ----------------------
441
442 :file:`/usr/local/etc/pysieved.ini` for regular manage sieve access and
443 :file:`/usr/local/etc/pysieved-notls.ini` for use with Roundcube webmail.
444 Pysieved uses dovecot for authentication.
445
446 .. index::
447 pair: rsyslog; configuration
448
449 Rsyslog configuration
450 ---------------------
451
452 Rsyslog is configured in :file:`/etc/rsyslog.conf` which includes files in
453 :file:`/etc/rsyslog.d/`. Consumption of kernel log messages and network input
454 is disabled. :file:`/etc/rsyslog.d/postfix.conf` configures a separate unix
455 socket to receive log messages from postfix and
456 :file:`/etc/rsyslog.d/remotelog.conf` contains commented settings for a
457 non-existant remote syslog server.
458
459 .. todo:: setup remote logging when a central logging container is available
460
461 .. index::
462 pair: xinetd; configuration
463
464 Xinetd configuration
465 --------------------
466
467 Xinetd listens on tcp ports 2000 and 2001 and spawn pysieved. Configuration for
468 these listeners is stored in :file:`/etc/xinetd.d/pysieved` and
469 :file:`/etc/xinetd.d/pysieved-notls`.
470
471 Email storage
472 -------------
473
474 Mail for :samp:`{user}` is stored in :samp:`/home/{user}/Maildir`.
475
476 .. todo::
477 move mail storage to a separate data volume to allow easier backup and OS
478 upgrades
479
480 Tasks
481 =====
482
483 .. index::
484 single: add email users
485
486 Adding email users
487 ------------------
488
489 1. create user in the database table ``cacertusers.user``:
490
491 .. code-block:: bash
492
493 mysql -p cacertusers
494
495 .. code-block:: sql
496
497 INSERT INTO user (username, fullnamealias, realname, password)
498 VALUES ('user', 'user.name', 'User Name', '$1$salt$passwordhash')
499
500 2. create the user's home directory and Maildir:
501
502 :samp:`install -o {user} -g {user} -m 0755 -d /home/{user}/Maildir`
503
504 .. note::
505
506 * a valid password hash for the password ``secret`` is
507 ``$1$caea3837$gPafod/Do/8Jj5M9HehhM.``
508 * users can reset their password via
509 https://community.cacert.org/password.php on :doc:`webmail`
510 * use the :download:`mail template
511 <../downloads/template_new_community_mailaddress.rfc822>` to send out to a
512 user's non-cacert.org mail account and make sure to encrypt the mail to a
513 known public key of that user
514
515 .. todo::
516 implement tooling to automate password salt generation and user creation
517
518 Setting up mail aliases
519 -----------------------
520
521 There are two types of aliases.
522
523 1. The first type are those that are never sent from. e.g.
524 postmaster@cacert.org. All these aliases are defined in
525 :file:`/etc/aliases`. Don't forget to run
526
527 .. code-block:: bash
528
529 postalias /etc/aliases
530
531 after any changes. Aliases for issue tracking are installed here as
532 :samp:`{issuetrackingaddress} : {issuetrackingaddress}@issue.cacert.org`.
533
534 2. The second type are those aliases that are used to send email too, e.g
535 pr@cacert.org. These aliases are recorded in the aliases table on the
536 cacertusers database. The reason for this implementation is to only allow
537 the designated person to send email from this email address.
538
539 Client certificate authentication
540 ---------------------------------
541
542 There were plans for X.509 certificate authentication for mail services, but
543 there is no progress so far.
544
545 Changes
546 =======
547
548 Planned
549 -------
550
551 .. todo:: switch to Puppet management
552 .. todo:: replace nrpe with icinga2 agent
553 .. todo:: update to Debian 6/7/8/9/10
554 .. todo:: implement CRL checking
555 .. todo:: setup IPv6
556
557 .. todo::
558 throttle brute force attack attempts using fail2ban or similar mechanism
559
560 .. todo::
561 consider to use LDAP to consolidate user, password and email information
562
563 System Future
564 -------------
565
566 .. todo::
567 The system has to be replaced with a new system using a current operating
568 system version
569
570 Additional documentation
571 ========================
572
573 .. seealso::
574
575 * :wiki:`PostfixConfiguration`
576 * :wiki:`SystemAdministration/Systems/Email` for some discussion on legal
577 implications related to mail archiving
578
579 References
580 ----------
581
582 Postfix documentation
583 http://www.postfix.org/documentation.html
584 Postfix Debian wiki page
585 https://wiki.debian.org/Postfix
586 Dovecot 1.x wiki
587 http://wiki1.dovecot.org/FrontPage