Improve system documentation
[cacert-infradocs.git] / docs / systems / translations.rst
1 .. index::
2 single: Systems; Translations
3
4 ============
5 Translations
6 ============
7
8 Purpose
9 =======
10
11 This system runs a `Pootle`_ translation server.
12
13 .. _Pootle: http://pootle.translatehouse.org/
14
15
16 Application Links
17 -----------------
18
19 Pootle web interface
20 https://translations.cacert.org/
21
22 Administration
23 ==============
24
25 System Administration
26 ---------------------
27
28 * Primary: :ref:`people_jandd`
29 * Secondary: None
30
31 .. todo:: find an additional admin
32
33 Application Administration
34 --------------------------
35
36 +-------------+---------------------+
37 | Application | Administrator(s) |
38 +=============+=====================+
39 | Pootle | :ref:`people_jandd` |
40 +-------------+---------------------+
41
42 Contact
43 -------
44
45 * translations-admin@cacert.org
46
47 Additional People
48 -----------------
49
50 :ref:`people_mario` has :program:`sudo` access on that machine too.
51
52 Basics
53 ======
54
55 Physical Location
56 -----------------
57
58 This system is located in an :term:`LXC` container on physical machine
59 :doc:`infra02`.
60
61 Logical Location
62 ----------------
63
64 :IP Internet: :ip:v4:`213.154.225.240`
65 :IP Intranet: :ip:v4:`172.16.2.31`
66 :IP Internal: :ip:v4:`10.0.0.31`
67 :IPv6: :ip:v6:`2001:7b8:616:162:2::31`
68 :MAC address: :mac:`00:ff:6c:7d:5b:c5` (eth0)
69
70 .. seealso::
71
72 See :doc:`../network`
73
74 .. index::
75 single: Monitoring; Translations
76
77 Monitoring
78 ----------
79
80 :internal checks: :monitor:`translations.infra.cacert.org`
81
82 DNS
83 ---
84
85 .. index::
86 single: DNS records; Translations
87
88 ============================== ======== ====================================================================
89 Name Type Content
90 ============================== ======== ====================================================================
91 l10n.cacert.org. IN CNAME translations.cacert.org.
92 translations.cacert.org. IN A 213.154.225.240
93 translations.cacert.org. IN AAAA 2001:7b8:616:162:2::31
94 translations.cacert.org. IN SSHFP 1 1 1128972FB54F927477A781718E2F9C114E9CA383
95 translations.cacert.org. IN SSHFP 1 2 F223904069AEAA2E0EAC5D9092AB7DEBAE70F06EC3C25E94F49F1B15F633ED5D
96 translations.cacert.org. IN SSHFP 2 1 3A36E5DF06304C481F01FC723FD88A086E82D986
97 translations.cacert.org. IN SSHFP 2 2 4A1FF7396AE874559CF196D54D5D7F6890DBA6DE73B46AF049258B1024CDACE2
98 translations.cacert.org. IN SSHFP 3 1 0F0CBD9C188D619D743859A249238F684D6CCA5F
99 translations.cacert.org. IN SSHFP 3 2 441D76EB651022A8C5810C6946CBDEC47504E97AD669B073EC9D6E27791A7C4D
100 translations.cacert.org. IN SSHFP 4 1 A4102E1FBF1BE1ACD53F2E7653DD8898E567C437
101 translations.cacert.org. IN SSHFP 4 2 6FE3334B51E68F9F650B00D13F504306029B71A76C5AFF54873D72B24ED19DD5
102 translations.intra.cacert.org. IN A 172.16.2.31
103 ============================== ======== ====================================================================
104
105 .. seealso::
106
107 See :wiki:`SystemAdministration/Procedures/DNSChanges`
108
109 Operating System
110 ----------------
111
112 .. index::
113 single: Debian GNU/Linux; Stretch
114 single: Debian GNU/Linux; 9.4
115
116 * Debian GNU/Linux 9.4
117
118 Applicable Documentation
119 ------------------------
120
121 This is it :-)
122
123 Services
124 ========
125
126 Listening services
127 ------------------
128
129 +----------+---------+---------+----------------------------+
130 | Port | Service | Origin | Purpose |
131 +==========+=========+=========+============================+
132 | 22/tcp | ssh | ANY | admin console access |
133 +----------+---------+---------+----------------------------+
134 | 25/tcp | smtp | local | mail delivery to local MTA |
135 +----------+---------+---------+----------------------------+
136 | 80/tcp | http | ANY | redirect to https |
137 +----------+---------+---------+----------------------------+
138 | 443/tcp | https | ANY | application |
139 +----------+---------+---------+----------------------------+
140 | 3306/tcp | mysql | local | MySQL database for Pootle |
141 +----------+---------+---------+----------------------------+
142 | 5666/tcp | nrpe | monitor | remote monitoring service |
143 +----------+---------+---------+----------------------------+
144 | 6379/tcp | redis | local | Redis in memory cache |
145 +----------+---------+---------+----------------------------+
146
147 Running services
148 ----------------
149
150 .. index::
151 single: apache httpd
152 single: cron
153 single: mariadb
154 single: nrpe
155 single: openssh
156 single: postfix
157 single: puppet agent
158 single: redis
159 single: rsyslog
160 single: supervisord
161
162 +--------------------+------------------------------+-----------------------------------------------------+
163 | Service | Usage | Start mechanism |
164 +====================+==============================+=====================================================+
165 | Apache httpd | Webserver for | init script |
166 | | Pootle | :file:`/etc/init.d/apache2` |
167 +--------------------+------------------------------+-----------------------------------------------------+
168 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
169 +--------------------+------------------------------+-----------------------------------------------------+
170 | MySQL | MySQL database | init script |
171 | | server for Pootle | :file:`/etc/init.d/mysql` |
172 +--------------------+------------------------------+-----------------------------------------------------+
173 | Postfix | SMTP server for | init script |
174 | | local mail | :file:`/etc/init.d/postfix` |
175 | | submission | |
176 +--------------------+------------------------------+-----------------------------------------------------+
177 | Puppet agent | local Puppet agent | init script |
178 | | | :file:`/etc/init.d/puppet` |
179 +--------------------+------------------------------+-----------------------------------------------------+
180 | Nagios NRPE server | remote monitoring | init script |
181 | | service queried by | :file:`/etc/init.d/nagios-nrpe-server` |
182 | | :doc:`monitor` | |
183 +--------------------+------------------------------+-----------------------------------------------------+
184 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
185 | | remote | |
186 | | administration | |
187 +--------------------+------------------------------+-----------------------------------------------------+
188 | Redis | Job queue for Pootle | init script :file:`/etc/init.d/redis-server` |
189 +--------------------+------------------------------+-----------------------------------------------------+
190 | rsyslog | syslog daemon | init script |
191 | | | :file:`/etc/init.d/syslog` |
192 +--------------------+------------------------------+-----------------------------------------------------+
193 | Supervisord | Supervisor for background | init script :file:`/etc/init.d/supervisor` |
194 | | tasks | |
195 +--------------------+------------------------------+-----------------------------------------------------+
196 | Pootle rqworker | Worker for Pootle background | supervisor task in |
197 | | tasks | :file:`/etc/supervisor/conf.d/pootle-rqworker.conf` |
198 +--------------------+------------------------------+-----------------------------------------------------+
199
200 Databases
201 ---------
202
203 +-------+--------+----------+
204 | RDBMS | Name | Used for |
205 +=======+========+==========+
206 | MySQL | pootle | Pootle |
207 +-------+--------+----------+
208
209 Connected Systems
210 -----------------
211
212 * :doc:`monitor`
213
214 Outbound network connections
215 ----------------------------
216
217 * :doc:`infra02` as resolving nameserver
218 * :doc:`emailout` as SMTP relay
219 * :doc:`puppet` (tcp/8140) as Puppet master
220 * :doc:`proxyout` as HTTP proxy for APT
221 * arbitrary Internet HTTP, HTTPS, FTP, FTPS, git servers for fetching Pootle
222 dependencies (via ``&CONTAINER_OUT_ELEVATED("translations");`` in
223 :file:`/etc/ferm/ferm.d/translations.conf` on :doc:`infra02`).
224
225 Security
226 ========
227
228 .. sshkeys::
229 :RSA: SHA256:8iOQQGmuqi4OrF2Qkqt9665w8G7Dwl6U9J8bFfYz7V0 MD5:df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
230 :DSA: SHA256:Sh/3OWrodFWc8ZbVTV1/aJDbpt5ztGrwSSWLECTNrOI MD5:07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
231 :ECDSA: SHA256:RB1262UQIqjFgQxpRsvexHUE6XrWabBz7J1uJ3kafE0 MD5:0a:39:d9:22:39:3a:48:5d:fb:a3:27:15:d9:30:a8:64
232 :ED25519: SHA256:b+MzS1Hmj59lCwDRP1BDBgKbcadsWv9Uhz1ysk7RndU MD5:ca:a6:93:70:8c:38:23:26:16:68:5b:87:16:ee:70:17
233
234 Dedicated user roles
235 --------------------
236
237 +---------------+----------------------------------+
238 | Group | Purpose |
239 +===============+==================================+
240 | pootle-update | Planned translation update group |
241 +---------------+----------------------------------+
242
243 Non-distribution packages and modifications
244 -------------------------------------------
245
246 Pootle is a Python/Django application that has been installed in a Python
247 virtualenv. Pootle and all its dependencies have been installed using:
248
249 .. code-block:: bash
250
251 cd /var/www/pootle
252 virtualenv pootle-2.8.2
253 ln -s pootle-2.8.2 current
254 chown -R pootle.www-data pootle-2.8.2
255 sudo -s -u pootle
256 . pootle-2.8.2/bin/activate
257 pip install --process-dependency-links Pootle[mysql]
258 pootle migrate
259
260 Pootle is installed in a versioned directory. The used version is a symlink in
261 :file:`/var/www/pootle/current`. The rationale is to avoid changes to many
262 different configuration files when updating to a newer Pootle version.
263
264 The installation needs an installed :program:`gcc` and a few library development
265 packages.
266
267 .. todo::
268
269 consider building the virtualenv on :doc:`jenkins` to avoid development tools
270 on this system
271
272 The Puppet agent package and a few dependencies are installed from the official
273 Puppet APT repository because the versions in Debian are too old to use modern
274 Puppet features.
275
276 Risk assessments on critical packages
277 -------------------------------------
278
279 System access is limited to http/https via Apache httpd which is restricted to
280 a minimal set of modules.
281
282 The system uses third party packages with a good security track record and
283 regular updates. The attack surface is small due to the tightly restricted
284 access to the system. The puppet agent is not exposed for access from outside
285 the system.
286
287 Pootle is based on Django 1.10 and should be updated to a newer version when it
288 becomes available. Pootle is run as a dedicated system user `pootle` that is
289 restricted via filesystem permissions.
290
291 The following change has been made to the translation toolkit filters that are
292 used by Pootle in :file:`/var/www/pootle/pootle-2.8.2/lib/python2.7/site-packages/translate/filters/checks.py`
293 to add CAcert specific translation checks:
294
295 .. code-block:: diff
296
297 commit 4d107e5019f4794b4581cadaf4e9a8339868f6a4
298 Author: Jan Dittberner <jandd@cacert.org>
299 Date: Fri Feb 23 20:39:03 2018 +0000
300
301 Add CAcert checkers
302
303 Signed-off-by: Jan Dittberner <jandd@cacert.org>
304
305 diff --git a/filters/checks.py b/filters/checks.py
306 index db10937..45b464c 100644
307 --- a/filters/checks.py
308 +++ b/filters/checks.py
309 @@ -2475,6 +2475,24 @@ class IOSChecker(StandardChecker):
310 StandardChecker.__init__(self, **kwargs)
311
312
313 +cacertconfig = CheckerConfig(
314 + notranslatewords = ["CAcert", "Assurer"],
315 + criticaltests = ["printf"],
316 +)
317 +
318 +
319 +class CAcertChecker(StandardChecker):
320 +
321 + def __init__(self, **kwargs):
322 + checkerconfig = kwargs.get("checkerconfig", None)
323 + if checkerconfig is None:
324 + checkerconfig = CheckerConfig()
325 + kwargs["checkerconfig"] = checkerconfig
326 +
327 + checkerconfig.update(cacertconfig)
328 + StandardChecker.__init__(self, **kwargs)
329 +
330 +
331 projectcheckers = {
332 "minimal": MinimalChecker,
333 "standard": StandardChecker,
334 @@ -2490,6 +2508,7 @@ projectcheckers = {
335 "terminology": TermChecker,
336 "l20n": L20nChecker,
337 "ios": IOSChecker,
338 + "cacert": CAcertChecker,
339 }
340
341
342 Critical Configuration items
343 ============================
344
345 The system configuration is managed via Puppet profiles. There should be no
346 configuration items outside of the Puppet repository.
347
348 .. todo:: move configuration of :doc:`translations` to Puppet code
349
350 Keys and X.509 certificates
351 ---------------------------
352
353 .. sslcert:: translations.cacert.org
354 :altnames: DNS:l10n.cacert.org, DNS:translations.cacert.org
355 :certfile: /etc/ssl/public/translations.c.o.chain.crt
356 :keyfile: /etc/ssl/private/translations.c.o.key
357 :serial: 138202
358 :expiration: Mar 16 11:47:46 2020 GMT
359 :sha1fp: 09:D7:6C:BA:EC:60:45:4A:93:77:39:D0:0A:FA:9B:0A:3D:17:3C:CA
360 :issuer: CA Cert Signing Authority
361
362 .. seealso::
363
364 * :wiki:`SystemAdministration/CertificateList`
365
366 Apache configuration
367 --------------------
368
369 The main configuration files for Apache httpd are:
370
371 * :file:`/etc/apache2/sites-available/pootle-nossl.conf`
372
373 defines the HTTP VirtualHost that redirects all requests to
374 https://translations.cacert.org/
375
376 * :file:`/etc/apache2/sites-available/pootle-ssl.conf`
377
378 defines the HTTPS VirtualHost for Pootle including the TLS and WSGI setup
379
380 Pootle configuration
381 --------------------
382
383 The main Pootle configuration file is
384 :file:`/var/www/pootle/current/pootle.conf`. The file defines the database
385 and CAcert specific settings.
386
387 Pootle runs some background jobs that are queued via redis and run from a
388 worker process. The worker process lifecycle is managed via
389 :program:`supervisord`. The supervisor configuration for this worker is in
390 :file:`/etc/supervisor/conf.d/pootle-rqworker.conf`.
391
392 The WSGI_ runner for Pootle is contained in :file:`/var/www/pootle/wsgi.py`
393 it references the symlinked Pootle instance directory
394 :file:`/var/www/pootle/current` and should not need changes when a new
395 Pootle version is installed.
396
397 .. _WSGI: https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
398
399 There are scripts in :file:`/usr/local/bin` that were implemented for an older
400 Pootle version and have to be checked/updated.
401
402 Tasks
403 =====
404
405 Changes
406 =======
407
408 Planned
409 -------
410
411 .. todo::
412
413 integrate the pootle projects with version control systems. The templates
414 (.pot files) in :file:`/var/www/pootle/po` can be updated and loaded into
415 Pootle by invoking::
416
417 pootle update_stores --project=<project_id> --language=templates
418
419 see the `Pootle documentation <http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/server/project_setup.html#project-setup-updating-strings>`_
420
421 .. todo::
422
423 update and improve the scripts in :file:`/usr/local/bin` and integrate
424 them with the :program:`sudo` system to allow members of the `pootle-update`
425 group to run them in the context of the `pootle` system user
426
427
428 System Future
429 -------------
430
431 * keep Pootle up to date
432
433 Additional documentation
434 ========================
435
436 .. seealso::
437
438 * :wiki:`PostfixConfiguration`
439
440 References
441 ----------
442
443 Apache httpd documentation
444 http://httpd.apache.org/docs/2.4/
445 MariaDB knowledge base
446 https://mariadb.com/kb/en/
447 mod_wsgi documentation
448 https://modwsgi.readthedocs.io/en/develop/
449 Pootle documentation
450 http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/
451 Redis documentation
452 https://redis.io/documentation
453 Supervisord documentation
454 http://supervisord.org/