572849b930c4c889d12265430c73c64d5f39c398
[cacert-infradocs.git] / docs / systems / translations.rst
1 .. index::
2 single: Systems; Translations
3
4 ============
5 Translations
6 ============
7
8 Purpose
9 =======
10
11 This system runs a `Pootle`_ translation server.
12
13 .. _Pootle: http://pootle.translatehouse.org/
14
15
16 Application Links
17 -----------------
18
19 Pootle web interface
20 https://translations.cacert.org/
21
22 Administration
23 ==============
24
25 System Administration
26 ---------------------
27
28 * Primary: :ref:`people_jandd`
29 * Secondary: None
30
31 .. todo:: find an additional admin
32
33 Application Administration
34 --------------------------
35
36 +-------------+---------------------+
37 | Application | Administrator(s) |
38 +=============+=====================+
39 | Pootle | :ref:`people_jandd` |
40 +-------------+---------------------+
41
42 Contact
43 -------
44
45 * translations-admin@cacert.org
46
47 Additional People
48 -----------------
49
50 :ref:`people_mario` has :program:`sudo` access on that machine too.
51
52 Basics
53 ======
54
55 Physical Location
56 -----------------
57
58 This system is located in an :term:`LXC` container on physical machine
59 :doc:`infra02`.
60
61 Logical Location
62 ----------------
63
64 :IP Internet: :ip:v4:`213.154.225.240`
65 :IP Intranet: :ip:v4:`172.16.2.31`
66 :IP Internal: :ip:v4:`10.0.0.31`
67 :IPv6: :ip:v6:`2001:7b8:616:162:2::31`
68 :MAC address: :mac:`00:ff:6c:7d:5b:c5` (eth0)
69
70 .. seealso::
71
72 See :doc:`../network`
73
74 DNS
75 ---
76
77 .. index::
78 single: DNS records; Translations
79
80 ============================== ======== ====================================================================
81 Name Type Content
82 ============================== ======== ====================================================================
83 l10n.cacert.org. IN CNAME translations.cacert.org.
84 translations.cacert.org. IN A 213.154.225.240
85 translations.cacert.org. IN AAAA 2001:7b8:616:162:2::31
86 translations.cacert.org. IN SSHFP 1 1 1128972FB54F927477A781718E2F9C114E9CA383
87 translations.cacert.org. IN SSHFP 1 2 F223904069AEAA2E0EAC5D9092AB7DEBAE70F06EC3C25E94F49F1B15F633ED5D
88 translations.cacert.org. IN SSHFP 2 1 3A36E5DF06304C481F01FC723FD88A086E82D986
89 translations.cacert.org. IN SSHFP 2 2 4A1FF7396AE874559CF196D54D5D7F6890DBA6DE73B46AF049258B1024CDACE2
90 translations.cacert.org. IN SSHFP 3 1 0F0CBD9C188D619D743859A249238F684D6CCA5F
91 translations.cacert.org. IN SSHFP 3 2 441D76EB651022A8C5810C6946CBDEC47504E97AD669B073EC9D6E27791A7C4D
92 translations.cacert.org. IN SSHFP 4 1 A4102E1FBF1BE1ACD53F2E7653DD8898E567C437
93 translations.cacert.org. IN SSHFP 4 2 6FE3334B51E68F9F650B00D13F504306029B71A76C5AFF54873D72B24ED19DD5
94 translations.intra.cacert.org. IN A 172.16.2.31
95 ============================== ======== ====================================================================
96
97 .. seealso::
98
99 See :wiki:`SystemAdministration/Procedures/DNSChanges`
100
101 Operating System
102 ----------------
103
104 .. index::
105 single: Debian GNU/Linux; Stretch
106 single: Debian GNU/Linux; 9.4
107
108 * Debian GNU/Linux 9.4
109
110 Applicable Documentation
111 ------------------------
112
113 This is it :-)
114
115 Services
116 ========
117
118 Listening services
119 ------------------
120
121 +----------+---------+---------+----------------------------+
122 | Port | Service | Origin | Purpose |
123 +==========+=========+=========+============================+
124 | 22/tcp | ssh | ANY | admin console access |
125 +----------+---------+---------+----------------------------+
126 | 25/tcp | smtp | local | mail delivery to local MTA |
127 +----------+---------+---------+----------------------------+
128 | 80/tcp | http | ANY | redirect to https |
129 +----------+---------+---------+----------------------------+
130 | 443/tcp | https | ANY | application |
131 +----------+---------+---------+----------------------------+
132 | 3306/tcp | mysql | local | MySQL database for Pootle |
133 +----------+---------+---------+----------------------------+
134 | 5666/tcp | nrpe | monitor | remote monitoring service |
135 +----------+---------+---------+----------------------------+
136 | 6379/tcp | redis | local | Redis in memory cache |
137 +----------+---------+---------+----------------------------+
138
139 Running services
140 ----------------
141
142 .. index::
143 single: apache httpd
144 single: cron
145 single: mariadb
146 single: nrpe
147 single: openssh
148 single: postfix
149 single: puppet agent
150 single: redis
151 single: rsyslog
152 single: supervisord
153
154 +--------------------+------------------------------+-----------------------------------------------------+
155 | Service | Usage | Start mechanism |
156 +====================+==============================+=====================================================+
157 | Apache httpd | Webserver for | init script |
158 | | Pootle | :file:`/etc/init.d/apache2` |
159 +--------------------+------------------------------+-----------------------------------------------------+
160 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
161 +--------------------+------------------------------+-----------------------------------------------------+
162 | MySQL | MySQL database | init script |
163 | | server for Pootle | :file:`/etc/init.d/mysql` |
164 +--------------------+------------------------------+-----------------------------------------------------+
165 | Postfix | SMTP server for | init script |
166 | | local mail | :file:`/etc/init.d/postfix` |
167 | | submission | |
168 +--------------------+------------------------------+-----------------------------------------------------+
169 | Puppet agent | local Puppet agent | init script |
170 | | | :file:`/etc/init.d/puppet` |
171 +--------------------+------------------------------+-----------------------------------------------------+
172 | Nagios NRPE server | remote monitoring | init script |
173 | | service queried by | :file:`/etc/init.d/nagios-nrpe-server` |
174 | | :doc:`monitor` | |
175 +--------------------+------------------------------+-----------------------------------------------------+
176 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
177 | | remote | |
178 | | administration | |
179 +--------------------+------------------------------+-----------------------------------------------------+
180 | Redis | Job queue for Pootle | init script :file:`/etc/init.d/redis-server` |
181 +--------------------+------------------------------+-----------------------------------------------------+
182 | rsyslog | syslog daemon | init script |
183 | | | :file:`/etc/init.d/syslog` |
184 +--------------------+------------------------------+-----------------------------------------------------+
185 | Supervisord | Supervisor for background | init script :file:`/etc/init.d/supervisor` |
186 | | tasks | |
187 +--------------------+------------------------------+-----------------------------------------------------+
188 | Pootle rqworker | Worker for Pootle background | supervisor task in |
189 | | tasks | :file:`/etc/supervisor/conf.d/pootle-rqworker.conf` |
190 +--------------------+------------------------------+-----------------------------------------------------+
191
192 Databases
193 ---------
194
195 +-------+--------+----------+
196 | RDBMS | Name | Used for |
197 +=======+========+==========+
198 | MySQL | pootle | Pootle |
199 +-------+--------+----------+
200
201 Connected Systems
202 -----------------
203
204 * :doc:`monitor`
205
206 Outbound network connections
207 ----------------------------
208
209 * :doc:`infra02` as resolving nameserver
210 * :doc:`emailout` as SMTP relay
211 * :doc:`puppet` (tcp/8140) as Puppet master
212 * :doc:`proxyout` as HTTP proxy for APT
213 * arbitrary Internet HTTP, HTTPS, FTP, FTPS, git servers for fetching Pootle
214 dependencies (via ``&CONTAINER_OUT_ELEVATED("translations");`` in
215 :file:`/etc/ferm/ferm.d/translations.conf` on :doc:`infra02`).
216
217 Security
218 ========
219
220 .. sshkeys::
221 :RSA: SHA256:8iOQQGmuqi4OrF2Qkqt9665w8G7Dwl6U9J8bFfYz7V0 MD5:df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
222 :DSA: SHA256:Sh/3OWrodFWc8ZbVTV1/aJDbpt5ztGrwSSWLECTNrOI MD5:07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
223 :ECDSA: SHA256:RB1262UQIqjFgQxpRsvexHUE6XrWabBz7J1uJ3kafE0 MD5:0a:39:d9:22:39:3a:48:5d:fb:a3:27:15:d9:30:a8:64
224 :ED25519: SHA256:b+MzS1Hmj59lCwDRP1BDBgKbcadsWv9Uhz1ysk7RndU MD5:ca:a6:93:70:8c:38:23:26:16:68:5b:87:16:ee:70:17
225
226 Dedicated user roles
227 --------------------
228
229 +---------------+----------------------------------+
230 | Group | Purpose |
231 +===============+==================================+
232 | pootle-update | Planned translation update group |
233 +---------------+----------------------------------+
234
235 Non-distribution packages and modifications
236 -------------------------------------------
237
238 Pootle is a Python/Django application that has been installed in a Python
239 virtualenv. Pootle and all its dependencies have been installed using:
240
241 .. code-block:: bash
242
243 cd /var/www/pootle
244 virtualenv pootle-2.8.2
245 ln -s pootle-2.8.2 current
246 chown -R pootle.www-data pootle-2.8.2
247 sudo -s -u pootle
248 . pootle-2.8.2/bin/activate
249 pip install --process-dependency-links Pootle[mysql]
250 pootle migrate
251
252 Pootle is installed in a versioned directory. The used version is a symlink in
253 :file:`/var/www/pootle/current`. The rationale is to avoid changes to many
254 different configuration files when updating to a newer Pootle version.
255
256 The installation needs an installed :program:`gcc` and a few library development
257 packages.
258
259 .. todo::
260
261 consider building the virtualenv on :doc:`jenkins` to avoid development tools
262 on this system
263
264 The Puppet agent package and a few dependencies are installed from the official
265 Puppet APT repository because the versions in Debian are too old to use modern
266 Puppet features.
267
268 Risk assessments on critical packages
269 -------------------------------------
270
271 System access is limited to http/https via Apache httpd which is restricted to
272 a minimal set of modules.
273
274 The system uses third party packages with a good security track record and
275 regular updates. The attack surface is small due to the tightly restricted
276 access to the system. The puppet agent is not exposed for access from outside
277 the system.
278
279 Pootle is based on Django 1.10 and should be updated to a newer version when it
280 becomes available. Pootle is run as a dedicated system user `pootle` that is
281 restricted via filesystem permissions.
282
283 The following change has been made to the translation toolkit filters that are
284 used by Pootle in :file:`/var/www/pootle/pootle-2.8.2/lib/python2.7/site-packages/translate/filters/checks.py`
285 to add CAcert specific translation checks:
286
287 .. code-block:: diff
288
289 commit 4d107e5019f4794b4581cadaf4e9a8339868f6a4
290 Author: Jan Dittberner <jandd@cacert.org>
291 Date: Fri Feb 23 20:39:03 2018 +0000
292
293 Add CAcert checkers
294
295 Signed-off-by: Jan Dittberner <jandd@cacert.org>
296
297 diff --git a/filters/checks.py b/filters/checks.py
298 index db10937..45b464c 100644
299 --- a/filters/checks.py
300 +++ b/filters/checks.py
301 @@ -2475,6 +2475,24 @@ class IOSChecker(StandardChecker):
302 StandardChecker.__init__(self, **kwargs)
303
304
305 +cacertconfig = CheckerConfig(
306 + notranslatewords = ["CAcert", "Assurer"],
307 + criticaltests = ["printf"],
308 +)
309 +
310 +
311 +class CAcertChecker(StandardChecker):
312 +
313 + def __init__(self, **kwargs):
314 + checkerconfig = kwargs.get("checkerconfig", None)
315 + if checkerconfig is None:
316 + checkerconfig = CheckerConfig()
317 + kwargs["checkerconfig"] = checkerconfig
318 +
319 + checkerconfig.update(cacertconfig)
320 + StandardChecker.__init__(self, **kwargs)
321 +
322 +
323 projectcheckers = {
324 "minimal": MinimalChecker,
325 "standard": StandardChecker,
326 @@ -2490,6 +2508,7 @@ projectcheckers = {
327 "terminology": TermChecker,
328 "l20n": L20nChecker,
329 "ios": IOSChecker,
330 + "cacert": CAcertChecker,
331 }
332
333
334 Critical Configuration items
335 ============================
336
337 The system configuration is managed via Puppet profiles. There should be no
338 configuration items outside of the Puppet repository.
339
340 .. todo:: move configuration of :doc:`translations` to Puppet code
341
342 Keys and X.509 certificates
343 ---------------------------
344
345 .. sslcert:: translations.cacert.org
346 :altnames: DNS:l10n.cacert.org, DNS:translations.cacert.org
347 :certfile: /etc/ssl/public/translations.c.o.chain.crt
348 :keyfile: /etc/ssl/private/translations.c.o.key
349 :serial: 138202
350 :expiration: Mar 16 11:47:46 2020 GMT
351 :sha1fp: 09:D7:6C:BA:EC:60:45:4A:93:77:39:D0:0A:FA:9B:0A:3D:17:3C:CA
352 :issuer: CA Cert Signing Authority
353
354 .. seealso::
355
356 * :wiki:`SystemAdministration/CertificateList`
357
358 Apache configuration
359 --------------------
360
361 The main configuration files for Apache httpd are:
362
363 * :file:`/etc/apache2/sites-available/pootle-nossl.conf`
364
365 defines the HTTP VirtualHost that redirects all requests to
366 https://translations.cacert.org/
367
368 * :file:`/etc/apache2/sites-available/pootle-ssl.conf`
369
370 defines the HTTPS VirtualHost for Pootle including the TLS and WSGI setup
371
372 Pootle configuration
373 --------------------
374
375 The main Pootle configuration file is
376 :file:`/var/www/pootle/current/pootle.conf`. The file defines the database
377 and CAcert specific settings.
378
379 Pootle runs some background jobs that are queued via redis and run from a
380 worker process. The worker process lifecycle is managed via
381 :program:`supervisord`. The supervisor configuration for this worker is in
382 :file:`/etc/supervisor/conf.d/pootle-rqworker.conf`.
383
384 The WSGI_ runner for Pootle is contained in :file:`/var/www/pootle/wsgi.py`
385 it references the symlinked Pootle instance directory
386 :file:`/var/www/pootle/current` and should not need changes when a new
387 Pootle version is installed.
388
389 .. _WSGI: https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
390
391 There are scripts in :file:`/usr/local/bin` that were implemented for an older
392 Pootle version and have to be checked/updated.
393
394 Tasks
395 =====
396
397 Planned
398 -------
399
400 .. todo::
401
402 integrate the pootle projects with version control systems. The templates
403 (.pot files) in :file:`/var/www/pootle/po` can be updated and loaded into
404 Pootle by invoking::
405
406 pootle update_stores --project=<project_id> --language=templates
407
408 see the `Pootle documentation <http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/server/project_setup.html#project-setup-updating-strings>`_
409
410 .. todo::
411
412 update and improve the scripts in :file:`/usr/local/bin` and integrate
413 them with the :program:`sudo` system to allow members of the `pootle-update`
414 group to run them in the context of the `pootle` system user
415
416 Changes
417 =======
418
419 System Future
420 -------------
421
422 * keep Pootle up to date
423
424 Additional documentation
425 ========================
426
427 .. seealso::
428
429 * :wiki:`PostfixConfiguration`
430
431 References
432 ----------
433
434 Apache httpd documentation
435 http://httpd.apache.org/docs/2.4/
436 MariaDB knowledge base
437 https://mariadb.com/kb/en/
438 mod_wsgi documentation
439 https://modwsgi.readthedocs.io/en/develop/
440 Pootle documentation
441 http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/
442 Redis documentation
443 https://redis.io/documentation
444 Supervisord documentation
445 http://supervisord.org/