Renew certificate for translations
[cacert-infradocs.git] / docs / systems / translations.rst
1 .. index::
2 single: Systems; Translations
3
4 ============
5 Translations
6 ============
7
8 Purpose
9 =======
10
11 This system runs a `Pootle`_ translation server.
12
13 .. _Pootle: http://pootle.translatehouse.org/
14
15
16 Application Links
17 -----------------
18
19 Pootle web interface
20 https://translations.cacert.org/
21
22 Administration
23 ==============
24
25 System Administration
26 ---------------------
27
28 * Primary: :ref:`people_jandd`
29 * Secondary: None
30
31 .. todo:: find an additional admin
32
33 Application Administration
34 --------------------------
35
36 +-------------+---------------------+
37 | Application | Administrator(s) |
38 +=============+=====================+
39 | Pootle | :ref:`people_jandd` |
40 +-------------+---------------------+
41
42 Contact
43 -------
44
45 * translations-admin@cacert.org
46
47 Additional People
48 -----------------
49
50 :ref:`people_mario` has :program:`sudo` access on that machine too.
51
52 Basics
53 ======
54
55 Physical Location
56 -----------------
57
58 This system is located in an :term:`LXC` container on physical machine
59 :doc:`infra02`.
60
61 Logical Location
62 ----------------
63
64 :IP Internet: :ip:v4:`213.154.225.240`
65 :IP Intranet: :ip:v4:`172.16.2.31`
66 :IP Internal: :ip:v4:`10.0.0.31`
67 :MAC address: :mac:`00:ff:6c:7d:5b:c5` (eth0)
68
69 .. seealso::
70
71 See :doc:`../network`
72
73 DNS
74 ---
75
76 .. index::
77 single: DNS records; Translations
78
79 ============================== ======== ==========================================
80 Name Type Content
81 ============================== ======== ==========================================
82 translations.cacert.org. IN A 213.154.225.240
83 translations.cacert.org. IN SSHFP 1 1 1128972FB54F927477A781718E2F9C114E9CA383
84 translations.cacert.org. IN SSHFP 2 1 3A36E5DF06304C481F01FC723FD88A086E82D986
85 translations.intra.cacert.org. IN A 172.16.2.31
86 ============================== ======== ==========================================
87
88 .. seealso::
89
90 See :wiki:`SystemAdministration/Procedures/DNSChanges`
91
92 Operating System
93 ----------------
94
95 .. index::
96 single: Debian GNU/Linux; Stretch
97 single: Debian GNU/Linux; 9.4
98
99 * Debian GNU/Linux 9.4
100
101 Applicable Documentation
102 ------------------------
103
104 This is it :-)
105
106 Services
107 ========
108
109 Listening services
110 ------------------
111
112 +----------+---------+---------+----------------------------+
113 | Port | Service | Origin | Purpose |
114 +==========+=========+=========+============================+
115 | 22/tcp | ssh | ANY | admin console access |
116 +----------+---------+---------+----------------------------+
117 | 25/tcp | smtp | local | mail delivery to local MTA |
118 +----------+---------+---------+----------------------------+
119 | 80/tcp | http | ANY | redirect to https |
120 +----------+---------+---------+----------------------------+
121 | 443/tcp | https | ANY | application |
122 +----------+---------+---------+----------------------------+
123 | 3306/tcp | mysql | local | MySQL database for Pootle |
124 +----------+---------+---------+----------------------------+
125 | 5666/tcp | nrpe | monitor | remote monitoring service |
126 +----------+---------+---------+----------------------------+
127 | 6379/tcp | redis | local | Redis in memory cache |
128 +----------+---------+---------+----------------------------+
129
130 Running services
131 ----------------
132
133 .. index::
134 single: Apache
135 single: MariaDB
136 single: Postfix
137 single: Redis
138 single: cron
139 single: nrpe
140 single: openssh
141 single: rsyslog
142 single: supervisord
143
144 +--------------------+------------------------------+-----------------------------------------------------+
145 | Service | Usage | Start mechanism |
146 +====================+==============================+=====================================================+
147 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
148 | | remote | |
149 | | administration | |
150 +--------------------+------------------------------+-----------------------------------------------------+
151 | Apache httpd | Webserver for | init script |
152 | | Pootle | :file:`/etc/init.d/apache2` |
153 +--------------------+------------------------------+-----------------------------------------------------+
154 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
155 +--------------------+------------------------------+-----------------------------------------------------+
156 | rsyslog | syslog daemon | init script |
157 | | | :file:`/etc/init.d/syslog` |
158 +--------------------+------------------------------+-----------------------------------------------------+
159 | MySQL | MySQL database | init script |
160 | | server for Pootle | :file:`/etc/init.d/mysql` |
161 +--------------------+------------------------------+-----------------------------------------------------+
162 | Postfix | SMTP server for | init script |
163 | | local mail | :file:`/etc/init.d/postfix` |
164 | | submission | |
165 +--------------------+------------------------------+-----------------------------------------------------+
166 | Nagios NRPE server | remote monitoring | init script |
167 | | service queried by | :file:`/etc/init.d/nagios-nrpe-server` |
168 | | :doc:`monitor` | |
169 +--------------------+------------------------------+-----------------------------------------------------+
170 | Redis | Job queue for Pootle | init script :file:`/etc/init.d/redis-server` |
171 +--------------------+------------------------------+-----------------------------------------------------+
172 | Supervisord | Supervisor for background | init script :file:`/etc/init.d/supervisor` |
173 | | tasks | |
174 +--------------------+------------------------------+-----------------------------------------------------+
175 | Pootle rqworker | Worker for Pootle background | supervisor task in |
176 | | tasks | :file:`/etc/supervisor/conf.d/pootle-rqworker.conf` |
177 +--------------------+------------------------------+-----------------------------------------------------+
178
179 Databases
180 ---------
181
182 +-------+--------+----------+
183 | RDBMS | Name | Used for |
184 +=======+========+==========+
185 | MySQL | pootle | Pootle |
186 +-------+--------+----------+
187
188 Connected Systems
189 -----------------
190
191 * :doc:`monitor`
192
193 Outbound network connections
194 ----------------------------
195
196 * DNS (53) resolving nameservers 172.16.2.2 and 172.16.2.3
197 * :doc:`emailout` as SMTP relay
198 * :doc:`puppet` (tcp/8140) as Puppet master
199 * :doc:`proxyout` as HTTP proxy for APT
200 * arbitrary Internet HTTP, HTTPS, FTP, FTPS, git servers for fetching Pootle
201 dependencies (via ``&CONTAINER_OUT_ELEVATED("translations");`` in
202 :file:`/etc/ferm/ferm.d/translations.conf` on :doc:`infra02`).
203
204 Security
205 ========
206
207 .. sshkeys::
208 :RSA: SHA256:8iOQQGmuqi4OrF2Qkqt9665w8G7Dwl6U9J8bFfYz7V0 MD5:df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
209 :DSA: SHA256:Sh/3OWrodFWc8ZbVTV1/aJDbpt5ztGrwSSWLECTNrOI MD5:07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
210 :ECDSA: SHA256:RB1262UQIqjFgQxpRsvexHUE6XrWabBz7J1uJ3kafE0 MD5:0a:39:d9:22:39:3a:48:5d:fb:a3:27:15:d9:30:a8:64
211 :ED25519: SHA256:b+MzS1Hmj59lCwDRP1BDBgKbcadsWv9Uhz1ysk7RndU MD5:ca:a6:93:70:8c:38:23:26:16:68:5b:87:16:ee:70:17
212
213 Dedicated user roles
214 --------------------
215
216 +---------------+----------------------------------+
217 | Group | Purpose |
218 +===============+==================================+
219 | pootle-update | Planned translation update group |
220 +---------------+----------------------------------+
221
222 Non-distribution packages and modifications
223 -------------------------------------------
224
225 Pootle is a Python/Django application that has been installed in a Python
226 virtualenv. Pootle and all its dependencies have been installed using:
227
228 .. code-block:: bash
229
230 cd /var/www/pootle
231 virtualenv pootle-2.8.2
232 ln -s pootle-2.8.2 current
233 chown -R pootle.www-data pootle-2.8.2
234 sudo -s -u pootle
235 . pootle-2.8.2/bin/activate
236 pip install --process-dependency-links Pootle[mysql]
237 pootle migrate
238
239 Pootle is installed in a versioned directory. The used version is a symlink in
240 :file:`/var/www/pootle/current`. The rationale is to avoid changes to many
241 different configuration files when updating to a newer Pootle version.
242
243 The installation needs an installed :program:`gcc` and a few library development
244 packages.
245
246 .. todo::
247
248 consider building the virtualenv on :doc:`jenkins` to avoid development tools
249 on this system
250
251 Risk assessments on critical packages
252 -------------------------------------
253
254 System access is limited to http/https via Apache httpd which is restricted to
255 a minimal set of modules.
256
257 Pootle is based on Django 1.10 and should be updated to a newer version when it
258 becomes available. Pootle is run as a dedicated system user `pootle` that is
259 restricted via filesystem permissions.
260
261 The following change has been made to the translation toolkit filters that are
262 used by Pootle in :file:`/var/www/pootle/pootle-2.8.2/lib/python2.7/site-packages/translate/filters/checks.py`
263 to add CAcert specific translation checks:
264
265 .. code-block:: diff
266
267 commit 4d107e5019f4794b4581cadaf4e9a8339868f6a4
268 Author: Jan Dittberner <jandd@cacert.org>
269 Date: Fri Feb 23 20:39:03 2018 +0000
270
271 Add CAcert checkers
272
273 Signed-off-by: Jan Dittberner <jandd@cacert.org>
274
275 diff --git a/filters/checks.py b/filters/checks.py
276 index db10937..45b464c 100644
277 --- a/filters/checks.py
278 +++ b/filters/checks.py
279 @@ -2475,6 +2475,24 @@ class IOSChecker(StandardChecker):
280 StandardChecker.__init__(self, **kwargs)
281
282
283 +cacertconfig = CheckerConfig(
284 + notranslatewords = ["CAcert", "Assurer"],
285 + criticaltests = ["printf"],
286 +)
287 +
288 +
289 +class CAcertChecker(StandardChecker):
290 +
291 + def __init__(self, **kwargs):
292 + checkerconfig = kwargs.get("checkerconfig", None)
293 + if checkerconfig is None:
294 + checkerconfig = CheckerConfig()
295 + kwargs["checkerconfig"] = checkerconfig
296 +
297 + checkerconfig.update(cacertconfig)
298 + StandardChecker.__init__(self, **kwargs)
299 +
300 +
301 projectcheckers = {
302 "minimal": MinimalChecker,
303 "standard": StandardChecker,
304 @@ -2490,6 +2508,7 @@ projectcheckers = {
305 "terminology": TermChecker,
306 "l20n": L20nChecker,
307 "ios": IOSChecker,
308 + "cacert": CAcertChecker,
309 }
310
311
312 Critical Configuration items
313 ============================
314
315 The system configuration is managed via Puppet profiles. There should be no
316 configuration items outside of the Puppet repository.
317
318 .. todo:: move configuration of :doc:`translations` to Puppet code
319
320 Keys and X.509 certificates
321 ---------------------------
322
323 .. sslcert:: translations.cacert.org
324 :altnames: DNS:l10n.cacert.org, DNS:translations.cacert.org
325 :certfile: /etc/ssl/public/translations.c.o.chain.crt
326 :keyfile: /etc/ssl/private/translations.c.o.key
327 :serial: 138202
328 :expiration: Mar 16 11:47:46 2020 GMT
329 :sha1fp: 09:D7:6C:BA:EC:60:45:4A:93:77:39:D0:0A:FA:9B:0A:3D:17:3C:CA
330 :issuer: CA Cert Signing Authority
331
332 .. seealso::
333
334 * :wiki:`SystemAdministration/CertificateList`
335
336 Apache configuration
337 --------------------
338
339 The main configuration files for Apache httpd are:
340
341 * :file:`/etc/apache2/sites-available/pootle-nossl.conf`
342
343 defines the HTTP VirtualHost that redirects all requests to
344 https://translations.cacert.org/
345
346 * :file:`/etc/apache2/sites-available/pootle-ssl.conf`
347
348 defines the HTTPS VirtualHost for Pootle including the TLS and WSGI setup
349
350 Pootle configuration
351 --------------------
352
353 The main Pootle configuration file is
354 :file:`/var/www/pootle/current/pootle.conf`. The file defines the database
355 and CAcert specific settings.
356
357 Pootle runs some background jobs that are queued via redis and run from a
358 worker process. The worker process lifecycle is managed via
359 :program:`supervisord`. The supervisor configuration for this worker is in
360 :file:`/etc/supervisor/conf.d/pootle-rqworker.conf`.
361
362 The WSGI_ runner for Pootle is contained in :file:`/var/www/pootle/wsgi.py`
363 it references the symlinked Pootle instance directory
364 :file:`/var/www/pootle/current` and should not need changes when a new
365 Pootle version is installed.
366
367 .. _WSGI: https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
368
369 There are scripts in :file:`/usr/local/bin` that were implemented for an older
370 Pootle version and have to be checked/updated.
371
372 Tasks
373 =====
374
375 Planned
376 -------
377
378 .. todo::
379
380 integrate the pootle projects with version control systems. The templates
381 (.pot files) in :file:`/var/www/pootle/po` can be updated and loaded into
382 Pootle by invoking::
383
384 pootle update_stores --project=<project_id> --language=templates
385
386 see the `Pootle documentation <http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/server/project_setup.html#project-setup-updating-strings>`_
387
388 .. todo::
389
390 update and improve the scripts in :file:`/usr/local/bin` and integrate
391 them with the :program:`sudo` system to allow members of the `pootle-update`
392 group to run them in the context of the `pootle` system user
393
394 Changes
395 =======
396
397 System Future
398 -------------
399
400 * keep Pootle up to date
401
402 Additional documentation
403 ========================
404
405 .. seealso::
406
407 * :wiki:`PostfixConfiguration`
408
409 References
410 ----------
411
412 Apache httpd documentation
413 http://httpd.apache.org/docs/2.4/
414 MariaDB knowledge base
415 https://mariadb.com/kb/en/
416 mod_wsgi documentation
417 https://modwsgi.readthedocs.io/en/develop/
418 Pootle documentation
419 http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/
420 Redis documentation
421 https://redis.io/documentation
422 Supervisord documentation
423 http://supervisord.org/