Add web and webstatic to Puppet
[cacert-infradocs.git] / docs / systems / translations.rst
1 .. index::
2 single: Systems; Translations
3
4 ============
5 Translations
6 ============
7
8 Purpose
9 =======
10
11 This system runs a `Pootle`_ translation server.
12
13 .. _Pootle: http://pootle.translatehouse.org/
14
15
16 Application Links
17 -----------------
18
19 Pootle web interface
20 https://translations.cacert.org/
21
22 Administration
23 ==============
24
25 System Administration
26 ---------------------
27
28 * Primary: :ref:`people_jandd`
29 * Secondary: None
30
31 .. todo:: find an additional admin
32
33 Application Administration
34 --------------------------
35
36 +-------------+---------------------+
37 | Application | Administrator(s) |
38 +=============+=====================+
39 | Pootle | :ref:`people_jandd` |
40 +-------------+---------------------+
41
42 Contact
43 -------
44
45 * translations-admin@cacert.org
46
47 Additional People
48 -----------------
49
50 :ref:`people_mario` has :program:`sudo` access on that machine too.
51
52 Basics
53 ======
54
55 Physical Location
56 -----------------
57
58 This system is located in an :term:`LXC` container on physical machine
59 :doc:`infra02`.
60
61 Logical Location
62 ----------------
63
64 :IP Internet: :ip:v4:`213.154.225.240`
65 :IP Intranet: :ip:v4:`172.16.2.31`
66 :IP Internal: :ip:v4:`10.0.0.31`
67 :MAC address: :mac:`00:ff:6c:7d:5b:c5` (eth0)
68
69 .. seealso::
70
71 See :doc:`../network`
72
73 DNS
74 ---
75
76 .. index::
77 single: DNS records; Translations
78
79 ============================== ======== ==========================================
80 Name Type Content
81 ============================== ======== ==========================================
82 translations.cacert.org. IN A 213.154.225.240
83 translations.cacert.org. IN SSHFP 1 1 1128972FB54F927477A781718E2F9C114E9CA383
84 translations.cacert.org. IN SSHFP 2 1 3A36E5DF06304C481F01FC723FD88A086E82D986
85 translations.intra.cacert.org. IN A 172.16.2.31
86 ============================== ======== ==========================================
87
88 .. seealso::
89
90 See :wiki:`SystemAdministration/Procedures/DNSChanges`
91
92 Operating System
93 ----------------
94
95 .. index::
96 single: Debian GNU/Linux; Stretch
97 single: Debian GNU/Linux; 9.4
98
99 * Debian GNU/Linux 9.4
100
101 Applicable Documentation
102 ------------------------
103
104 This is it :-)
105
106 Services
107 ========
108
109 Listening services
110 ------------------
111
112 +----------+---------+---------+----------------------------+
113 | Port | Service | Origin | Purpose |
114 +==========+=========+=========+============================+
115 | 22/tcp | ssh | ANY | admin console access |
116 +----------+---------+---------+----------------------------+
117 | 25/tcp | smtp | local | mail delivery to local MTA |
118 +----------+---------+---------+----------------------------+
119 | 80/tcp | http | ANY | redirect to https |
120 +----------+---------+---------+----------------------------+
121 | 443/tcp | https | ANY | application |
122 +----------+---------+---------+----------------------------+
123 | 3306/tcp | mysql | local | MySQL database for Pootle |
124 +----------+---------+---------+----------------------------+
125 | 5666/tcp | nrpe | monitor | remote monitoring service |
126 +----------+---------+---------+----------------------------+
127 | 6379/tcp | redis | local | Redis in memory cache |
128 +----------+---------+---------+----------------------------+
129
130 Running services
131 ----------------
132
133 .. index::
134 single: apache httpd
135 single: cron
136 single: mariadb
137 single: nrpe
138 single: openssh
139 single: postfix
140 single: puppet agent
141 single: redis
142 single: rsyslog
143 single: supervisord
144
145 +--------------------+------------------------------+-----------------------------------------------------+
146 | Service | Usage | Start mechanism |
147 +====================+==============================+=====================================================+
148 | Apache httpd | Webserver for | init script |
149 | | Pootle | :file:`/etc/init.d/apache2` |
150 +--------------------+------------------------------+-----------------------------------------------------+
151 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
152 +--------------------+------------------------------+-----------------------------------------------------+
153 | MySQL | MySQL database | init script |
154 | | server for Pootle | :file:`/etc/init.d/mysql` |
155 +--------------------+------------------------------+-----------------------------------------------------+
156 | Postfix | SMTP server for | init script |
157 | | local mail | :file:`/etc/init.d/postfix` |
158 | | submission | |
159 +--------------------+------------------------------+-----------------------------------------------------+
160 | Puppet agent | local Puppet agent | init script |
161 | | | :file:`/etc/init.d/puppet` |
162 +--------------------+------------------------------+-----------------------------------------------------+
163 | Nagios NRPE server | remote monitoring | init script |
164 | | service queried by | :file:`/etc/init.d/nagios-nrpe-server` |
165 | | :doc:`monitor` | |
166 +--------------------+------------------------------+-----------------------------------------------------+
167 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
168 | | remote | |
169 | | administration | |
170 +--------------------+------------------------------+-----------------------------------------------------+
171 | Redis | Job queue for Pootle | init script :file:`/etc/init.d/redis-server` |
172 +--------------------+------------------------------+-----------------------------------------------------+
173 | rsyslog | syslog daemon | init script |
174 | | | :file:`/etc/init.d/syslog` |
175 +--------------------+------------------------------+-----------------------------------------------------+
176 | Supervisord | Supervisor for background | init script :file:`/etc/init.d/supervisor` |
177 | | tasks | |
178 +--------------------+------------------------------+-----------------------------------------------------+
179 | Pootle rqworker | Worker for Pootle background | supervisor task in |
180 | | tasks | :file:`/etc/supervisor/conf.d/pootle-rqworker.conf` |
181 +--------------------+------------------------------+-----------------------------------------------------+
182
183 Databases
184 ---------
185
186 +-------+--------+----------+
187 | RDBMS | Name | Used for |
188 +=======+========+==========+
189 | MySQL | pootle | Pootle |
190 +-------+--------+----------+
191
192 Connected Systems
193 -----------------
194
195 * :doc:`monitor`
196
197 Outbound network connections
198 ----------------------------
199
200 * :doc:`infra02` as resolving nameserver
201 * :doc:`emailout` as SMTP relay
202 * :doc:`puppet` (tcp/8140) as Puppet master
203 * :doc:`proxyout` as HTTP proxy for APT
204 * arbitrary Internet HTTP, HTTPS, FTP, FTPS, git servers for fetching Pootle
205 dependencies (via ``&CONTAINER_OUT_ELEVATED("translations");`` in
206 :file:`/etc/ferm/ferm.d/translations.conf` on :doc:`infra02`).
207
208 Security
209 ========
210
211 .. sshkeys::
212 :RSA: SHA256:8iOQQGmuqi4OrF2Qkqt9665w8G7Dwl6U9J8bFfYz7V0 MD5:df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
213 :DSA: SHA256:Sh/3OWrodFWc8ZbVTV1/aJDbpt5ztGrwSSWLECTNrOI MD5:07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
214 :ECDSA: SHA256:RB1262UQIqjFgQxpRsvexHUE6XrWabBz7J1uJ3kafE0 MD5:0a:39:d9:22:39:3a:48:5d:fb:a3:27:15:d9:30:a8:64
215 :ED25519: SHA256:b+MzS1Hmj59lCwDRP1BDBgKbcadsWv9Uhz1ysk7RndU MD5:ca:a6:93:70:8c:38:23:26:16:68:5b:87:16:ee:70:17
216
217 Dedicated user roles
218 --------------------
219
220 +---------------+----------------------------------+
221 | Group | Purpose |
222 +===============+==================================+
223 | pootle-update | Planned translation update group |
224 +---------------+----------------------------------+
225
226 Non-distribution packages and modifications
227 -------------------------------------------
228
229 Pootle is a Python/Django application that has been installed in a Python
230 virtualenv. Pootle and all its dependencies have been installed using:
231
232 .. code-block:: bash
233
234 cd /var/www/pootle
235 virtualenv pootle-2.8.2
236 ln -s pootle-2.8.2 current
237 chown -R pootle.www-data pootle-2.8.2
238 sudo -s -u pootle
239 . pootle-2.8.2/bin/activate
240 pip install --process-dependency-links Pootle[mysql]
241 pootle migrate
242
243 Pootle is installed in a versioned directory. The used version is a symlink in
244 :file:`/var/www/pootle/current`. The rationale is to avoid changes to many
245 different configuration files when updating to a newer Pootle version.
246
247 The installation needs an installed :program:`gcc` and a few library development
248 packages.
249
250 .. todo::
251
252 consider building the virtualenv on :doc:`jenkins` to avoid development tools
253 on this system
254
255 The Puppet agent package and a few dependencies are installed from the official
256 Puppet APT repository because the versions in Debian are too old to use modern
257 Puppet features.
258
259 Risk assessments on critical packages
260 -------------------------------------
261
262 System access is limited to http/https via Apache httpd which is restricted to
263 a minimal set of modules.
264
265 The system uses third party packages with a good security track record and
266 regular updates. The attack surface is small due to the tightly restricted
267 access to the system. The puppet agent is not exposed for access from outside
268 the system.
269
270 Pootle is based on Django 1.10 and should be updated to a newer version when it
271 becomes available. Pootle is run as a dedicated system user `pootle` that is
272 restricted via filesystem permissions.
273
274 The following change has been made to the translation toolkit filters that are
275 used by Pootle in :file:`/var/www/pootle/pootle-2.8.2/lib/python2.7/site-packages/translate/filters/checks.py`
276 to add CAcert specific translation checks:
277
278 .. code-block:: diff
279
280 commit 4d107e5019f4794b4581cadaf4e9a8339868f6a4
281 Author: Jan Dittberner <jandd@cacert.org>
282 Date: Fri Feb 23 20:39:03 2018 +0000
283
284 Add CAcert checkers
285
286 Signed-off-by: Jan Dittberner <jandd@cacert.org>
287
288 diff --git a/filters/checks.py b/filters/checks.py
289 index db10937..45b464c 100644
290 --- a/filters/checks.py
291 +++ b/filters/checks.py
292 @@ -2475,6 +2475,24 @@ class IOSChecker(StandardChecker):
293 StandardChecker.__init__(self, **kwargs)
294
295
296 +cacertconfig = CheckerConfig(
297 + notranslatewords = ["CAcert", "Assurer"],
298 + criticaltests = ["printf"],
299 +)
300 +
301 +
302 +class CAcertChecker(StandardChecker):
303 +
304 + def __init__(self, **kwargs):
305 + checkerconfig = kwargs.get("checkerconfig", None)
306 + if checkerconfig is None:
307 + checkerconfig = CheckerConfig()
308 + kwargs["checkerconfig"] = checkerconfig
309 +
310 + checkerconfig.update(cacertconfig)
311 + StandardChecker.__init__(self, **kwargs)
312 +
313 +
314 projectcheckers = {
315 "minimal": MinimalChecker,
316 "standard": StandardChecker,
317 @@ -2490,6 +2508,7 @@ projectcheckers = {
318 "terminology": TermChecker,
319 "l20n": L20nChecker,
320 "ios": IOSChecker,
321 + "cacert": CAcertChecker,
322 }
323
324
325 Critical Configuration items
326 ============================
327
328 The system configuration is managed via Puppet profiles. There should be no
329 configuration items outside of the Puppet repository.
330
331 .. todo:: move configuration of :doc:`translations` to Puppet code
332
333 Keys and X.509 certificates
334 ---------------------------
335
336 .. sslcert:: translations.cacert.org
337 :altnames: DNS:l10n.cacert.org, DNS:translations.cacert.org
338 :certfile: /etc/ssl/public/translations.c.o.chain.crt
339 :keyfile: /etc/ssl/private/translations.c.o.key
340 :serial: 138202
341 :expiration: Mar 16 11:47:46 2020 GMT
342 :sha1fp: 09:D7:6C:BA:EC:60:45:4A:93:77:39:D0:0A:FA:9B:0A:3D:17:3C:CA
343 :issuer: CA Cert Signing Authority
344
345 .. seealso::
346
347 * :wiki:`SystemAdministration/CertificateList`
348
349 Apache configuration
350 --------------------
351
352 The main configuration files for Apache httpd are:
353
354 * :file:`/etc/apache2/sites-available/pootle-nossl.conf`
355
356 defines the HTTP VirtualHost that redirects all requests to
357 https://translations.cacert.org/
358
359 * :file:`/etc/apache2/sites-available/pootle-ssl.conf`
360
361 defines the HTTPS VirtualHost for Pootle including the TLS and WSGI setup
362
363 Pootle configuration
364 --------------------
365
366 The main Pootle configuration file is
367 :file:`/var/www/pootle/current/pootle.conf`. The file defines the database
368 and CAcert specific settings.
369
370 Pootle runs some background jobs that are queued via redis and run from a
371 worker process. The worker process lifecycle is managed via
372 :program:`supervisord`. The supervisor configuration for this worker is in
373 :file:`/etc/supervisor/conf.d/pootle-rqworker.conf`.
374
375 The WSGI_ runner for Pootle is contained in :file:`/var/www/pootle/wsgi.py`
376 it references the symlinked Pootle instance directory
377 :file:`/var/www/pootle/current` and should not need changes when a new
378 Pootle version is installed.
379
380 .. _WSGI: https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
381
382 There are scripts in :file:`/usr/local/bin` that were implemented for an older
383 Pootle version and have to be checked/updated.
384
385 Tasks
386 =====
387
388 Planned
389 -------
390
391 .. todo::
392
393 integrate the pootle projects with version control systems. The templates
394 (.pot files) in :file:`/var/www/pootle/po` can be updated and loaded into
395 Pootle by invoking::
396
397 pootle update_stores --project=<project_id> --language=templates
398
399 see the `Pootle documentation <http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/server/project_setup.html#project-setup-updating-strings>`_
400
401 .. todo::
402
403 update and improve the scripts in :file:`/usr/local/bin` and integrate
404 them with the :program:`sudo` system to allow members of the `pootle-update`
405 group to run them in the context of the `pootle` system user
406
407 Changes
408 =======
409
410 System Future
411 -------------
412
413 * keep Pootle up to date
414
415 Additional documentation
416 ========================
417
418 .. seealso::
419
420 * :wiki:`PostfixConfiguration`
421
422 References
423 ----------
424
425 Apache httpd documentation
426 http://httpd.apache.org/docs/2.4/
427 MariaDB knowledge base
428 https://mariadb.com/kb/en/
429 mod_wsgi documentation
430 https://modwsgi.readthedocs.io/en/develop/
431 Pootle documentation
432 http://docs.translatehouse.org/projects/pootle/en/stable-2.8.x/
433 Redis documentation
434 https://redis.io/documentation
435 Supervisord documentation
436 http://supervisord.org/