Add IPv6 for monitor
[cacert-infradocs.git] / docs / systems / monitor.rst
1 .. index::
2 single: Systems; Monitor
3
4 =======
5 Monitor
6 =======
7
8 Purpose
9 =======
10
11 This system hosts an `Icinga`_ instance to centrally monitor the services in
12 the CAcert network (especially for security updates and certificate
13 expiry).
14
15 .. note::
16
17 To access the system you need a client certificate where the first email
18 address in the Subject Distinguished Name field is a cacert.org address.
19 Subject Alternative Names are not checked.
20
21 If you are the administrator of a service please ask the monitor admins to
22 add your system to the monitoring configuration and add you as system
23 contact to allow for notifications and tasks like service outage
24 acknowledgement, adding notes, rescheduling checks or setting downtimes for
25 your service.
26
27 .. _Icinga: https://www.icinga.org/
28
29 Application Links
30 -----------------
31
32 The Icinga classic frontend
33 https://monitor.cacert.org/
34
35 Administration
36 ==============
37
38 System Administration
39 ---------------------
40
41 * Primary: :ref:`people_jandd`
42 * Secondary: None
43
44 Application Administration
45 --------------------------
46
47 +-------------+-----------------------+
48 | Application | Administrator(s) |
49 +=============+=======================+
50 | Icinga | :ref:`people_jandd` |
51 +-------------+-----------------------+
52
53 Contact
54 -------
55
56 * monitor-admin@cacert.org
57
58 Additional People
59 -----------------
60
61 :ref:`people_jandd` and :ref:`people_mario` have :program:`sudo` access on that
62 machine too.
63
64 Basics
65 ======
66
67 Physical Location
68 -----------------
69
70 This system is located in an :term:`LXC` container on physical machine
71 :doc:`infra02`.
72
73 Logical Location
74 ----------------
75
76 :IP Internet: :ip:v4:`213.154.225.230`
77 :IP Intranet: :ip:v4:`172.16.2.18`
78 :IP Internal: :ip:v4:`10.0.0.18`
79 :IPv6: :ip:v6:`2001:7b8:616:162:2::18`
80 :MAC address: :mac:`00:ff:73:b3:17:43` (eth0)
81
82 .. seealso::
83
84 See :doc:`../network`
85
86 DNS
87 ---
88
89 .. index::
90 single: DNS records; Monitor
91
92 =================== ======== =========================
93 Name Type Content
94 =================== ======== =========================
95 monitor.cacert.org. IN CNAME infrastructure.cacert.org
96 =================== ======== =========================
97
98 .. seealso::
99
100 See :wiki:`SystemAdministration/Procedures/DNSChanges`
101
102 Operating System
103 ----------------
104
105 .. index::
106 single: Debian GNU/Linux; Stretch
107 single: Debian GNU/Linux; 9.3
108
109 * Debian GNU/Linux 9.3
110
111 Applicable Documentation
112 ------------------------
113
114 This is it :-)
115
116 .. seealso::
117
118 :ref:`Setup package update monitoring for a new container
119 <setup_apt_checking>`
120
121 Services
122 ========
123
124 Listening services
125 ------------------
126
127 +----------+---------+---------+-----------------------------+
128 | Port | Service | Origin | Purpose |
129 +==========+=========+=========+=============================+
130 | 22/tcp | ssh | ANY | admin console access |
131 +----------+---------+---------+-----------------------------+
132 | 25/tcp | smtp | local | mail delivery to local MTA |
133 +----------+---------+---------+-----------------------------+
134 | 80/tcp | http | ANY | Icinga classic web frontend |
135 +----------+---------+---------+-----------------------------+
136 | 443/tcp | https | ANY | Icinga classic web frontend |
137 +----------+---------+---------+-----------------------------+
138 | 5666/tcp | nrpe | monitor | remote monitoring service |
139 +----------+---------+---------+-----------------------------+
140 | 5432/tcp | pgsql | local | PostgreSQL database for IDO |
141 +----------+---------+---------+-----------------------------+
142
143 .. note::
144
145 The ssh port is reachable via NAT on infrastructure.cacert.org:11822
146
147
148 Running services
149 ----------------
150
151 .. index::
152 single: Apache
153 single: Icinga
154 single: IDO2DB
155 single: Postfix
156 single: PostgreSQL
157 single: cron
158 single: nrpe
159 single: openssh
160
161 +--------------------+--------------------+----------------------------------------+
162 | Service | Usage | Start mechanism |
163 +====================+====================+========================================+
164 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
165 | | remote | |
166 | | administration | |
167 +--------------------+--------------------+----------------------------------------+
168 | Apache httpd | Webserver for | init script |
169 | | Icinga classic | :file:`/etc/init.d/apache2` |
170 +--------------------+--------------------+----------------------------------------+
171 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
172 +--------------------+--------------------+----------------------------------------+
173 | rsyslog | syslog daemon | init script |
174 | | | :file:`/etc/init.d/syslog` |
175 +--------------------+--------------------+----------------------------------------+
176 | Icinga | Icinga monitoring | init script |
177 | | daemon | :file:`/etc/init.d/icinga` |
178 +--------------------+--------------------+----------------------------------------+
179 | IDO2DB | IDO database | init script |
180 | | writer daemon | :file:`/etc/init.d/ido2db` |
181 +--------------------+--------------------+----------------------------------------+
182 | PostgreSQL | PostgreSQL | init script |
183 | | database server | :file:`/etc/init.d/postgresql` |
184 | | for IDO | |
185 +--------------------+--------------------+----------------------------------------+
186 | Postfix | SMTP server for | init script |
187 | | local mail | :file:`/etc/init.d/postfix` |
188 | | submission | |
189 +--------------------+--------------------+----------------------------------------+
190 | Nagios NRPE server | remote monitoring | init script |
191 | | service by | :file:`/etc/init.d/nagios-nrpe-server` |
192 | | this system itself | |
193 +--------------------+--------------------+----------------------------------------+
194
195 Databases
196 ---------
197
198 +------------+--------+-----------------+
199 | RDBMS | Name | Used for |
200 +============+========+=================+
201 | PostgreSQL | icinga | Icinga IDO data |
202 +------------+--------+-----------------+
203
204 Connected Systems
205 -----------------
206
207 None
208
209 Outbound network connections
210 ----------------------------
211
212 * DNS (53) resolving nameservers 172.16.2.2 and 172.16.2.3
213 * :doc:`emailout` as SMTP relay
214 * :doc:`proxyout` as HTTP proxy for APT
215 * crl.cacert.org (rsync) for getting CRLs
216 * all :ip:v4range:`10.0.0.0/24` and :ip:v4range:`172.16.2.0/24` systems for
217 monitoring their services
218
219 .. todo:: add IPv6 ranges when they are monitored
220
221 Security
222 ========
223
224 .. sshkeys::
225 :RSA: SHA256:8iOQQGmuqi4OrF2Qkqt9665w8G7Dwl6U9J8bFfYz7V0 MD5:df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
226 :DSA: SHA256:Sh/3OWrodFWc8ZbVTV1/aJDbpt5ztGrwSSWLECTNrOI MD5:07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
227 :ECDSA: SHA256:GWvYqhQUt9INh/7VRVu6Z2YORoy/YzgBxNBmX+ZvMsk MD5:48:46:b1:5a:4e:05:64:8a:c3:76:33:77:20:91:14:70
228 :ED25519: SHA256:L5roC867bvxDJ0ckbhIQOt2A9Nh1RQBVuIJFWwrPLG0 MD5:10:94:56:09:5b:a2:28:ab:11:e0:0f:6e:e4:0c:38:bb
229
230
231 Non-distribution packages and modifications
232 -------------------------------------------
233
234 * None
235
236 Risk assessments on critical packages
237 -------------------------------------
238
239 Icinga and the classic frontend are a bit aged but have a good security track
240 record.
241
242 Apache httpd has a good reputation and is a low risk package.
243
244 NRPE is flawed and should be replaced. The risk is somewhat mitigated by
245 firewalling on :doc:`the infrastructure host <infra02>`.
246
247 Critical Configuration items
248 ============================
249
250 Keys and X.509 certificates
251 ---------------------------
252
253 * :file:`/etc/ssl/certs/monitor.c.o.pem` server certificate
254 * :file:`/etc/ssl/private/monitor.c.o.priv` server key
255 * :file:`/etc/ssl/certs/cacert.allcerts.pem` CAcert.org Class 1 and Class 3 CA
256 certificates (allowed CA certificates for client certificates and the
257 certificate chain for the server certificate)
258 * :file:`/var/local/ssl/crls/`
259
260 .. seealso::
261
262 * :wiki:`SystemAdministration/CertificateList`
263
264 CRL fetch job
265 -------------
266
267 The script :file:`/etc/cron.hourly/update-crls` is used to fetch CRLs once per
268 hour.
269
270 Apache httpd configuration
271 --------------------------
272
273 The HTTP and HTTPS VirtualHost configuration is defined in
274 :file:`/etc/apache2/sites-available/icinga-nossl` and
275 :file:`/etc/apache2/sites-available/icinga` the HTTP VirtualHost redirects to
276 the HTTPS VirtualHost.
277
278 Icinga configuration
279 --------------------
280
281 The Icinga configuration is stored in the :file:`/etc/icinga/` directory.
282 Database configuration for IDO is stored in :file:`ido2db.cfg`. The Icinga
283 classic frontend configuration is stored in :file:`cgi.cfg`. Host and service
284 configurations are defined in the :file:`objects/` subdirectory.
285
286 Tasks
287 =====
288
289 Planned
290 -------
291
292 .. todo:: switch to Icinga2 and Icingaweb2
293
294 Changes
295 =======
296
297 System Future
298 -------------
299
300 * No plans
301
302 Additional documentation
303 ========================
304
305 .. seealso::
306
307 * :wiki:`PostfixConfiguration`
308
309 References
310 ----------
311
312 Wiki page for this system
313 :wiki:`SystemAdministration/Systems/Monitor`