6e188afd74d47a85f854c73cc978a89e046ad48b
[cacert-infradocs.git] / docs / systems / monitor.rst
1 .. index::
2 single: Systems; Monitor
3
4 =======
5 Monitor
6 =======
7
8 Purpose
9 =======
10
11 This system hosts an `Icinga`_ instance to centrally monitor the services in
12 the CAcert network (especially for security updates and certificate
13 expiry).
14
15 .. note::
16
17 To access the system you need a client certificate where the first email
18 address in the Subject Distinguished Name field is a cacert.org address.
19 Subject Alternative Names are not checked.
20
21 If you are the administrator of a service please ask the monitor admins to
22 add your system to the monitoring configuration and add you as system
23 contact to allow for notifications and tasks like service outage
24 acknowledgement, adding notes, rescheduling checks or setting downtimes for
25 your service.
26
27 .. _Icinga: https://www.icinga.org/
28
29 Application Links
30 -----------------
31
32 The Icinga classic frontend
33 https://monitor.cacert.org/
34
35 Administration
36 ==============
37
38 System Administration
39 ---------------------
40
41 * Primary: :ref:`people_jandd`
42 * Secondary: None
43
44 Application Administration
45 --------------------------
46
47 +-------------+-----------------------+
48 | Application | Administrator(s) |
49 +=============+=======================+
50 | Icinga | :ref:`people_martin`, |
51 | | :ref:`people_neo`, |
52 | | :ref:`people_jandd` |
53 +-------------+-----------------------+
54
55 Contact
56 -------
57
58 * monitor-admin@cacert.org
59
60 Additional People
61 -----------------
62
63 :ref:`people_jandd` and :ref:`people_mario` have :program:`sudo` access on that
64 machine too.
65
66 Basics
67 ======
68
69 Physical Location
70 -----------------
71
72 This system is located in an :term:`LXC` container on physical machine
73 :doc:`infra02`.
74
75 Logical Location
76 ----------------
77
78 :IP Internet: :ip:v4:`213.154.225.230`
79 :IP Intranet: :ip:v4:`172.16.2.18`
80 :IP Internal: :ip:v4:`10.0.0.18`
81 :MAC address: :mac:`10.0.0.18` (eth0)
82
83 .. seealso::
84
85 See :doc:`../network`
86
87 DNS
88 ---
89
90 .. index::
91 single: DNS records; Monitor
92
93 =================== ======== =========================
94 Name Type Content
95 =================== ======== =========================
96 monitor.cacert.org. IN CNAME infrastructure.cacert.org
97 =================== ======== =========================
98
99 .. seealso::
100
101 See :wiki:`SystemAdministration/Procedures/DNSChanges`
102
103 Operating System
104 ----------------
105
106 .. index::
107 single: Debian GNU/Linux; Wheezy
108 single: Debian GNU/Linux; 7.11
109
110 * Debian GNU/Linux 7.11
111
112 Applicable Documentation
113 ------------------------
114
115 This is it :-)
116
117 .. seealso::
118
119 :ref:`Setup package update monitoring for a new container
120 <setup_apt_checking>`
121
122 Services
123 ========
124
125 Listening services
126 ------------------
127
128 +----------+---------+---------+-----------------------------+
129 | Port | Service | Origin | Purpose |
130 +==========+=========+=========+=============================+
131 | 22/tcp | ssh | ANY | admin console access |
132 +----------+---------+---------+-----------------------------+
133 | 25/tcp | smtp | local | mail delivery to local MTA |
134 +----------+---------+---------+-----------------------------+
135 | 80/tcp | http | ANY | Icinga classic web frontend |
136 +----------+---------+---------+-----------------------------+
137 | 443/tcp | https | ANY | Icinga classic web frontend |
138 +----------+---------+---------+-----------------------------+
139 | 5666/tcp | nrpe | monitor | remote monitoring service |
140 +----------+---------+---------+-----------------------------+
141 | 5432/tcp | pgsql | local | PostgreSQL database for IDO |
142 +----------+---------+---------+-----------------------------+
143
144 .. note::
145
146 The ssh port is reachable via NAT on infrastructure.cacert.org:11822
147
148
149 Running services
150 ----------------
151
152 .. index::
153 single: Apache
154 single: Icinga
155 single: IDO2DB
156 single: Postfix
157 single: PostgreSQL
158 single: cron
159 single: nrpe
160 single: openssh
161
162 +--------------------+--------------------+----------------------------------------+
163 | Service | Usage | Start mechanism |
164 +====================+====================+========================================+
165 | openssh server | ssh daemon for | init script :file:`/etc/init.d/ssh` |
166 | | remote | |
167 | | administration | |
168 +--------------------+--------------------+----------------------------------------+
169 | Apache httpd | Webserver for | init script |
170 | | Icinga classic | :file:`/etc/init.d/apache2` |
171 +--------------------+--------------------+----------------------------------------+
172 | cron | job scheduler | init script :file:`/etc/init.d/cron` |
173 +--------------------+--------------------+----------------------------------------+
174 | rsyslog | syslog daemon | init script |
175 | | | :file:`/etc/init.d/syslog` |
176 +--------------------+--------------------+----------------------------------------+
177 | Icinga | Icinga monitoring | init script |
178 | | daemon | :file:`/etc/init.d/icinga` |
179 +--------------------+--------------------+----------------------------------------+
180 | IDO2DB | IDO database | init script |
181 | | writer daemon | :file:`/etc/init.d/ido2db` |
182 +--------------------+--------------------+----------------------------------------+
183 | PostgreSQL | PostgreSQL | init script |
184 | | database server | :file:`/etc/init.d/postgresql` |
185 | | for IDO | |
186 +--------------------+--------------------+----------------------------------------+
187 | Postfix | SMTP server for | init script |
188 | | local mail | :file:`/etc/init.d/postfix` |
189 | | submission | |
190 +--------------------+--------------------+----------------------------------------+
191 | Nagios NRPE server | remote monitoring | init script |
192 | | service by | :file:`/etc/init.d/nagios-nrpe-server` |
193 | | this system itself | |
194 +--------------------+--------------------+----------------------------------------+
195
196 Databases
197 ---------
198
199 +------------+--------+-----------------+
200 | RDBMS | Name | Used for |
201 +============+========+=================+
202 | PostgreSQL | icinga | Icinga IDO data |
203 +------------+--------+-----------------+
204
205 Connected Systems
206 -----------------
207
208 None
209
210 Outbound network connections
211 ----------------------------
212
213 * DNS (53) resolving nameservers 172.16.2.2 and 172.16.2.3
214 * :doc:`emailout` as SMTP relay
215 * ftp.nl.debian.org as Debian mirror
216 * security.debian.org for Debian security updates
217 * crl.cacert.org (rsync) for getting CRLs
218 * all :ip:v4range:`10.0.0.0/24` and :ip:v4range:`172.16.2.0/24` systems for
219 monitoring their services
220
221 .. todo:: add IPv6 ranges when they are monitored
222
223 Security
224 ========
225
226 .. sshkeys::
227 :RSA: df:98:f5:ea:05:c1:47:52:97:58:8f:42:55:d6:d9:b6
228 :DSA: 07:2b:10:b1:6d:79:35:0f:83:aa:fc:ba:d6:2f:51:dc
229 :ECDSA: 48:46:b1:5a:4e:05:64:8a:c3:76:33:77:20:91:14:70
230
231 Non-distribution packages and modifications
232 -------------------------------------------
233
234 * None
235
236 Risk assessments on critical packages
237 -------------------------------------
238
239 Icinga and the classic frontend are a bit aged but have a good security track
240 record.
241
242 Apache httpd has a good reputation and is a low risk package.
243
244 NRPE is flawed and should be replaced. The risk is somewhat mitigated by
245 firewalling on :doc:`the infrastructure host <infra02>`.
246
247 Critical Configuration items
248 ============================
249
250 Keys and X.509 certificates
251 ---------------------------
252
253 * :file:`/etc/ssl/certs/monitor.c.o.pem` server certificate
254 * :file:`/etc/ssl/private/monitor.c.o.priv` server key
255 * :file:`/etc/ssl/certs/cacert.allcerts.pem` CAcert.org Class 1 and Class 3 CA
256 certificates (allowed CA certificates for client certificates and the
257 certificate chain for the server certificate)
258 * :file:`/var/local/ssl/crls/`
259
260 .. seealso::
261
262 * :wiki:`SystemAdministration/CertificateList`
263
264 CRL fetch job
265 -------------
266
267 The script :file:`/etc/cron.hourly/update-crls` is used to fetch CRLs once per
268 hour.
269
270 Apache httpd configuration
271 --------------------------
272
273 The HTTP and HTTPS VirtualHost configuration is defined in
274 :file:`/etc/apache2/sites-available/icinga-nossl` and
275 :file:`/etc/apache2/sites-available/icinga` the HTTP VirtualHost redirects to
276 the HTTPS VirtualHost.
277
278 Icinga configuration
279 --------------------
280
281 The Icinga configuration is stored in the :file:`/etc/icinga/` directory.
282 Database configuration for IDO is stored in :file:`ido2db.cfg`. The Icinga
283 classic frontend configuration is stored in :file:`cgi.cfg`. Host and service
284 configurations are defined in the :file:`objects/` subdirectory.
285
286 Tasks
287 =====
288
289 Planned
290 -------
291
292 .. todo:: upgrade to Debian Jessie
293 .. todo:: switch to Icinga2 and Icingaweb2
294
295 Changes
296 =======
297
298 System Future
299 -------------
300
301 * No plans
302
303 Additional documentation
304 ========================
305
306 .. seealso::
307
308 * :wiki:`PostfixConfiguration`
309
310 References
311 ----------
312
313 Wiki page for this system
314 :wiki:`SystemAdministration/Systems/Monitor`