docs/systems.rst

   1 ====================
   2 Non-Critical Systems
   3 ====================
   4
   5 Non-critical systems are those that are managed by the infrastructure
   6 administrator team.
   7
   8 .. toctree::
   9    :maxdepth: 1
  10
  11    systems/infra02
  12    systems/blog
  13    systems/board
  14    systems/bugs
  15    systems/cats
  16    systems/email
  17    systems/emailout
  18    systems/git
  19    systems/irc
  20    systems/ircserver
  21    systems/issue
  22    systems/monitor
  23    systems/puppet
  24    systems/proxyin
  25    systems/proxyout
  26    systems/svn
  27    systems/webmail
  28    systems/webstatic
  29
  30
  31 General
  32 =======
  33
  34 .. todo:: consider whether a central MySQL service should be setup
  35
  36    Many containers contain their own instance of MySQL. It might be a better
  37    idea to centralize the MySQL setups in a single container.
  38
  39 .. todo:: consider whether a central PostgreSQL service should be setup
  40
  41 .. todo::
  42
  43    setup a central syslog service and install syslog clients in each container
  44
  45 .. _setup_apt_checking:
  46
  47 .. topic:: Setup package update monitoring for a new container
  48
  49    For Icinga to be able to check the update status of packages on you server
  50    you need to install NRPE, a helper service. Install the necessary packages::
  51
  52       sudo aptitude install nagios-plugins-basic nagios-nrpe-server
  53
  54    Put :doc:`systems/monitor` on the list of allowed hosts to access the NRPE
  55    service by adding the following line to :file:`/etc/nagios/nrpe_local.cfg`::
  56
  57       allowed_hosts=172.16.2.18
  58
  59    Tell the NRPE service that there is such a thing as the check_apt command by
  60    creating the file :file:`/etc/nagios/nrpe.d/apt.cfg` with the following
  61    contents::
  62
  63       # 'check_apt' command definition
  64       command[check_apt]=/usr/lib/nagios/plugins/check_apt
  65
  66       # 'check_apt_distupgrade' command definition
  67       command[check_apt_distupgrade]=/usr/lib/nagios/plugins/check_apt -d
  68
  69    Restart the NRPE service::
  70
  71       sudo service nagios-nrpe-server restart
  72
  73    Check that everything went well by going to https://monitor.cacert.org/,
  74    going to the APT service on the host and clicking :guilabel:`"Re-schedule
  75    the next check of this service"`. Make sure that :guilabel:`"Force Check"`
  76    is checked and click :guilabel:`"Commit"`. Now you should see a page with a
  77    green background. If not something went wrong, please contact the
  78    :doc:`systems/monitor` administrators with the details.
  79
  80    That's it, now the package update status should be properly displayed in
  81    Icinga.
  82
  83 .. todo:: think about replacing nrpe with Icinga2 satellites
  84
  85 Checklist
  86 =========
  87
  88 .. index::
  89    single: etckeeper
  90    single: nrpe
  91
  92 * All containers should be monitored by :doc:`systems/monitor` and should
  93   therefore have :program:`nagios-nrpe-server` installed
  94 * All containers should use :program:`etckeeper` to put their local setup into
  95   version control. All local setup should use :file:`/etc` to make sure it is
  96   handled by :program:`etckeeper`
  97 * All infrastructure systems must send their mail via :doc:`systems/emailout`
  98 * All infrastructure systems should have an system-admin@cacert.org alias to
  99   reach their admins
 100 * The installation of :index:`systemd-sysv` in containers can be blocked by
 101   putting the following lines in :file:`/etc/apt/preferences.d/systemd-sysv`::
 102
 103     Package: systemd-sysv
 104     Pin: release a=stable
 105     Pin-Priority: -1
 106
 107 .. todo:: document how to setup the system-admin alias on the email system