summaryrefslogtreecommitdiff
path: root/source
diff options
context:
space:
mode:
authorJan Dittberner <jandd@cacert.org>2018-10-28 19:22:32 +0100
committerJan Dittberner <jandd@cacert.org>2018-10-28 19:25:06 +0100
commitcff7cc67797d3b2a7ab03c6bf2b726d5725f26cd (patch)
treef51f20447c655dbd05873bcc27ae2443825e7496 /source
parente2045ba3c673588d80923e734d311e2a1b5b424a (diff)
downloadcacert-codedocs-cff7cc67797d3b2a7ab03c6bf2b726d5725f26cd.tar.gz
cacert-codedocs-cff7cc67797d3b2a7ab03c6bf2b726d5725f26cd.tar.xz
cacert-codedocs-cff7cc67797d3b2a7ab03c6bf2b726d5725f26cd.zip
Start documenting CommModule
- add a file meant to collect general observations - add a file meant to collect information related to the database schema - add a glossary file - add documentation for the CommModule files in source/directories.rst - start signer protocol specification in source/signer.rst - add support for block and sequence diagrams via sphinxcontrib-blockdiag and sphinxcontrib-seqdiag
Diffstat (limited to 'source')
-rw-r--r--source/conf.py3
-rw-r--r--source/directories.rst167
-rw-r--r--source/general.rst26
-rw-r--r--source/glossary.rst16
-rw-r--r--source/index.rst4
-rw-r--r--source/signer.rst222
6 files changed, 437 insertions, 1 deletions
diff --git a/source/conf.py b/source/conf.py
index 266ca3e..5ebc900 100644
--- a/source/conf.py
+++ b/source/conf.py
@@ -62,6 +62,9 @@ extensions = [
'sphinx.ext.extlinks',
'sphinx.ext.todo',
'sphinx.ext.ifconfig',
+ 'sphinxcontrib.phpdomain',
+ 'sphinxcontrib.blockdiag',
+ 'sphinxcontrib.seqdiag',
]
# Add any paths that contain templates here, relative to this directory.
diff --git a/source/directories.rst b/source/directories.rst
index fd67962..cce7201 100644
--- a/source/directories.rst
+++ b/source/directories.rst
@@ -14,11 +14,15 @@ The root directory contains
.. _GPL: https://www.gnu.org/licenses/old-licenses/gpl-2.0
+.. index:: cgi-bin
+
Directory :file:`cgi-bin`
=========================
The `cgi-bin` directory contains
+.. index:: php
+
.. _cgi-bin-siteseal-cgi:
- :file:`siteseal.cgi` a PHP CGI script that generates some JavaScript code
@@ -28,6 +32,169 @@ The `cgi-bin` directory contains
.. todo: check whether this is linked anywhere or can be removed
+.. index:: commmodule
+.. index:: Perl
+.. index:: bash
+
+Directory :file:`CommModule`
+============================
+
+This directory contains the CommModule that is implemented in Perl:
+
+.. _commmodule-client-pl:
+
+- :file:`client.pl` the real client, running on the webserver
+
+ The style of the Perl code seems a bit inconsistent (mix of uppercase and
+ lowercase function names, usage of brackets). The code uses database polling
+ in a loop. It might be a better idea to use some kind of queueing (Redis,
+ AMQP, ...) to not waste resources when there is nothing to do). Function
+ parameters are not named which makes the code hard to read.
+
+ The script calls several system binaries that need to be present in
+ compatible versions:
+
+ - :program:`openssl`
+ - :program:`xdelta`
+
+ The script uses several Perl standard library modules as well as the
+ following third party modules:
+
+ .. index:: Perl, thirdparty
+
+ - `DBD::mysql <https://metacpan.org/pod/DBD::mysql>`_
+ - `DBI <https://metacpan.org/pod/DBI>`_
+ - `Device::SerialPort <https://metacpan.org/pod/Device::SerialPort>`_
+ - `File::CounterFile <https://metacpan.org/pod/File::CounterFile>`_
+
+ The script references several openssl configuration files in the HandleCerts
+ function that are not included in the code repository. There are some
+ openssl configuration files with similar names in
+ https://svn.cacert.org/CAcert/SystemAdministration/signer/
+
+ The database password is parsed from
+ :ref:`includes/mysql.php <includes-mysql-php>` and relies on the
+ exact code that is defined there. Database name, user and host are hardcoded
+ in the DBI->connect call.
+
+ The script implements the client side of the signer protocol which is
+ specified in :doc:`signer`.
+
+ The script performs the following operations:
+
+ - parse password from :file:`includes/mysql.php`
+ - read a list of CRL files and logs their SHA-1 hashes
+ - read :file:`serial.conf`, create a Device::SerialPort instance `$portObj`,
+ sets serial parameters and saves :file:`serial.conf`
+ - run a main loop as long as a file :file:`./client.pl-active` is present.
+ The main loop performs the following tasks
+
+ - handle pending OpenPGP key signing request via ``HandleGPG()``
+ - handle pending certificate signing requests:
+
+ - personal client certificates via ``HandleCerts(0, 0)``
+ - personal server certificates via ``HandleCerts(0, 1)``
+ - organization client certificates via ``HandleCerts(1, 0)``
+ - organization server certificates via ``HandleCerts(1, 1)``
+
+ - handle pending certificate revocation requests
+
+ - personal client certificates via ``RevokeCerts(0, 0)``
+ - personal server certificates via ``RevokeCerts(0, 1)``
+ - organization client certificates via ``RevokeCerts(1, 0)``
+ - organization server certificates via ``RevokeCerts(1, 1)``
+
+ - refresh :term:`CRLs <CRL>` via ``RefreshCRLs()`` in every 100st
+ iteration
+ - send a :ref:`NUL request <signer-nul-request-format>` to keep the signer
+ connection alive
+ - sleep for 2.7 seconds
+
+ There is potential for optimization in the main loop. The CRL update could
+ be performed if a certificate has been revoked. The NUL request needs only
+ to be sent if no other request has been sent.
+
+ .. todo:: describe more in-depth what each of the main loop steps does
+
+- :file:`commdaemon` a script to run :ref:`client.pl <commmodule-client-pl>`
+ or :ref:`server.pl <commmodule-server-pl>`
+
+ This bash script is automatically restarting the :file:`{script}` given as
+ the first parameter as long as a file :file:`{script}-active` exists.
+ Informational messages and errors are logged to syslog via
+ :command:`logger`.
+
+ The script is most probably used to recover from crashed scripts. This
+ could be implemented via :command:`supervisor` or :command:`systemd`
+ instead of a custom script.
+
+- :file:`commmodule` a script for startup/shutdown of CommModule from
+ /etc/init.d
+- :file:`logclean.sh` maintenance script for logfiles generated by CommModule
+- :file:`serial.conf` serial port configuration file
+
+.. _commmodule-server-pl:
+
+- :file:`server.pl` the real server, running on the signing server
+
+ This script implements the signer (server) side of the signer protocol and
+ performs the actual signing operations.
+
+ The script contains a some code that is duplicated by
+ :ref:`client.pl <commmodule-client-pl>`.
+
+- :file:`usbclient.pl` obsoleted USB version of
+ :ref:`client.pl <commmodule-client-pl>` above
+
+.. todo: remove unused file (usbclient.pl)
+
+.. todo: add a serial.conf template and move the actual serial.conf into
+ configuration management
+
+.. todo: clarify why log rotation is implemented with a custom
+ logclean.sh script instead of using logrotate
+
+Directory :file:`includes`
+==============================
+
+.. _includes-mysql-php:
+.. _includes-mysql-php-sample:
+
+- :file:`mysql.php.sample` is a template for the database connection handling
+ code that is meant to be copied to :file:`mysql.php`.
+
+ The template defines the MySQL connection as a session variable `mconn` and
+ tries to connect to that database. It also defines the session variables
+ `normalhostname`, `securehostname` and `tverify`.
+
+ The template defines a function :php:func:`sendmail` for sending mails.
+
+ .. php:function:: sendmail($to, $subject, $message, $from, $replyto="", \
+ $toname="", $fromname="", $errorsto="returns@cacert.org", \
+ $use_utf8=true)
+
+ Send an email. The function reimplements functionality that is readily
+ available in PHP. The function does not properly escape headers and
+ sends raw SMTP commands.
+
+ :param string $to: recipient email address
+ :param string $subject: subject
+ :param string $message: email body
+ :param string $from: from email address
+ :param string $replyto: reply-to email address
+ :param string $fromname: unused in the code
+ :param string $toname: unused in the code
+ :param string $errorsto: email address used for Sender and Errors-To
+ headers
+ :param bool $use_utf8: decides whether the Content-Type header uses
+ a charset parameter of utf-8 or iso-8859-1
+
+ Configuration and actual code are mixed. It would be better to have a
+ separate file that just includes configuration.
+
+ This file is parsed by :ref:`CommModule/client.pl <commmodule-client-pl>`
+ format changes might break the CommModule code.
+
Directory :file:`www`
=====================
diff --git a/source/general.rst b/source/general.rst
new file mode 100644
index 0000000..908a017
--- /dev/null
+++ b/source/general.rst
@@ -0,0 +1,26 @@
+====================
+General observations
+====================
+
+License
+=======
+
+The code is licensed under the terms of the GPL version 2 upgrading to GPL 3
+would require consent from all former contributors. Copyright years of files
+have not been consistently incremented/updated on changes.
+
+Languages
+=========
+
+The code base is a mix of Perl, Shell and PHP code. Most of the code is
+implemented in PHP.
+
+Code structure
+==============
+
+Comments and inline documentation
+=================================
+
+The code base is not documented in a good way, there are neither class nor
+method or function comments. Comments are just used for the license header
+in most of the files. \ No newline at end of file
diff --git a/source/glossary.rst b/source/glossary.rst
new file mode 100644
index 0000000..b3b2383
--- /dev/null
+++ b/source/glossary.rst
@@ -0,0 +1,16 @@
+========
+Glossary
+========
+
+.. glossary::
+
+ CRL
+ Definition from :rfc:`5280`:
+
+ X.509 defines one method of certificate revocation. This method
+ involves each CA periodically issuing a signed data structure called
+ a certificate revocation list (CRL). A CRL is a time-stamped list
+ identifying revoked certificates that is signed by a CA or CRL
+ issuer and made freely available in a public repository. Each
+ revoked certificate is identified in a CRL by its certificate serial
+ number. \ No newline at end of file
diff --git a/source/index.rst b/source/index.rst
index d49e27a..8ecf58f 100644
--- a/source/index.rst
+++ b/source/index.rst
@@ -20,9 +20,12 @@ contribution. The canonical repository is the :cacertgit:`cacert-devel` though.
:maxdepth: 2
:caption: Contents:
+ general
directories
database
+ signer
building
+ glossary
Filesystem structure
--------------------
@@ -39,5 +42,4 @@ Indices and tables
==================
* :ref:`genindex`
-* :ref:`modindex`
* :ref:`search`
diff --git a/source/signer.rst b/source/signer.rst
new file mode 100644
index 0000000..299562a
--- /dev/null
+++ b/source/signer.rst
@@ -0,0 +1,222 @@
+===================
+The Signer Protocol
+===================
+
+Communication with the signer is performed via a serial connection. That has
+to be established by the client before speaking the protocol defined here.
+
+.. _signer-request-data-format:
+
+Signer request data format specification
+========================================
+
+Protocol data is encoded with the following format:
+
+.. table:: signer request message format
+
+ ======= ==============================================================
+ Byte Data
+ ======= ==============================================================
+ 0-2 length of header + data in network byte order
+ 3-12 action specific header
+ 13-15 length of first action specific content in network byte order
+ 15-N fist action specific content string
+ N+1-N+3 length of second action specific content in network byte order
+ N+4-M second action specific content string
+ M+1-M+3 lenght of third action specific content in network byte order
+ M+4-End third action specific content string
+ ======= ==============================================================
+
+Due to the length encoding in 3 bytes the messages can have a maximum length
+of 8\ :sup:`3` = 2\ :sup:`24` Bytes which is around 16 MiB.
+
+General header format
+---------------------
+
+Every protocol header (bytes 3-12 of protocol message) follows the same 8 byte
+structure. The content of bits 3-8 are protocol action specific.
+
+.. table:: general request header format
+
+ ==== =============================================================
+ Byte Value
+ ==== =============================================================
+ 0 Version (``0x01``)
+ 1 Action
+ 2 System (``0x01`` for :ref:`client.pl <commmodule-client-pl>`)
+ 3 8 bits root
+ 4 8 bits configuration
+ 5 8 bits parameter
+ 6-7 16 bits parameter
+ 8 8 bits parameter
+ ==== =============================================================
+
+.. _signer-nul-request-format:
+
+Format of NUL Requests
+----------------------
+
+NUL requests are sent at the end of each iteration in
+:ref:`client.pl <commmodule-client-pl>`'s main loop.
+
+.. table:: NUL request header format
+
+ ==== =========================================================
+ Byte Value
+ ==== =========================================================
+ 0 Version (``0x01``)
+ 1 Action ``0x00``
+ 2 System (``0x01`` for :ref:`client.pl <commmodule-client-pl>`)
+ 3 ``0x00``
+ 4 ``0x00``
+ 5 ``0x00``
+ 6-7 ``0x0000``
+ 8 ``0x00``
+ ==== =========================================================
+
+**NUL Request Payload:**
+
+- GMT timestamp in %m%d%H%M%Y.%S format
+- ""
+- ""
+
+Format of X.509 signing request messages
+----------------------------------------
+
+X.509 signing request messages are sent in
+:ref:`client.pl <commmodule-client-pl>`'s main loop for each requested
+certificate.
+
+.. table:: X.509 certificate signing request header format
+
+ ==== =========================================================
+ Byte Value
+ ==== =========================================================
+ 0 Version (0x01)
+ 1 Action 0x01
+ 2 System (0x01 for :ref:`client.pl <commmodule-client-pl>`)
+ 3 Root
+ 4 Profile (see table :ref:`table-cert-profiles`)
+ 5 Message Digest Id (see table :ref:`table-md-ids`)
+ 6-7 Days in big-endian format
+ 8 Key type (``0x01`` for 'NS', ``0x00`` for others)
+ ==== =========================================================
+
+.. todo:: describe which root is identified by which root id
+
+The key type is stored in the column *keytype* of the certificate request
+table which is one of
+
+- *domaincerts*
+- *emailcerts*
+- *orgdomaincerts*
+- *orgemailcerts*
+
+.. todo:: describe what 'NS' means for key type
+
+
+**X.509 Signing Request Payload:**
+
+- "$content"
+- "$SAN"
+- "$subject"
+
+.. _table-cert-profiles:
+
+.. table:: Certificate profile ids
+
+ == ======================
+ Id Profile
+ == ======================
+ 0 Client (personal)
+ 1 Client (Organization)
+ 2 Client (Codesigning)
+ 3 Client (Machine)
+ 4 Client (ADS)
+ 5 Server (personal)
+ 6 Server (Organization)
+ 7 Server (Jabber)
+ 8 Server (OCSP)
+ 9 Server (Timestamp)
+ 10 Proxy
+ 11 SubCA
+ == ======================
+
+.. note::
+
+ :ref:`client.pl <commmodule-client-pl>` supports profiles 0, 1, 2, 4,
+ 5, 6, 8 and 9 only.
+
+.. _table-md-ids:
+
+.. table:: Message digest ids
+
+ == ==========
+ Id Algorithm
+ == ==========
+ 1 MD5
+ 2 SHA-1
+ 3 RIPE-MD160
+ 8 SHA-256
+ 9 SHA-384
+ 10 SHA-512
+ == ==========
+
+
+.. todo:: describe other request types
+
+.. _signer-response-data-format:
+
+Signer response data format specification
+=========================================
+
+.. todo:: describe signer response
+
+Protocol messages
+=================
+
+.. _signer-message-handshake:
+
+Handshake
+---------
+
+#. client sends 1 byte ``0x02`` to serial port
+#. client reads 1 byte from serial port (with a 20 second timeout)
+#. client checks whether the byte is ``0x10``
+
+.. seqdiag::
+
+ seqdiag handhake {
+ client -> server [label = "0x02"];
+ client <-- server [label = "0x10"];
+ }
+
+If anything different is received there was a protocol error and no further
+messages should be sent over the serial connection.
+
+Send data
+---------
+
+:Preconditions:
+ successful :ref:`Handshake <signer-message-handshake>`,
+ data is encoded according to the :ref:`signer-request-data-format`
+
+#. client builds byte wise xor of all data bytes into 1 byte $xor
+#. client sends concatenated $data string + xor-Byte + "rie4Ech7"
+#. client reads 1 byte (with a 5 second timeout)
+#. if received byte is ``0x11`` try again
+#. if received byte is ``0x10`` the message has been sent successfully
+
+.. seqdiag::
+
+ seqdiag with_retry {
+ client -> client [label = "xor $data"];
+ client -> server [label = "$data . $xor . \"rie4Ech7\""];
+ client <-- server [label = "0x11"];
+ client -> server [label = "$data . $xor . \"rie4Ech7\""];
+ client <-- server [label = "0x10"];
+ }
+
+If anything different is received there was a protocol error and no further
+messages should be sent over the serial connection.
+