Use the Nitrokey HSM or SmartCard-HSM with sc-hsm-embedded, mod_nss and Apache (read only module)

15-07-2016 | Remy van Elst


Table of Contents


The NitroKey HSM in a sealed package

This is a guide on using the Nitrokey HSM with sc-hsm-embedded module instead of the PC/SC daemon and OpenSC, mod_nss and the Apache webserver. This is an extension on the earlier guide, with new benchmarks. The sc-hsm-embedded module is not using a global lock like OpenSC, therefore providing better performance. The sc-hsm-embedded module is also a read only module, suitable for embedded systems. Read only also makes it more secure when deployed, even when the user pin leaks out an attacker cannot create new keypairs or delete the current ones.

The HSM allows you to store the private key for a SSL certificate inside the HSM (instead of on the filesystem), so that it can never leave the device and thus never be stolen.

The guide covers the installation of the sc-hsm-embedded module, configuration of and benchmarks from Apache with the HSM and different key sizes.

Introduction

The SmartCard-HSM

The Nitrokey HSM is an open hardware and open software device. It is a USB version of the SmartCard-HSM. Both the SmartCard-HSM as the Nitrokey HSM have sources available and are fully supported by the OpenSC project.

If you are new to the NitroKey HSM/SmartCard HSM, please also read my getting started article. It explains what the HSM is, how to set it up and how to use it with OpenSSH for example.

I have multiple articles on this nice device, so make sure to read the others as well.

To follow this article, I recommend you first read the article on the Nitrokey HSM with mod_nss and Apache I wrote earlier. The other article has more explanation and examples, this article will focus on the installation of the module and the benchmarks.

In a recent conversation I had with Andreas Schwier, one of the people behind the SmardCard-HSM project, I was informed about the sc-hsm-embedded module. Andreas told me that they, in their load tests, experienced bad performance with OpenSC because of a global lock. Their benchmarks were done with their own PKCS#11 module, which we are covering here in this article.

The sc-hsm-embedded is described as:

Light-weight, read-only PKCS#11 library for using the SmartCard-HSM in embedded systems.

So you will not be able to create keypairs with this module or write certificates to the HSM.

This guide was tested on Ubuntu 14.04 and Arch Linux with a NitroKey HSM.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

Installation of the sc-hsm-embedded module

For Arch Linux I made an AUR package of the module and an AUR package for the OpenSSL Engine.

Do note that you can install this module next to the OpenSC package. They don't conflict with one another.

On Ubuntu, install the development tools and pcsclite packages:

apt-get install automake build-essential pkg-config libtool libusb-1.0-0-dev libpcsclite-dev checkinstall

Clone the git repository:

git clone https://github.com/CardContact/sc-hsm-embedded.git sc-hsm-embedded-2.9
cd sc-hsm-embedded-2.9

Execute the following commands to generate a /.configure file using the autotools:

libtoolize --force
aclocal
autoheader
automake --force-missing --add-missing
autoconf

Continue the compilation:

./configure
make

I like to create a debian package with checkinstall instead of a make install. It's certainly not the official way to create debian packages, but it does allow you to keep a system clean and allows for easier upgrading. Also, a package is much more convinient if you are distributing it in a repo or with Ansible. Create the package:

checkinstall

The default settings should be OK, but change them as you like. The final output should be along the lines of:

**********************************************************************

 Done. The new package has been installed and saved to

 /root/repo/sc-hsm-embedded-2.9/sc-hsm-embedded_2.9-1_amd64.deb

 You can remove it from your system anytime using: 

      dpkg -r sc-hsm-embedded

**********************************************************************

On any other distro, use their native way to build a package (hi PKGBUILD) or just do a plain simple make install.

There should now be a new library on your system:

file /usr/local/lib/libsc-hsm-pkcs11.so
/usr/local/lib/libsc-hsm-pkcs11.so: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=1ede3f07f0094711931f5e25d9716622742356da, stripped

On Arch:

file /usr/lib/libsc-hsm-pkcs11.so
/usr/lib/libsc-hsm-pkcs11.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=3854bdd3a096b19c927817a486140bc2750555f8, stripped

Later in the article, place the correct path to this .so file in the configuration. Or, if you want to be able to copy and paste the commands, create a symlink:

ln -s /usr/lib/libsc-hsm-pkcs11.so /usr/local/lib/libsc-hsm-pkcs11.so

You can test the new module with their own test tool or pkcs11-tool:

pkcs11-tool --module libsc-hsm-pkcs11.so --login --pin 648219 --show-info

Output:

Cryptoki version 2.20
Manufacturer     CardContact (www.cardcontact.de)
Library          SmartCard-HSM R/O with PC/SC (ver 2.8)
Using slot 1 with a present token (0x5)

Compared with the opensc-pkcs11 module:

 pkcs11-tool --module opensc-pkcs11.so --login --show-info --pin 648219

Output:

Cryptoki version 2.20
Manufacturer     OpenSC Project
Library          OpenSC smartcard framework (ver 0.16)
Using slot 1 with a present token (0x4)

You will notice that most operations with pkcs11-tool, like creating a keypair, will fail:

pkcs11-tool --module libsc-hsm-pkcs11.so --login --pin 648219 --keypairgen --key-type rsa:1024 --id 1 --label "HSM RSA Key Remy"

Output:

Using slot 1 with a present token (0x5)
error: Generate RSA mechanism not supported

Aborting.

This is expected since it's a read only module. Read only also makes it more secure when deployed, even when the user pin leaks out an attacker cannot create new keypairs or delete the current ones.

A delete operation will not give any error messages:

pkcs11-tool --module libsc-hsm-pkcs11.so --login --pin 648219 --delete-object --type privkey --id 3

Output:

Using slot 1 with a present token (0x5)

But, if you list the objects before and after you will see that the key you tried to delete is still there.

Just listing data does work:

pkcs11-tool --module libsc-hsm-pkcs11.so --login --pin 648219 --list-objects --id 01

Output:

Using slot 1 with a present token (0x5)
Public Key Object; RSA 2048 bits
  label:      1024cert
  ID:         03
  Usage:      encrypt, verify
Certificate Object, type = X.509 cert
  label:      1024cert
  ID:         03

Reading the data does work as well:

pkcs11-tool --module libsc-hsm-pkcs11.so --login --pin 648219 --read-object --id 1 --type cert | base64

Output:

Using slot 1 with a present token (0x5)
MIIDcjCCAloCCQDIi8zcoGtnejANBgkqhkiG9w0BAQsFADB7MQswCQYDVQQGEwJOTDEVMBMGA1UE
[...]
qJ49qqLd5I24yKXxh9qTYrXDxHPAExqXHnwydXiDRQ==

Creating the keys and certificates

We will now start with creating the keypairs in the HSM and generating the certificates.

Please do read the article on the Nitrokey HSM with mod_nss and Apache to find out how to fully configure the NitroKey HSM/SmartCard HSM with mod_nss. This is a more compact version without all the explanation.

I did create three test keypairs, one RSA 1024 bit, one RSA 2048 bit and one EC pair. They have ID 1, 2 and, suprisingly, 3. If you want to follow along, also create the keys. Start with the 1024 bit key:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --keypairgen --key-type rsa:1024 --id 1 --label "httpd1024" 

Output:

Key pair generated:
Private Key Object; RSA 
  label:      httpd1024
  ID:         01
  Usage:      decrypt, sign, unwrap
Public Key Object; RSA 1024 bits
  label:      httpd1024
  ID:         01
  Usage:      encrypt, verify, wrap

2048 bit key:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --keypairgen --key-type rsa:2048 --id 2 --label "httpd2048" 

Output:

Key pair generated:
Private Key Object; RSA 
  label:      httpd2048
  ID:         02
  Usage:      decrypt, sign, unwrap
Public Key Object; RSA 2048 bits
  label:      httpd2048
  ID:         02
  Usage:      encrypt, verify, wrap

And the EC key:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --keypairgen --key-type EC:prime256v1 --id 3 --label "httpdECprime256v1"

Output:

Private Key Object; EC
  label:      httpdECprime256v1
  ID:         03
  Usage:      sign, derive
Public Key Object; EC  EC_POINT 256 bits
  EC_POINT:   044104ca3ff52e48bd85f3878ef8e1b59498c9fcc7741a1e547f2849ff4a5f716d96ea49ab8f25c61e7b4ad899fd3df8996767e70672ceb9297758695810fbc30ba660
  EC_PARAMS:  06082a8648ce3d030107
  label:      httpdECprime256v1
  ID:         03
  Usage:      verify

We need to generate (self signed) certificates for all of these keypairs. We will use OpenSSL with a configfile do to that. See my tutorial on the Nitrokey if you want to generate a CSR and what to put in hsm.conf.

I did have some issues when providing the slot and ID, OpenSSL gave errors like these:

engine "pkcs11" set.
Invalid slot number: 1
PKCS11_get_private_key returned NULL
cannot load Private Key from engine
140378622912152:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:eng_pkey.c:124:
unable to load Private Key

Turns out, I was using the wrong slot number:

pkcs11-tool --list-slots

Output:

Available slots:
Slot 0 (0x0): Lenovo Integrated Smart Card Reader 00 00
  (empty)
Slot 1 (0x4): Nitrokey Nitrokey HSM (010000000000000000000000) 01 00
  token label        : SmartCard-HSM (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : rng, login required, PIN initialized, token initialized
  hardware version   : 24.13
  firmware version   : 2.0
  serial num         : DENK0100186

I was using slot 1, but that did not work. Slot 4 (0x4) did work. You can provide the slot and ID to OpenSSL in multiple formats:

supported formats: <id>, <slot>:<id>, id_<id>, slot_<slot>-id_<id>, label_<label>, slot_<slot>-label_<label>
where <slot> is the slot number as normal integer,
and <id> is the id number as hex string.
and <label> is the textual key label string.

Generate the 1024 bit RSA certificate with ID 1:

OPENSSL_CONF=./hsm.conf openssl req -engine pkcs11 -keyform engine -new -key slot_4-id_1 -nodes -days 3560 -x509 -sha256 -out "rsa1024.tst.raymii.org.pem" -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=rsa1024.tst.raymii.org"

The 2048 bit RSA certificate with ID 2:

OPENSSL_CONF=./hsm.conf openssl req -engine pkcs11 -keyform engine -new -key slot_4-id_2 -nodes -days 3560 -x509 -sha256 -out "rsa2048.tst.raymii.org.pem" -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=rsa2048.tst.raymii.org"

The EC certificate:

OPENSSL_CONF=./hsm.conf openssl req -engine pkcs11 -keyform engine -new -key slot_4-id_3 -nodes -days 3560 -x509 -sha256 -out "httpdECprime256v1.tst.raymii.org.pem" -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=httpdECprime256v1.tst.raymii.org"

Do note that you can also generate a CSR and submit that to your certificate provider The getting started article covers that as well.

Convert the PEM certificates into DER format, since DER is what the HSM uses:

openssl x509 -in rsa1024.tst.raymii.org.pem -out rsa1024.tst.raymii.org.der -outform der

openssl x509 -in rsa2048.tst.raymii.org.pem -out rsa2048.tst.raymii.org.der -outform der

openssl x509 -in httpdECprime256v1.tst.raymii.org.pem -out httpdECprime256v1.tst.raymii.org.der -outform der

Load the DER certificates into the HSM together with the keys:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --write-object rsa1024.tst.raymii.org.der --type cert --id 1 --label 'rsa1024'

Output:

Using slot 1 with a present token (0x4)
Created certificate:
Certificate Object, type = X.509 cert
  label:      rsa1024
  ID:         01

THe 2048 bit RSA key:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --write-object rsa2048.tst.raymii.org.der --type cert --id 2 --label 'rsa2048'

Output:

Created certificate:
Certificate Object, type = X.509 cert
  label:      rsa2048
  ID:         02

The EC key:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --write-object httpdECprime256v1.tst.raymii.org.der --type cert --id 3 --label 'ECprime256v1'

Output:

Created certificate:
Certificate Object, type = X.509 cert
  label:      ECprime256v1
  ID:         03

My HSM now has the following configuration:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --list-objects 

Output:

Using slot 1 with a present token (0x4)
Private Key Object; RSA 
  label:      rsa1024
  ID:         01
  Usage:      decrypt, sign, unwrap
Certificate Object, type = X.509 cert
  label:      rsa1024
  ID:         01
Public Key Object; RSA 1024 bits
  label:      rsa1024
  ID:         01
  Usage:      encrypt, verify
Certificate Object, type = X.509 cert
  label:      rsa2048
  ID:         02
Public Key Object; RSA 1024 bits
  label:      rsa2048
  ID:         02
  Usage:      encrypt, verify
Private Key Object; RSA 
  label:      rsa2048
  ID:         02
  Usage:      decrypt, sign, unwrap
Private Key Object; EC
  label:      ECprime256v1
  ID:         03
  Usage:      sign, derive
Certificate Object, type = X.509 cert
  label:      ECprime256v1
  ID:         03
Public Key Object; RSA 1024 bits
  label:      ECprime256v1
  ID:         03
  Usage:      encrypt, verify

Configuring Apache and mod_nss

Make sure that you have Apache and mod_nss installed:

apt-get install apache2 libapache2-mod-nss

Enable mod_nss with the following command:

a2enmod nss

Disable mod_ssl:

a2dismod ssl

On other Linux distro's you might need to manually change config files. On my Arch installation I had to add the following to /etc/http/conf/http.conf:

Include conf/extra/nss.conf

Configuring the NSS certificate database

NSS requires a certificate database. We also need to tell NSS to load the pkcs11 module.

Create the database folder:

mkdir -p /etc/nss/db/

Create a new NSS database:

certutil -N -d /etc/nss/db/

It will ask you for a password. Enter a secure one, or, when testing, just pres RETURN twice. Output:

Enter a password which will be used to encrypt your keys.
The password should be at least 8 characters long,
and should contain at least one non-alphabetic character.

Enter new password: 
Re-enter password: 

Add the HSM module to NSS. This is where things differ from the other tutorial. We add the sc-hsm-embedded module here and give it a different name:

modutil -add hsm -libfile /usr/lib/libsc-hsm-pkcs11.so -dbdir /etc/nss/db/

Enable the module:

modutil -enable hsm -dbdir /etc/nss/db/

The other module we named pkcs11. If you want to, you could add them both and switch if needed.

We need the Token Name to put in the mod_nss configuration. Find it:

modutil -dbdir /etc/nss/db/ -list 

Output, snipped:

  2. hsm
  library name: /usr/lib/libsc-hsm-pkcs11.so
   slots: 2 slots attached
  status: loaded

   slot: Nitrokey Nitrokey HSM (010000000000000000000000) 01 00
  token: SmartCard-HSM
-----------------------------------------------------------

The name in this case is SmartCard-HSM. List all the certificates with the certutil command:

certutil -d /etc/nss/db/ -h 'SmartCard-HSM' -L 

Output:

SmartCard-HSM:ECprime256v1                                   u,u,u
SmartCard-HSM:rsa2048                                        u,u,u
SmartCard-HSM:rsa1024                                        u,u,u

You can view the certificates with the following command:

certutil -d /etc/nss/db -h 'SmartCard-HSM' -L -n "SmartCard-HSM:rsa1024" 

Output:

Enter Password or Pin for "SmartCard-HSM": 648219
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            00:ab:93:b2:05:d0:f7:70:6e
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=rsa1024.tst.raymii.org,OU=IT Dept,O=Sparkling Network,L=R
            otterdam,ST=Zuid Holland,C=NL"
        Validity:
            Not Before: Fri Jul 15 07:38:14 2016
            Not After : Tue Apr 14 07:38:14 2026
        Subject: "CN=rsa1024.tst.raymii.org,OU=IT Dept,O=Sparkling Network,L=
            Rotterdam,ST=Zuid Holland,C=NL"

(-n names the specific certificate we want to list/show. -a enables ASCII output.)

Place the PIN for the slot in this file:

cat /etc/nss/db/pin.txt
SmartCard-HSM:648219

The format is: tokenname:pin.

Also make sure to give the correct permissions to the NSS db:

chown -R http:root /etc/nss/db/

Apache mod_nss configuration

Make sure you have disabled mod_ssl in the Apache configuration and have enabled mod_nss. This is my nss.conf file:

# grep -v -e "#" -e '^$' /etc/httpd/conf/extra/nss.conf
LoadModule nss_module modules/libmodnss.so
Listen 8443
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl
NSSPassPhraseDialog file:/etc/nss/db/pin.txt
NSSPassPhraseHelper /usr/bin/nss_pcache
NSSEnforceValidCerts off
NSSSessionCacheTimeout 100
NSSSession3CacheTimeout 86400
NSSRandomSeed startup builtin
NSSRenegotiation off
NSSRequireSafeNegotiation off
<VirtualHost _default_:8443>
  DocumentRoot "/etc/httpd/htdocs"
  ServerName rsa1024.tst.raymii.org:8443
  ErrorLog /etc/httpd/logs/error_log
  TransferLog /etc/httpd/logs/access_log
  LogLevel info
  NSSEngine on
  NSSCipherSuite ALL
  NSSProtocol TLSv1.0,TLSv1.1,TLSv1.2
  NSSNickname "SmartCard-HSM:rsa1024"
  NSSCertificateDatabase /etc/nss/db
  <Files ~ "\.(cgi|shtml|phtml|php3?)$">
      NSSOptions +StdEnvVars
  </Files>
  <Directory "/var/www/cgi-bin">
      NSSOptions +StdEnvVars
  </Directory>
</VirtualHost>  

On Arch I had to comment out NSSSessionCacheSize and replace NSSPassPhraseHelper /usr/libexec/nss_pcache with NSSPassPhraseHelper /usr/bin/nss_pcache. For the testing certificate I also had to add NSSEnforceValidCerts off. The rest is just the example config.

Make sure the certificate name is correct here:

NSSNickname "SmartCard-HSM (UserPIN):httpdcert"

For the two other keys, change the two variables when testing the different keys

  ServerName rsa1024.tst.raymii.org:8443
  NSSNickname "SmartCard-HSM:rsa1024"

Respectively to their other names.

Restart the webserver to make the configuration active.

Benchmarking 2048 bit RSA

The Nitrokey HSM is not a fast HSM. I've worked with HSM's that are capable of multiple hundreds of signs per second, and this HSM is not one that can do that. But, as we know, that's not the intended target for the device. It's meant for safe and secure key storage, for example to encrypt files or protect SSH or S/MIME keys. I did a few siege benchmarks, and as you can see it is quite slow.

The page that was being loaded is a plain text file containing only the text Jeej it works!.

Remember to update your httpd config if needed:

ServerName rsa2048.tst.raymii.org:8443
NSSNickname "SmartCard-HSM:rsa2048"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             36 hits
Availability:             100.00 %
Elapsed time:             29.10 secs
Data transferred:         0.00 MB
Response time:            1.63 secs
Transaction rate:         1.24 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              2.02
Successful transactions:  36
Failed transactions:      0
Longest transaction:      5.13
Shortest transaction:     0.73

10 concurrent users:

siege -c10 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             38 hits
Availability:             100.00 %
Elapsed time:             29.97 secs
Data transferred:         0.00 MB
Response time:            4.57 secs
Transaction rate:         1.27 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              5.80
Successful transactions:  38
Failed transactions:      0
Longest transaction:      11.15
Shortest transaction:     0.74

20 concurrent users:

siege -c20 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             37 hits
Availability:             100.00 %
Elapsed time:             29.22 secs
Data transferred:         0.00 MB
Response time:            9.62 secs
Transaction rate:         1.27 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              12.18
Successful transactions:  37
Failed transactions:      0
Longest transaction:      20.47
Shortest transaction:     1.65

60 benchmark mode:

siege -c60 -b -t30S https://127.0.0.1:8443

Result:

This specific test failed, where the NSS log has the following errors:

Broken pipe: SSL library error -5992 writing data
SSL input filter read failed.
SSL Library Error: -8023 Unknown
AH01382: Request header read timeout

Benchmarking 1024 bit RSA

A product folder mentions that the HSM should be a bit faster with smaller keys. Now do note that it's not recommended to use a 1024 bit key in production. But, this is just a benchmark test.

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://127.0.0.1:8443

Result:

Lifting the server siege...
Transactions:             49 hits
Availability:             100.00 %
Elapsed time:             29.38 secs
Data transferred:         0.00 MB
Response time:            0.30 secs
Transaction rate:         1.67 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              0.50
Successful transactions:  49
Failed transactions:      0
Longest transaction:      1.09
Shortest transaction:     0.22

10 concurrent users:

siege -c10 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             109 hits
Availability:             100.00 %
Elapsed time:             29.57 secs
Data transferred:         0.00 MB
Response time:            0.56 secs
Transaction rate:         3.69 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              2.06
Successful transactions:  109
Failed transactions:      0
Longest transaction:      2.39
Shortest transaction:     0.22

20 concurrent users:

siege -c20 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             134 hits
Availability:             100.00 %
Elapsed time:             29.41 secs
Data transferred:         0.00 MB
Response time:            2.02 secs
Transaction rate:         4.56 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              9.19
Successful transactions:  134
Failed transactions:      0
Longest transaction:      8.12
Shortest transaction:     0.23

60 benchmark mode:

siege -c60 -b -t30S https://127.0.0.1:8443

Result:

Transactions:             86 hits
Availability:             66.15 %
Elapsed time:             29.09 secs
Data transferred:         0.00 MB
Response time:            11.14 secs
Transaction rate:         2.96 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              32.94
Successful transactions:  86
Failed transactions:      44
Longest transaction:      22.15
Shortest transaction:     0.87

Benchmarking the EC prime256v1 key

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:                     55 hits
Availability:                 100.00 %
Elapsed time:                  29.26 secs
Data transferred:               0.00 MB
Response time:                  0.32 secs
Transaction rate:               1.88 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.60
Successful transactions:          55
Failed transactions:               0
Longest transaction:            1.18
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             96 hits
Availability:             100.00 %
Elapsed time:             29.04 secs
Data transferred:         0.00 MB
Response time:            0.76 secs
Transaction rate:         3.31 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              2.52
Successful transactions:  96
Failed transactions:      0
Longest transaction:      3.15
Shortest transaction:     0.25

20 concurrent users:

siege -c20 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             117 hits
Availability:             100.00 %
Elapsed time:             29.98 secs
Data transferred:         0.00 MB
Response time:            2.57 secs
Transaction rate:         3.90 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              10.02
Successful transactions:  117
Failed transactions:      0
Longest transaction:      7.42
Shortest transaction:     0.49

30 concurrent users:

siege -c30 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             108 hits
Availability:             91.53 %
Elapsed time:             29.09 secs
Data transferred:         0.00 MB
Response time:            5.09 secs
Transaction rate:         3.71 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              18.90
Successful transactions:  108
Failed transactions:      10
Longest transaction:      19.20
Shortest transaction:     0.51

40 concurrent users:

siege -c40 -d5 -t30S https://127.0.0.1:8443

Result:

Transactions:             105 hits
Availability:             98.13 %
Elapsed time:             29.52 secs
Data transferred:         0.00 MB
Response time:            6.43 secs
Transaction rate:         3.56 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              22.86
Successful transactions:  105
Failed transactions:      2
Longest transaction:      20.14
Shortest transaction:     0.50

60 benchmark mode:

siege -c60 -b -t30S https://127.0.0.1:8443

Result:

Lifting the server siege...
Transactions:             58 hits
Availability:             56.31 %
Elapsed time:             29.42 secs
Data transferred:         0.00 MB
Response time:            10.78 secs
Transaction rate:         1.97 trans/sec
Throughput:               0.00 MB/sec
Concurrency:              21.25
Successful transactions:  58
Failed transactions:      45
Longest transaction:      19.18
Shortest transaction:     2.12

Benchmarking with PHP and Wordpress

I setup a default PHP (7), MariaDB and Wordpress install with this mod_nss Apache and ran the some benchmarks on that as well to show how a 'real' site performs. This with the EC key we also used above. Just using the site, browsing with firefox/chrome and updating/editing posts also works fine, feels snappy enough.

20 concurrent users:

siege -c20 -d5 -t30S 'https://127.0.0.1:8443/wordpress/'

Result:

Transactions:             132 hits
Availability:             92.96 %
Elapsed time:             29.64 secs
Data transferred:         1.13 MB
Response time:            3.83 secs
Transaction rate:         4.45 trans/sec
Throughput:               0.04 MB/sec
Concurrency:              17.05
Successful transactions:  132
Failed transactions:      10
Longest transaction:      8.15
Shortest transaction:     0.44

Comparing and conclusion

Most of the benchmarks are comparable to the benchmarks with the OpenSC module. Except for the EC keypair, since that failed with the OpenSC module. I did try to switch to the other NSS certificate DB (with the opensc module) but that failed. So, if you want to use an EC keypair, you need to use this module.

As said in the other article, the HSM is not a fast HSM, but that is not the target. It's target is to enable everyone to have secure key storage for a reasonable price, instead of thousands of dollars for a big-company-name HSM.

This read only module is very nice if you are deploying this HSM in untrusted places, for example at a client or in a datacenter. It also seperates the HSM management from the actual usage by the webserver. If you need to renew the certificate or generate a new keypair, you need to take the device to a machine with OpenSC and all the tools installed (which can be an offline machine), instead of the webserver being able to do this. If the webserver ever gets compromised, they will not be able to edit/delete or otherwise write to the HSM.

One of the next articles will be about a HSM cluster with multiple HSM's, which will result in a speed increase.


Tags: apache, cryptoki, hsm, mod_nss, nitrokey, nitrokey-hsm, openssl, pkcs11, safenet, sc-hsm-embedded, smartcard, smartcard-hsm,