Skip to main content

Raymii.org Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Nitrokey HSM/SmartCard-HSM and Raspberry Pi web cluster

Published: 01-08-2016 | Author: Remy van Elst | Text only version of this article


Table of Contents


The 3-Pi HSM cluster to be used for the cluster articles

This article sets up a Nitrokey HSM/SmartCard-HSM web cluster and has a lot ofbenchmarks. This specific HSM is not a fast HSM since it's very inexpensive andtargeted at secure key storage, not performance. But, what if you do want moreperformance? Then you scale horizontally, just add some more HSM's and aloadbalancer in front.

You want to put your private key material inside an HSM because it cannot bestolen that way. A HSM does not allow key material to be exported, so nobody cansecretly copy the keys and use them without your knowledge. If the HSM istampered with, it will also wipe itself, so brute forcing it will not work.

The cluster consists of Raspberry Pi's and Nitrokey HSM's and SmartCard-HSM's,softwarewise we use Apache, mod_nss and haproxy.

This is the first time I had an actual use case for Raspberry Pi's in a cluster,and I really enjoyed doing it. I might even, in the future, add some more Pi'son top and do some more benchmarks. But I already spent about two weeks workingon this single article so three was just fine for now.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

Do note that two devices were sponsored for this article.

We benchmark a small HTML file and a Wordpress site using:

We do these benchmarks with the OpenSC module and with the sc-hsm-embeddedmodule to see if that makes any difference.

The full raw results are provided at the end of the article. I first talk a bitmore about the HSM's, the cluster setup and the issues I had with the RaspberryPi's. Then we set up the three HSM devices and the load balancer. Finally,before the raw results, we have nice charts and interpretation of the charts,plus an unexpected conclusion.

Introduction

The Nitrokey HSM and the SmartCard-HSM

The Nitrokey HSM is an open hardware and open software device. It is a USBversion of the SmartCard-HSM. Both the SmartCard-HSM as the NitrokeyHSM have sources available and are fully supported by the OpenSCproject.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

The SmartCard-HSM

If you are new to the NitroKey HSM/SmartCard HSM, please also read my gettingstarted article. It explains what the HSM is, how to set it up and how touse it with OpenSSH for example.

I have multiple articles on this nice device, so make sure to read theothers as well.

How many HSM's?

Three Nitrokey's in their bags

This guide uses three Nitrokey HSM devices. I've generated three keypairs on oneof the HSM's, one RSA 1024, one RSA 2048 and one EC key, just as we did in themod_nss tutorial. Please consult that article first, since the keygeneration and certificate loading part is not included in this guide.

I'll refer to the HSM where the DKEK was initialized and the keys were generatedas HSM 1. The other two HSM's are referred two as HSM 2 and HSM 3.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

This guide uses three Raspberry Pi 3 comuters, wired network with Raspbiantesting (2016-07-29) and with the three HSM devices plugged in.

At first I tried to use the HSM's on the host computer, with different virtualhosts. That works when using the OpenSC module, but not with the sc-hsm-embeddedmodule. That module doesn't support token labels, yet, so there was no wayto distinguish between them. I did try to setup three different NSS databaseswhere the HSM's were only enabled by ID but that still resulted in the three ofthem being used. To make all tests equal, I went for a second option.

That second option was using three different virtual machines on the same PCwith VirtualBox and USB passthrough in VirtualBox:

That however gave all kinds of errors with the HSM's in use. At first it workedjust fine with mod_nss but whenever I tried to do more than 5 concurrentactions on the HSM the connection was lost and it became unresponsive, to thepart that even sc-hsm-tool did not recognize the HSM and a VM reboot wasrequired. So that wasn't a viable solution either. On to the third option itwas.

Cluster setup

THe final solution was using three different computers. Since I wanted to keepit simple, fair and not too expensive I decided to buy three Raspberry Pi 3's. Ibought them from the dutch store Kiwi-Electronics including two stackablecases. (These links are not affiliate links, just had a great experiencethere. Their order confirmation lists the ordering IP, and in my case it was myIPv6 address. Yay +1 for them!). The case comes from ModMyPi and I mighteven just recreate it in Inkscape and use my lasercutter in the future forexpansion.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

The Raspberry Pi 3 Model B's have the following specs:

They also have Wifi and bluetooth but I did not use that. All the benchmarkswere done over the wired network. The OS is Raspbian Testing. Since thecurrent normal Raspbian ships OpenSC 0.14 which doesn't support the SmartCard-HSM/Nitrokey HSM an upgrade was required. The current testing ships with OpenSC0.16, which does work with the SmartCard-HSM/Nitrokey HSM.

To combine the three machines into one single service I used haproxy in TCPmode. haproxy is a very fast and scalable HTTP(s)/TCP load balancer. I've usedit in production for many years now and have been happy with it ever since.

I did also try nginx since that also supports TCP and UDP loadbalancing. The results were very comparable, so I think the load balanceris not the limiting factor here.

The software stack used on Raspbian Testing is the following:

I'm using mpm_event and php-fpm instead of mpm_prefork and mod-phpbecause of issues with the initialization of the HSM by all the workers. Seethe mailinglist thread here for more information. Otherwise all kinds oferrors like SSL Library Error: -8152 The key does not support the requestedoperation and SSL Library Error: -8023 Unknown and SSL Library Error: -12216Attempt to write encrypted data to underlying socket failed and SSL inputfilter read failed. occur.

To install all the software and configure everything except for the HSM keygeneration I've created a set of Ansible playbooks. Because nobody wants tohandcraft three special snowflakes. I've burned through 7 MicroSD cards beforegetting to a working setup. I also tried to use Arch and Ubuntu 16.04 but thoseimages all had their own instabillites, so I just settled on Raspbian testing.

I might put the playbooks on here someday, but now they are intertwined to muchwith my personal playbooks to make sense without it.

How do you keep the HSM's apart? Well, I've used a very high-tech solution forthat, namely using three different coloured key-cords:

Stickers might also be an option.

Initialize the new HSM's

Plug in HSM 2.

If you execute sc-hsm-tool it will notify you that the new HSM has never beeninitialized:

$ sc-hsm-tool 

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00Version              : 2.0SmartCard-HSM has never been initialized. Please use --initialize to set SO-PIN and user PIN.

HSM 1 was initialized with one DKEK share. We initialize the new HSMs and importthe DKEK share, to make sure the key backups work. We also give it a differentlabel.

sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 648219 --dkek-shares 1 --label 'hsm2'

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00

Import the DKEK share:

sc-hsm-tool --import-dkek-share dkek-share-1.pbe

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00Enter password to decrypt DKEK share : 123456789Deciphering DKEK share, please wait...DKEK share importedDKEK shares          : 1DKEK key check value : 0FB85F69F6EBF256

Repeat the above process for HSM 3 and any more HSM's you have. Make sure togive them descriptive labels. Unplug the other HSM's when initializing one, tomake sure you don't overwrite the wrong one.

The new HSM's are now initialized with the same DKEK as the old HSM. The nextstep is to securely backup the existing keys from the old HSM and import theminto the new HSMs. This works because we're using the same DKEK.

Backup and restore the keys

Plug in HSM 1.

Wrap (export) the keys on HSM 1:

sc-hsm-tool --wrap-key wrap-key-1.bin --key-reference 1 --pin 648219sc-hsm-tool --wrap-key wrap-key-2.bin --key-reference 2 --pin 648219sc-hsm-tool --wrap-key wrap-key-3.bin --key-reference 3 --pin 648219

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00

Plug in HSM 2.

Unwrap (import) the keys on the HSM 2:

sc-hsm-tool --unwrap-key wrap-key-1.bin --key-reference 1 --pin 648219sc-hsm-tool --unwrap-key wrap-key-2.bin --key-reference 2 --pin 648219sc-hsm-tool --unwrap-key wrap-key-3.bin --key-reference 3 --pin 648219

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00Wrapped key contains:  Key blob  Private Key Description (PRKD)  CertificateKey successfully imported

Repeat this for HSM 3.

With all the keys imported and three HSM's plugged in, pkcs11-tool gives somenice output:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --list-slotsAvailable slots:Slot 0 (0x0): Nitrokey Nitrokey HSM (010000000000000000000000) 00 00  token label        : hsm3 (UserPIN)  token manufacturer : www.CardContact.de  token model        : PKCS#15 emulated  token flags        : rng, login required, PIN initialized, token initialized  hardware version   : 24.13  firmware version   : 2.0  serial num         : DENK0100485Slot 1 (0x4): Lenovo Integrated Smart Card Reader 01 00  (empty)Slot 2 (0x8): Nitrokey Nitrokey HSM (010000000000000000000000) 02 00  token label        : hsm1 (UserPIN)  token manufacturer : www.CardContact.de  token model        : PKCS#15 emulated  token flags        : rng, login required, PIN initialized, token initialized  hardware version   : 24.13  firmware version   : 2.0  serial num         : DENK0100186Slot 3 (0xc): Nitrokey Nitrokey HSM (010000000000000000000000) 03 00  token label        : hsm2 (UserPIN)  token manufacturer : www.CardContact.de  token model        : PKCS#15 emulated  token flags        : rng, login required, PIN initialized, token initialized  hardware version   : 24.13  firmware version   : 2.0  serial num         : DENK0100436

If you want to do operations on a specific device you can add the --slotparameter to the pkcs11-tool command. For example, to generate a key just onHSM 2 (slot 3/c):

$ pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --keypairgen --key-type EC:prime256v1 --slot c --id 10 --label "ect"

Raspberry Pi setup

The Raspberry Pi's are on the network via a wired connection. They run RaspbianTesting because of the OpenSC version.

The following steps should be done on the three Pi's:

Please consult the mod_nss opensc guide or the mod_nss with sc-hsm-embedded guide for the specific setup and repeat that on all the RaspberryPi's. I'm not going to cover the setup here any further.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

Apache setup

Remember to not use mod_php and mpm_prefork. I used mpm_event and php-fpm (PHP 7).

Here is the mpm_event configuration:

    StartServers             20    MinSpareThreads          250    MaxSpareThreads          500    ThreadLimit              64    ThreadsPerChild          25    MaxRequestWorkers        500    MaxConnectionsPerChild   150

I couldn't get fastcgi to run, but it seems Apache uses its own module now(proxy_fcgi).

Make sure to restart Apache after configuring.

Repeat this step on all the VM's.

Configure haproxy

Make sure you have haproxy installed:

apt-get install haproxy

I'm using version 1.6.6. I've got a few other articles on HAproxy ifyou're interested. HAproxy does not run on the Raspberry Pi's, but on my localhost (A Lenovo Thinkpad x240, i5, 8GB RAM, Arch linux), so don't install this inthe VM's.

HAproxy can do many things in http mode, but we're not using that. We will beusing tcp mode with a roundrobin configuration. This means that whenrequests come in, haproxy proxy's the TCP connection to the backends. Requestone goes to hsm1, request two to hsm 2, request 3 to hsm 3 and request 4 goes tohsm 1. This way the server can handle more concurrent requests. This is my basichaproxy configuration file, /etc/haproxy/haproxy.cfg:

global    maxconn     20000    log         hsmcluster.nl local0    user        haproxy    chroot      /usr/share/haproxy    pidfile     /run/haproxy.pid    daemonfrontend hsm    bind *:443    mode tcp    default_backend hsm    timeout client 1mbackend hsm    mode tcp    balance roundrobin    timeout connect 10s    timeout server 1m    server hsm01 10.0.0.106:8443    server hsm02 10.0.0.107:8443    server hsm03 10.0.0.108:8443

My VM's have the 10.0.0.106, 107 and 108 addresses.

Restart haproxy after changing the config.

In my hosts file I've setup the domain hsmcluster.nl on localhost.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

NGINX in TCP mode

If you prefer to use NGINX then you can use this example configuration:

worker_processes  1;events {    worker_connections  1024;}stream {    server {      listen 443;      proxy_pass hsm_backend;    }    upstream hsm_backend {      server 10.0.0.106:443;      server 10.0.0.100:443;      server 10.0.0.105:443;    }}

Charts and result interpretation

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

Here below I'll look into the various aspects and results of the benchmark. Thelast section of the article gives you the raw numbers/benchmark siege results toplay around yourself. This is the more readable part if you're not into numbercrunching.

Siege result parsing

Siege gives you different types of metrics. We do the same benchmark over andover so we're interesed in some, not all. The time for example is 30 secondseverywhere. Here is, from the manual, the explanation of the metrics weuse.

mod_ssl. no HSM

As we can see here the 1024 bit RSA key is the fastest. 2048 bit RSA is just atidbit slower, but still acceptable. 4096 bit RSA keys take a huge dump down intransactions and the response time and concurrency go up.

8192 bit RSA keys slow down to a grinding halt, huge response time, lowtransaction rate. It might be super secure, but super slow as well.

Same goes for the EC prime256v1 keys. Most of the time EC keys are faster, butit seems the Pi's have trouble with it, just as much as with the 8192 bit RSAkeys.

1024 bit RSA key

We can see that the HSM is around 8 times slower than using regular mod_ssl.But, we knew that already. Adding a HSM to the loadbalancer doubles theperformance, and adding two HSM's to the loadbalancer triples it, as we wouldexpect. So, going down that route, if we have 8 HSM's, it would be just as fastas regular mod_ssl.

Something that caught my eye was that with one HSM the OpenSC module wasfaster. (Remember, best of three for the tests, every test is done three times,best result is kept.). When using multiple HSM's we see a small gain intransactions when using sc-hsm-embedded. I'm not quite sure why that is, butit's something that steps out.

But, as we all know, using an 1024 bit RSA key is considered insecure. So don'tdo that in production.

Here is the same data with only the HSM's so that you can see the differencesbetter between OpenSC and sc-embedded-hsm:

2048 bit RSA key

2048 bit keys are way harder for the HSM, but not at all for regular mod_ssl.In this case it is about 26 times slower when using a HSM, comparing to no HSM.Here we see again that the transaction rate is doubled and trippled when addingHSM's to the cluster. The same thing here with sc-hsm-embedded, when used with 1HSM it's slower, but when scaling up it becomes a little bit faster.

Here is the same data with only the HSM's so that you can see the differencesbetter between OpenSC and sc-embedded-hsm:

prime256v1 EC key

I think the Pi's have trouble with the prime256v1 EC key. When using just oneHSM, the performance almost doubles. When we use three HSM's the performance isalmost as good as with the 1024 bit RSA key. In all cases OpenSC was slower thanthe sc-hsm-embedded module.

What suprises me the most is that EC algorithms are supposed to be faster thenRSA. It might be that non-Pi hardware has the AES-NI extension or something.

Different keysizes, 1 HSM

This is an interesting one. We see that the EC prime256v1 key is almost as fastas the RSA 1024 bit key. Also, OpenSC is slower here with the EC keypair. Mostbrowsers support prime256v1, also named NIST curve P-256. There however aresome concerns since the NSA is involved. Keep that in mind. Still, thefastest modern algorithm in this HSM.

Burst mode

The burst mode benchmark is different than the above ones. The above tests takea random amount of seconds between 1 and 5 and delay each connection thatamount. This gives you a more realistic test than when your just hammering everysecond. Hammering every second is a performance measure, because you can thenmeasure how many connections your server could handle at peak times.

This test fires of 60 connections for half a minute and doesnt take any timebetween them. Just bang bang bang. The results differ from the 20 connectiontest because the HSM then has some time to recover, so to say, betweenconnections. The more HSM's you add to the cluster, the more time each one willhave to recover, the better your tests will be.

The burst benchmark doesn't hit the wordpress site but the small text file.

The chart above is for an 1024 bit RSA key. We see that without the HSM it isthe fastest, topping a whopping 7000+ transactions. Here is the picture withoutthat, so make it more clear what the differences with HSM are.

Same results as above, adding more HSM's gives better performance. Note that sc-hsm-embedded is only faster in the 3 HSM test.

This is the score for 2048 bit keys, including the test without the HSM:

Same as above, more HSM makes stuff faster, without the HSM we see an enourmusspeed bump. Here's the graph without the last part:

Strange, sc-embedded-hsm is only faster here with 2 HSM's and OpenSC wassignificantly lower with 2 HSM's. I did rone this specific test again a fewtimes but all gave comparable results.

Last but not least, the prime256v1 EC key. This is the only time where the HSMis faster, I suspect because it provides offloading (the Pi lacks cryptohardware?).

Using the HSM here almost doubles the performance, using three HSM's you getalmost 6 times the performance of no HSM. I did not expect this at all, but I dofind it awesome.

Conclusion

In general this was what I expected, except for the EC part. As said at thestart of the article, the Nitrokey/SmartCard-HSM is not built for this use case,although it works absolutely fine, just a bit slower.

I've browsed the wordpress site, did some searching, installed some plugins(slider, contact form etc) and created a few blog posts, which all works justfine. I have 500/500 mbit fiber at home so I do notice the delay, but theperformance is comparable to a location with regular DSL and a 8/2 mbit speed.Which means I'm spoiled and most people will not notice the difference.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10credit). (referral link)

Below I'll talk more about something I tried first for the graphs and then giveyou more information on the benchmarks and the raw results.

Chart creation

I first tried to use GNUPlot to create charts based on the siege log file withthe following gnuplot file:

set term png truecolor size 600,600set output "data.png"set title "2048 bit RSA key"set boxwidth 1 relativeset gridset key outside;set key top;set style fill transparent solid 0.5 border rgb"black"set style data histogramset style fill solid borderset style histogram clusteredplot for [COL=2:4] 'siegedata' using COL:xticlabels(1) title columnheader 

The siege log is transformed to a usable datafile using the following commands:

awk -F, '{print $2":"$5":"$8}' siege.log  | awk '{print NR-1"-PI:",$0}' | sed 's/\s\+//g'  | sed -e '1s/^...../Number /' -e 's/:/ /g'

This is the result of the command:

Number Trans RespTime Concurrent1-HSM 9 0.83 0.252-HSM 23 1.06 0.823-HSM 124 2.34 9.68

The original siege log was:

      Date & Time,  Trans,  Elap Time,  Data Trans,  Resp Time,  TransRate,  Throughput,  Concurrent,    OKAY,   Failed1 2016-07-31 08:35:21,      9,      29.86,           0,       0.83,        0.30,        0.00,        0.25,       9,       02 2016-07-31 08:36:39,     23,      29.60,           0,       1.06,        0.78,        0.00,        0.82,      23,      213 2016-07-31 08:40:15,    124,      29.99,           0,       2.34,        4.13,        0.00,        9.68,     124,       0

This would give me a graph like below:

But that doesn't scale very well for larger and smaller numbers and I didn'tlike the overall look. So I looked around for simple online charting servicesand via Opensource.com found Datawrapper. It's a nice service, lots ofoptions while keeping it simple. I shoved them $12 to export the images becauseI'm to lazy to host it myself or take screenshots. People put effort intohosting and development, so let's reward them for it.

Benchmark process

All the benchmarks were done three times and the best result is used. Themachine that runs haproxy wasn't doing anything else at the time, measured withnethogs. No spotify or skype skewing the benchmarks. All was done via a wiredgigabit network, cat6 cabling.

I've also done benchmarks without the HSM, just regular apache with mod_ssl.Why not mod_nss you might ask? Well, because nobody will setup mod_nss whenthey can use mod_ssl. All the guides use mod_ssl.and the most sites onlineare using it. Only if you have special software or need PKCS#11 you need to usemod_nss.

Benchmarks without the HSM

The below benchmarks does not use the HSM, just regular mod_ssl and acertificate file. We're benchmarking one small page with only the contents 'Jeejit works!'. We're also benchmarking a Wordpress 4.5.3 install with the defaultcontent after install and the Hemmingway theme.

I'm not benchmarking multiple Pi's since the result of one Pi and the result ofthree Pi's was so comparable, I suspect they are fast enough and that mod_ssldoesn't have any bottlenecks there. Except for some tests, those just blew upthe Pi's. (8192 bit keys).

The HSM does not support 4096 or 8192 bit RSA keys, which is why I didnot testthose with the HSM's.

Here below are all the benchmarking results. Every benchmark was done threetimes, best result was kept.

Benchmarking 1024 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificateopenssl req -nodes -x509 -sha256 -newkey rsa:1024 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     63 hitsAvailability:                 100.00 %Elapsed time:                  29.47 secsData transferred:               0.00 MBResponse time:                  0.04 secsTransaction rate:               2.14 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.08Successful transactions:          63Failed transactions:               0Longest transaction:            0.06Shortest transaction:           0.02

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    119 hitsAvailability:                 100.00 %Elapsed time:                  29.67 secsData transferred:               0.00 MBResponse time:                  0.04 secsTransaction rate:               4.01 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.16Successful transactions:         119Failed transactions:               0Longest transaction:            0.10Shortest transaction:           0.02

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    239 hitsAvailability:                 100.00 %Elapsed time:                  29.80 secsData transferred:               0.00 MBResponse time:                  0.04 secsTransaction rate:               8.02 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.30Successful transactions:         239Failed transactions:               0Longest transaction:            0.13Shortest transaction:           0.02

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                   7331 hitsAvailability:                 100.00 %Elapsed time:                  29.18 secsData transferred:               0.09 MBResponse time:                  0.23 secsTransaction rate:             251.23 trans/secThroughput:                     0.00 MB/secConcurrency:                   56.54Successful transactions:        7331Failed transactions:               0Longest transaction:            1.47Shortest transaction:           0.03

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    772 hitsAvailability:                 100.00 %Elapsed time:                  29.84 secsData transferred:               5.63 MBResponse time:                  0.06 secsTransaction rate:              25.87 trans/secThroughput:                     0.19 MB/secConcurrency:                    1.67Successful transactions:         772Failed transactions:               0Longest transaction:            0.75Shortest transaction:           0.02

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                   1353 hitsAvailability:                 100.00 %Elapsed time:                  29.09 secsData transferred:               9.87 MBResponse time:                  0.09 secsTransaction rate:              46.51 trans/secThroughput:                     0.34 MB/secConcurrency:                    4.23Successful transactions:        1353Failed transactions:               0Longest transaction:            2.46Shortest transaction:           0.02

Benchmarking 2048 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificateopenssl req -nodes -x509 -sha256 -newkey rsa:2048 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hitsAvailability:                 100.00 %Elapsed time:                  29.28 secsData transferred:               0.00 MBResponse time:                  0.06 secsTransaction rate:               1.74 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.10Successful transactions:          51Failed transactions:               0Longest transaction:            0.10Shortest transaction:           0.04

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hitsAvailability:                 100.00 %Elapsed time:                  29.68 secsData transferred:               0.00 MBResponse time:                  0.07 secsTransaction rate:               4.25 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.28Successful transactions:         126Failed transactions:               0Longest transaction:            0.19Shortest transaction:           0.04

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Lifting the server siege...Transactions:                    253 hitsAvailability:                 100.00 %Elapsed time:                  29.57 secsData transferred:               0.00 MBResponse time:                  0.07 secsTransaction rate:               8.56 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.60Successful transactions:         253Failed transactions:               0Longest transaction:            0.28Shortest transaction:           0.04

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                   2999 hitsAvailability:                 100.00 %Elapsed time:                  29.92 secsData transferred:               0.04 MBResponse time:                  0.58 secsTransaction rate:             100.23 trans/secThroughput:                     0.00 MB/secConcurrency:                   58.55Successful transactions:        2999Failed transactions:               0Longest transaction:            1.87Shortest transaction:           0.14

60 benchmark mode with 3 Pi's in haproxy:

Transactions:                   4890 hitsAvailability:                 100.00 %Elapsed time:                  29.80 secsData transferred:               0.06 MBResponse time:                  0.36 secsTransaction rate:             164.09 trans/secThroughput:                     0.00 MB/secConcurrency:                   58.74Successful transactions:        4890Failed transactions:               0Longest transaction:            1.42Shortest transaction:           0.04

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    707 hitsAvailability:                 100.00 %Elapsed time:                  29.66 secsData transferred:               5.16 MBResponse time:                  0.10 secsTransaction rate:              23.84 trans/secThroughput:                     0.17 MB/secConcurrency:                    2.29Successful transactions:         707Failed transactions:               0Longest transaction:            1.86Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                   1267 hitsAvailability:                 100.00 %Elapsed time:                  29.62 secsData transferred:               9.25 MBResponse time:                  0.12 secsTransaction rate:              42.78 trans/secThroughput:                     0.31 MB/secConcurrency:                    5.26Successful transactions:        1267Failed transactions:               0Longest transaction:            2.55Shortest transaction:           0.04

Benchmarking 4096 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificateopenssl req -nodes -x509 -sha256 -newkey rsa:4096 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hitsAvailability:                 100.00 %Elapsed time:                  29.47 secsData transferred:               0.00 MBResponse time:                  0.22 secsTransaction rate:               1.73 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.38Successful transactions:          51Failed transactions:               0Longest transaction:            0.45Shortest transaction:           0.19

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    117 hitsAvailability:                 100.00 %Elapsed time:                  29.85 secsData transferred:               0.00 MBResponse time:                  0.29 secsTransaction rate:               3.92 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.14Successful transactions:         117Failed transactions:               0Longest transaction:            0.87Shortest transaction:           0.19  

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    234 hitsAvailability:                 100.00 %Elapsed time:                  29.30 secsData transferred:               0.00 MBResponse time:                  0.29 secsTransaction rate:               7.99 trans/secThroughput:                     0.00 MB/secConcurrency:                    2.31Successful transactions:         234Failed transactions:               0Longest transaction:            1.14Shortest transaction:           0.19

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    591 hitsAvailability:                 100.00 %Elapsed time:                  29.94 secsData transferred:               0.01 MBResponse time:                  2.91 secsTransaction rate:              19.74 trans/secThroughput:                     0.00 MB/secConcurrency:                   57.35Successful transactions:         592Failed transactions:               0Longest transaction:            3.82Shortest transaction:           0.40

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    483 hitsAvailability:                 100.00 %Elapsed time:                  29.22 secsData transferred:               3.53 MBResponse time:                  0.27 secsTransaction rate:              16.53 trans/secThroughput:                     0.12 MB/secConcurrency:                    4.46Successful transactions:         483Failed transactions:               0Longest transaction:            1.98Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    577 hitsAvailability:                 100.00 %Elapsed time:                  29.69 secsData transferred:               4.08 MBResponse time:                  0.67 secsTransaction rate:              19.43 trans/secThroughput:                     0.14 MB/secConcurrency:                   13.10Successful transactions:         577Failed transactions:               0Longest transaction:            2.37Shortest transaction:           0.04

Benchmarking 8192 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificateopenssl req -nodes -x509 -sha256 -newkey rsa:8192 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     39 hitsAvailability:                 100.00 %Elapsed time:                  29.84 secsData transferred:               0.00 MBResponse time:                  1.36 secsTransaction rate:               1.31 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.77Successful transactions:          39Failed transactions:               0Longest transaction:            1.81Shortest transaction:           1.30

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     70 hitsAvailability:                 100.00 %Elapsed time:                  29.94 secsData transferred:               0.00 MBResponse time:                  1.81 secsTransaction rate:               2.34 trans/secThroughput:                     0.00 MB/secConcurrency:                    4.22Successful transactions:          70Failed transactions:               0Longest transaction:            3.38Shortest transaction:           1.30

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     71 hitsAvailability:                 100.00 %Elapsed time:                  29.25 secsData transferred:               0.00 MBResponse time:                  5.13 secsTransaction rate:               2.43 trans/secThroughput:                     0.00 MB/secConcurrency:                   12.44Successful transactions:          71Failed transactions:               0Longest transaction:            6.78Shortest transaction:           1.36

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

This benchmark was done against THREE Pi's, one or two would fail and give thePi a load of +100.

Result:

Transactions:                    176 hitsAvailability:                 100.00 %Elapsed time:                  29.75 secsData transferred:               0.00 MBResponse time:                  5.44 secsTransaction rate:               5.92 trans/secThroughput:                     0.00 MB/secConcurrency:                   32.21Successful transactions:         176Failed transactions:               0Longest transaction:           22.08Shortest transaction:           1.30

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     40 hitsAvailability:                 100.00 %Elapsed time:                  29.05 secsData transferred:               0.08 MBResponse time:                  6.76 secsTransaction rate:               1.38 trans/secThroughput:                     0.00 MB/secConcurrency:                    9.31Successful transactions:          40Failed transactions:               0Longest transaction:            7.89Shortest transaction:           5.62

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     62 hitsAvailability:                 100.00 %Elapsed time:                  29.39 secsData transferred:               0.10 MBResponse time:                  7.28 secsTransaction rate:               2.11 trans/secThroughput:                     0.00 MB/secConcurrency:                   15.36Successful transactions:          62Failed transactions:               0Longest transaction:            8.41Shortest transaction:           5.33

Benchmarking prime256v1 (NIST curve P-256) EC key without HSM

Self signed with OpenSSL:

#keyopenssl ecparam -out ec_key.pem -name pi1.hsmcluster.nl.key -name prime256v1 -genkey#certificateopenssl req -new -key pi1.hsmcluster.nl.key -x509 -nodes -days 365 -out pi1.hsmcluster.nl.cert -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     41 hitsAvailability:                 100.00 %Elapsed time:                  29.79 secsData transferred:               0.00 MBResponse time:                  1.37 secsTransaction rate:               1.38 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.89Successful transactions:          41Failed transactions:               0Longest transaction:            2.12Shortest transaction:           1.29

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     70 hitsAvailability:                 100.00 %Elapsed time:                  29.31 secsData transferred:               0.00 MBResponse time:                  1.72 secsTransaction rate:               2.39 trans/secThroughput:                     0.00 MB/secConcurrency:                    4.11Successful transactions:          70Failed transactions:               0Longest transaction:            3.46Shortest transaction:           1.29

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     76 hitsAvailability:                 100.00 %Elapsed time:                  29.35 secsData transferred:               0.00 MBResponse time:                  5.10 secsTransaction rate:               2.59 trans/secThroughput:                     0.00 MB/secConcurrency:                   13.21Successful transactions:          76Failed transactions:               0Longest transaction:            6.59Shortest transaction:           1.33

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

This benchmark was done against THREE Pi's, one or two would fail and give thePi a load of +100.

Result:

Transactions:                     60 hitsAvailability:                 100.00 %Elapsed time:                  29.66 secsData transferred:               0.00 MBResponse time:                 23.60 secsTransaction rate:               2.02 trans/secThroughput:                     0.00 MB/secConcurrency:                   47.74Successful transactions:          60Failed transactions:               0Longest transaction:           26.21Shortest transaction:          20.71

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     86 hitsAvailability:                 100.00 %Elapsed time:                  29.18 secsData transferred:               0.55 MBResponse time:                  2.93 secsTransaction rate:               2.95 trans/secThroughput:                     0.02 MB/secConcurrency:                    8.63Successful transactions:          86Failed transactions:               0Longest transaction:            4.49Shortest transaction:           0.05

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     80 hitsAvailability:                 100.00 %Elapsed time:                  29.19 secsData transferred:               0.17 MBResponse time:                  6.75 secsTransaction rate:               2.74 trans/secThroughput:                     0.01 MB/secConcurrency:                   18.50Successful transactions:          80Failed transactions:               0Longest transaction:            8.17Shortest transaction:           6.10

OpenSC benchmarks

The below benchmarks utilize the opensc-pkcs11 module with mod_nss. We'rebenchmarking one small page with only the contents 'Jeej it works!'. We're alsobenchmarking a Wordpress 4.5.3 install with the default content after installand the Hemmingway theme.

Note to self, the below command formats the siege output directly to space-seperated (instead of tab) markdown code output:

siege  -c10 -d5 -t30S 'https://hsmcluster.nl/' 2>&1 | grep -v '==> GET ' | expand | sed 's/^/    /'

1 HSM (OpenSC)

Benchmarking 1024 bit RSA key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     52 hitsAvailability:                 100.00 %Elapsed time:                  29.52 secsData transferred:               0.00 MBResponse time:                  0.33 secsTransaction rate:               1.76 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.57Successful transactions:          52Failed transactions:               0Longest transaction:            1.06Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    101 hitsAvailability:                 100.00 %Elapsed time:                  29.07 secsData transferred:               0.00 MBResponse time:                  0.52 secsTransaction rate:               3.47 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.80Successful transactions:         101Failed transactions:               0Longest transaction:            2.06Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    142 hitsAvailability:                 100.00 %Elapsed time:                  29.86 secsData transferred:               0.00 MBResponse time:                  1.78 secsTransaction rate:               4.76 trans/secThroughput:                     0.00 MB/secConcurrency:                    8.47Successful transactions:         142Failed transactions:               0Longest transaction:            5.07Shortest transaction:           0.23

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hitsAvailability:                 100.00 %Elapsed time:                  29.37 secsData transferred:               0.00 MBResponse time:                  8.96 secsTransaction rate:               4.29 trans/secThroughput:                     0.00 MB/secConcurrency:                   38.46Successful transactions:         126Failed transactions:               0Longest transaction:           19.99Shortest transaction:           0.60

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    162 hitsAvailability:                 100.00 %Elapsed time:                  29.13 secsData transferred:               1.14 MBResponse time:                  1.50 secsTransaction rate:               5.56 trans/secThroughput:                     0.04 MB/secConcurrency:                    8.35Successful transactions:         162Failed transactions:               0Longest transaction:            8.85Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    165 hitsAvailability:                 100.00 %Elapsed time:                  29.77 secsData transferred:               1.07 MBResponse time:                  3.08 secsTransaction rate:               5.54 trans/secThroughput:                     0.04 MB/secConcurrency:                   17.06Successful transactions:         165Failed transactions:               0Longest transaction:           10.86Shortest transaction:           0.05
Benchmarking 2048 bit RSA key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     35 hitsAvailability:                 100.00 %Elapsed time:                  29.01 secsData transferred:               0.00 MBResponse time:                  1.23 secsTransaction rate:               1.21 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.49Successful transactions:          35Failed transactions:               0Longest transaction:            3.48Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     45 hitsAvailability:                 100.00 %Elapsed time:                  29.92 secsData transferred:               0.00 MBResponse time:                  3.79 secsTransaction rate:               1.50 trans/secThroughput:                     0.00 MB/secConcurrency:                    5.70Successful transactions:          45Failed transactions:               0Longest transaction:           17.70Shortest transaction:           0.68

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hitsAvailability:                 100.00 %Elapsed time:                  29.15 secsData transferred:               0.00 MBResponse time:                  8.83 secsTransaction rate:               1.51 trans/secThroughput:                     0.00 MB/secConcurrency:                   13.32Successful transactions:          44Failed transactions:               0Longest transaction:           27.20Shortest transaction:           0.69

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     42 hitsAvailability:                 100.00 %Elapsed time:                  29.35 secsData transferred:               0.00 MBResponse time:                 12.53 secsTransaction rate:               1.43 trans/secThroughput:                     0.00 MB/secConcurrency:                   17.94Successful transactions:          42Failed transactions:               0Longest transaction:           28.97Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     49 hitsAvailability:                 100.00 %Elapsed time:                  29.24 secsData transferred:               0.24 MBResponse time:                  4.40 secsTransaction rate:               1.68 trans/secThroughput:                     0.01 MB/secConcurrency:                    7.37Successful transactions:          49Failed transactions:               0Longest transaction:           24.25Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     46 hitsAvailability:                 100.00 %Elapsed time:                  29.98 secsData transferred:               0.13 MBResponse time:                  9.63 secsTransaction rate:               1.53 trans/secThroughput:                     0.00 MB/secConcurrency:                   14.77Successful transactions:          46Failed transactions:               0Longest transaction:           29.81Shortest transaction:           0.05
Benchmarking EC prime256v1 key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     52 hitsAvailability:                 100.00 %Elapsed time:                  29.95 secsData transferred:               0.00 MBResponse time:                  0.48 secsTransaction rate:               1.74 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.84Successful transactions:          52Failed transactions:               0Longest transaction:            1.91Shortest transaction:           0.27

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     92 hitsAvailability:                 100.00 %Elapsed time:                  29.49 secsData transferred:               0.00 MBResponse time:                  1.01 secsTransaction rate:               3.12 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.16Successful transactions:          92Failed transactions:               0Longest transaction:            4.65Shortest transaction:           0.27

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    103 hitsAvailability:                 100.00 %Elapsed time:                  29.96 secsData transferred:               0.00 MBResponse time:                  3.35 secsTransaction rate:               3.44 trans/secThroughput:                     0.00 MB/secConcurrency:                   11.53Successful transactions:         103Failed transactions:               0Longest transaction:            8.37Shortest transaction:           0.28

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     97 hitsAvailability:                 100.00 %Elapsed time:                  29.90 secsData transferred:               0.00 MBResponse time:                 10.76 secsTransaction rate:               3.24 trans/secThroughput:                     0.00 MB/secConcurrency:                   34.89Successful transactions:          97Failed transactions:               0Longest transaction:           29.19Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    120 hitsAvailability:                 100.00 %Elapsed time:                  29.84 secsData transferred:               0.80 MBResponse time:                  2.11 secsTransaction rate:               4.02 trans/secThroughput:                     0.03 MB/secConcurrency:                    8.49Successful transactions:         120Failed transactions:               0Longest transaction:            8.04Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    116 hitsAvailability:                 100.00 %Elapsed time:                  29.87 secsData transferred:               0.59 MBResponse time:                  4.25 secsTransaction rate:               3.88 trans/secThroughput:                     0.02 MB/secConcurrency:                   16.52Successful transactions:         116Failed transactions:               0Longest transaction:           14.15Shortest transaction:           0.05

2 HSM's (OpenSC)

Benchmarking 1024 bit RSA key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hitsAvailability:                 100.00 %Elapsed time:                  29.17 secsData transferred:               0.00 MBResponse time:                  0.28 secsTransaction rate:               1.75 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.49Successful transactions:          51Failed transactions:               0Longest transaction:            1.06Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    107 hitsAvailability:                 100.00 %Elapsed time:                  29.23 secsData transferred:               0.00 MBResponse time:                  0.30 secsTransaction rate:               3.66 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.09Successful transactions:         107Failed transactions:               0Longest transaction:            1.31Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    217 hitsAvailability:                 100.00 %Elapsed time:                  29.96 secsData transferred:               0.00 MBResponse time:                  0.58 secsTransaction rate:               7.24 trans/secThroughput:                     0.00 MB/secConcurrency:                    4.21Successful transactions:         217Failed transactions:               0Longest transaction:            4.26Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    277 hitsAvailability:                 100.00 %Elapsed time:                  29.72 secsData transferred:               0.00 MBResponse time:                  5.66 secsTransaction rate:               9.32 trans/secThroughput:                     0.00 MB/secConcurrency:                   52.77Successful transactions:         277Failed transactions:               0Longest transaction:           18.27Shortest transaction:           0.47

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    286 hitsAvailability:                 100.00 %Elapsed time:                  29.76 secsData transferred:               2.04 MBResponse time:                  0.78 secsTransaction rate:               9.61 trans/secThroughput:                     0.07 MB/secConcurrency:                    7.45Successful transactions:         286Failed transactions:               0Longest transaction:            4.67Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    298 hitsAvailability:                 100.00 %Elapsed time:                  29.12 secsData transferred:               2.04 MBResponse time:                  1.51 secsTransaction rate:              10.23 trans/secThroughput:                     0.07 MB/secConcurrency:                   15.49Successful transactions:         298Failed transactions:               0Longest transaction:            9.48Shortest transaction:           0.04
Benchmarking 2048 bit RSA key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     47 hitsAvailability:                 100.00 %Elapsed time:                  29.00 secsData transferred:               0.00 MBResponse time:                  1.01 secsTransaction rate:               1.62 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.63Successful transactions:          47Failed transactions:               0Longest transaction:            3.08Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     75 hitsAvailability:                 100.00 %Elapsed time:                  29.05 secsData transferred:               0.00 MBResponse time:                  1.58 secsTransaction rate:               2.58 trans/secThroughput:                     0.00 MB/secConcurrency:                    4.07Successful transactions:          75Failed transactions:               0Longest transaction:            7.71Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     82 hitsAvailability:                 100.00 %Elapsed time:                  29.74 secsData transferred:               0.00 MBResponse time:                  4.25 secsTransaction rate:               2.76 trans/secThroughput:                     0.00 MB/secConcurrency:                   11.72Successful transactions:          82Failed transactions:               0Longest transaction:           17.67Shortest transaction:           0.67

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     46 hitsAvailability:                 100.00 %Elapsed time:                  29.11 secsData transferred:               0.00 MBResponse time:                 15.96 secsTransaction rate:               1.58 trans/secThroughput:                     0.00 MB/secConcurrency:                   25.22Successful transactions:          46Failed transactions:               0Longest transaction:           28.88Shortest transaction:           2.22

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     92 hitsAvailability:                 100.00 %Elapsed time:                  29.44 secsData transferred:               0.61 MBResponse time:                  2.61 secsTransaction rate:               3.12 trans/secThroughput:                     0.02 MB/secConcurrency:                    8.16Successful transactions:          92Failed transactions:               0Longest transaction:           13.43Shortest transaction:           0.05

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     75 hitsAvailability:                 100.00 %Elapsed time:                  29.60 secsData transferred:               0.41 MBResponse time:                  5.35 secsTransaction rate:               2.53 trans/secThroughput:                     0.01 MB/secConcurrency:                   13.56Successful transactions:          75Failed transactions:               0Longest transaction:           21.47Shortest transaction:           0.00
Benchmarking EC prime256v1 key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     60 hitsAvailability:                 100.00 %Elapsed time:                  29.30 secsData transferred:               0.00 MBResponse time:                  0.31 secsTransaction rate:               2.05 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.64Successful transactions:          60Failed transactions:               0Longest transaction:            1.09Shortest transaction:           0.26

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    106 hitsAvailability:                 100.00 %Elapsed time:                  29.71 secsData transferred:               0.00 MBResponse time:                  0.42 secsTransaction rate:               3.57 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.50Successful transactions:         106Failed transactions:               0Longest transaction:            1.58Shortest transaction:           0.26

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    184 hitsAvailability:                 100.00 %Elapsed time:                  29.61 secsData transferred:               0.00 MBResponse time:                  0.80 secsTransaction rate:               6.21 trans/secThroughput:                     0.00 MB/secConcurrency:                    4.99Successful transactions:         184Failed transactions:               0Longest transaction:            4.12Shortest transaction:           0.26  

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    201 hitsAvailability:                 100.00 %Elapsed time:                  29.50 secsData transferred:               0.00 MBResponse time:                  5.56 secsTransaction rate:               6.81 trans/secThroughput:                     0.00 MB/secConcurrency:                   37.90Successful transactions:         201Failed transactions:               0Longest transaction:           27.49Shortest transaction:           0.26

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    237 hitsAvailability:                 100.00 %Elapsed time:                  29.56 secsData transferred:               1.69 MBResponse time:                  0.90 secsTransaction rate:               8.02 trans/secThroughput:                     0.06 MB/secConcurrency:                    7.25Successful transactions:         237Failed transactions:               0Longest transaction:            4.72Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    240 hitsAvailability:                 100.00 %Elapsed time:                  29.28 secsData transferred:               1.67 MBResponse time:                  2.08 secsTransaction rate:               8.20 trans/secThroughput:                     0.06 MB/secConcurrency:                   17.03Successful transactions:         240Failed transactions:               0Longest transaction:            7.64Shortest transaction:           0.04

3 HSM's (OpenSC)

Benchmarking 1024 bit RSA key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     54 hitsAvailability:                 100.00 %Elapsed time:                  29.88 secsData transferred:               0.00 MBResponse time:                  0.25 secsTransaction rate:               1.81 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.46Successful transactions:          54Failed transactions:               0Longest transaction:            0.66Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    118 hitsAvailability:                 100.00 %Elapsed time:                  29.72 secsData transferred:               0.00 MBResponse time:                  0.28 secsTransaction rate:               3.97 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.12Successful transactions:         118Failed transactions:               0Longest transaction:            1.66Shortest transaction:           0.22 

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    205 hitsAvailability:                 100.00 %Elapsed time:                  29.85 secsData transferred:               0.00 MBResponse time:                  0.32 secsTransaction rate:               6.87 trans/secThroughput:                     0.00 MB/secConcurrency:                    2.19Successful transactions:         206Failed transactions:               0Longest transaction:            1.69Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    407 hitsAvailability:                 100.00 %Elapsed time:                  29.07 secsData transferred:               0.01 MBResponse time:                  3.65 secsTransaction rate:              14.00 trans/secThroughput:                     0.00 MB/secConcurrency:                   51.11Successful transactions:         407Failed transactions:               0Longest transaction:           17.38Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    382 hitsAvailability:                 100.00 %Elapsed time:                  29.48 secsData transferred:               2.73 MBResponse time:                  0.40 secsTransaction rate:              12.96 trans/secThroughput:                     0.09 MB/secConcurrency:                    5.22Successful transactions:         382Failed transactions:               0Longest transaction:            1.99Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    449 hitsAvailability:                 100.00 %Elapsed time:                  29.31 secsData transferred:               3.18 MBResponse time:                  0.99 secsTransaction rate:              15.32 trans/secThroughput:                     0.11 MB/secConcurrency:                   15.12Successful transactions:         449Failed transactions:               0Longest transaction:            6.49Shortest transaction:           0.04
Benchmarking 2048 bit RSA key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hitsAvailability:                 100.00 %Elapsed time:                  29.10 secsData transferred:               0.00 MBResponse time:                  0.75 secsTransaction rate:               1.51 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.13Successful transactions:          44Failed transactions:               0Longest transaction:            1.50Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     89 hitsAvailability:                 100.00 %Elapsed time:                  29.34 secsData transferred:               0.00 MBResponse time:                  0.90 secsTransaction rate:               3.03 trans/secThroughput:                     0.00 MB/secConcurrency:                    2.72Successful transactions:          89Failed transactions:               0Longest transaction:            2.67Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    127 hitsAvailability:                 100.00 %Elapsed time:                  29.99 secsData transferred:               0.00 MBResponse time:                  2.05 secsTransaction rate:               4.23 trans/secThroughput:                     0.00 MB/secConcurrency:                    8.69Successful transactions:         127Failed transactions:               0Longest transaction:           11.20Shortest transaction:           0.66

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    128 hitsAvailability:                 100.00 %Elapsed time:                  29.53 secsData transferred:               0.00 MBResponse time:                  9.24 secsTransaction rate:               4.33 trans/secThroughput:                     0.00 MB/secConcurrency:                   40.07Successful transactions:         128Failed transactions:               0Longest transaction:           23.46Shortest transaction:           0.66

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    139 hitsAvailability:                 100.00 %Elapsed time:                  29.43 secsData transferred:               0.97 MBResponse time:                  1.79 secsTransaction rate:               4.72 trans/secThroughput:                     0.03 MB/secConcurrency:                    8.48Successful transactions:         139Failed transactions:               0Longest transaction:           11.38Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    140 hitsAvailability:                 100.00 %Elapsed time:                  29.39 secsData transferred:               0.89 MBResponse time:                  3.58 secsTransaction rate:               4.76 trans/secThroughput:                     0.03 MB/secConcurrency:                   17.04Successful transactions:         140Failed transactions:               0Longest transaction:           21.96Shortest transaction:           0.05
Benchmarking EC prime256v1 key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hitsAvailability:                 100.00 %Elapsed time:                  29.73 secsData transferred:               0.00 MBResponse time:                  0.31 secsTransaction rate:               1.72 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.53Successful transactions:          51Failed transactions:               0Longest transaction:            0.67Shortest transaction:           0.25    

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    110 hitsAvailability:                 100.00 %Elapsed time:                  29.41 secsData transferred:               0.00 MBResponse time:                  0.34 secsTransaction rate:               3.74 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.26Successful transactions:         110Failed transactions:               0Longest transaction:            1.29Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    221 hitsAvailability:                 100.00 %Elapsed time:                  29.82 secsData transferred:               0.00 MBResponse time:                  0.44 secsTransaction rate:               7.41 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.24Successful transactions:         221Failed transactions:               0Longest transaction:            2.12Shortest transaction:           0.24

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    331 hitsAvailability:                 100.00 %Elapsed time:                  29.64 secsData transferred:               0.00 MBResponse time:                  4.29 secsTransaction rate:              11.17 trans/secThroughput:                     0.00 MB/secConcurrency:                   47.90Successful transactions:         331Failed transactions:               0Longest transaction:           25.01Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    336 hitsAvailability:                 100.00 %Elapsed time:                  29.48 secsData transferred:               2.42 MBResponse time:                  0.54 secsTransaction rate:              11.40 trans/secThroughput:                     0.08 MB/secConcurrency:                    6.10Successful transactions:         336Failed transactions:               0Longest transaction:            2.70Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    377 hitsAvailability:                 100.00 %Elapsed time:                  29.86 secsData transferred:               2.59 MBResponse time:                  1.21 secsTransaction rate:              12.63 trans/secThroughput:                     0.09 MB/secConcurrency:                   15.27Successful transactions:         377Failed transactions:               0Longest transaction:            5.02Shortest transaction:           0.04

sc-hsm-embedded benchmarks

The below benchmarks utilize the read only libsc-hsm-embedded module withmod_nss. Read more on the sc-hsm-embedded module here.

This module is targeted at embedded use in devices, but can also be usedregularly. I recommend it over the OpenSC module, since in production you don'twant to be able to write to the HSM. You should have a seperate, non-networkedworkstation for that.

1 HSM (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     56 hitsAvailability:                 100.00 %Elapsed time:                  29.48 secsData transferred:               0.00 MBResponse time:                  0.31 secsTransaction rate:               1.90 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.60Successful transactions:          56Failed transactions:               0Longest transaction:            1.13Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    100 hitsAvailability:                 100.00 %Elapsed time:                  29.18 secsData transferred:               0.00 MBResponse time:                  0.52 secsTransaction rate:               3.43 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.78Successful transactions:         100Failed transactions:               0Longest transaction:            2.24Shortest transaction:           0.24

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    131 hitsAvailability:                 100.00 %Elapsed time:                  29.17 secsData transferred:               0.00 MBResponse time:                  2.04 secsTransaction rate:               4.49 trans/secThroughput:                     0.00 MB/secConcurrency:                    9.17Successful transactions:         131Failed transactions:               0Longest transaction:            6.56Shortest transaction:           0.28

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    124 hitsAvailability:                 100.00 %Elapsed time:                  29.50 secsData transferred:               0.00 MBResponse time:                 10.24 secsTransaction rate:               4.20 trans/secThroughput:                     0.00 MB/secConcurrency:                   43.03Successful transactions:         124Failed transactions:               0Longest transaction:           19.99Shortest transaction:           0.94

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    148 hitsAvailability:                 100.00 %Elapsed time:                  29.63 secsData transferred:               1.02 MBResponse time:                  1.68 secsTransaction rate:               4.99 trans/secThroughput:                     0.03 MB/secConcurrency:                    8.39Successful transactions:         148Failed transactions:               0Longest transaction:            5.48Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    144 hitsAvailability:                 100.00 %Elapsed time:                  29.12 secsData transferred:               0.98 MBResponse time:                  3.63 secsTransaction rate:               4.95 trans/secThroughput:                     0.03 MB/secConcurrency:                   17.94Successful transactions:         144Failed transactions:               0Longest transaction:           11.69Shortest transaction:           0.04
Benchmarking 2048 bit RSA key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     36 hitsAvailability:                 100.00 %Elapsed time:                  29.78 secsData transferred:               0.00 MBResponse time:                  1.59 secsTransaction rate:               1.21 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.93Successful transactions:          36Failed transactions:               0Longest transaction:            3.89Shortest transaction:           0.67

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hitsAvailability:                 100.00 %Elapsed time:                  29.19 secsData transferred:               0.00 MBResponse time:                  4.21 secsTransaction rate:               1.51 trans/secThroughput:                     0.00 MB/secConcurrency:                    6.34Successful transactions:          44Failed transactions:               0Longest transaction:            9.43Shortest transaction:           0.68

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     39 hitsAvailability:                 100.00 %Elapsed time:                  29.06 secsData transferred:               0.00 MBResponse time:                  9.92 secsTransaction rate:               1.34 trans/secThroughput:                     0.00 MB/secConcurrency:                   13.32Successful transactions:          39Failed transactions:               0Longest transaction:           16.25Shortest transaction:           1.60

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     23 hitsAvailability:                 100.00 %Elapsed time:                  29.82 secsData transferred:               0.00 MBResponse time:                 17.65 secsTransaction rate:               0.77 trans/secThroughput:                     0.00 MB/secConcurrency:                   13.61Successful transactions:          23Failed transactions:               0Longest transaction:           29.12Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     40 hitsAvailability:                 100.00 %Elapsed time:                  29.75 secsData transferred:               0.17 MBResponse time:                  6.87 secsTransaction rate:               1.34 trans/secThroughput:                     0.01 MB/secConcurrency:                    9.23Successful transactions:          40Failed transactions:               0Longest transaction:           16.12Shortest transaction:           1.66

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     33 hitsAvailability:                 100.00 %Elapsed time:                  29.01 secsData transferred:               0.07 MBResponse time:                 11.64 secsTransaction rate:               1.14 trans/secThroughput:                     0.00 MB/secConcurrency:                   13.24Successful transactions:          33Failed transactions:               0Longest transaction:           26.08Shortest transaction:           3.18
Benchmarking EC prime256v1 key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hitsAvailability:                 100.00 %Elapsed time:                  29.15 secsData transferred:               0.00 MBResponse time:                  0.37 secsTransaction rate:               2.02 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.75Successful transactions:          59Failed transactions:               0Longest transaction:            1.22Shortest transaction:           0.25

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    103 hitsAvailability:                 100.00 %Elapsed time:                  29.67 secsData transferred:               0.00 MBResponse time:                  0.57 secsTransaction rate:               3.47 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.98Successful transactions:         103Failed transactions:               0Longest transaction:            2.86Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    122 hitsAvailability:                 100.00 %Elapsed time:                  29.85 secsData transferred:               0.00 MBResponse time:                  2.10 secsTransaction rate:               4.09 trans/secThroughput:                     0.00 MB/secConcurrency:                    8.59Successful transactions:         122Failed transactions:               0Longest transaction:            6.55Shortest transaction:           0.30

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    109 hitsAvailability:                 100.00 %Elapsed time:                  29.45 secsData transferred:               0.00 MBResponse time:                 11.17 secsTransaction rate:               3.70 trans/secThroughput:                     0.00 MB/secConcurrency:                   41.33Successful transactions:         109Failed transactions:               0Longest transaction:           23.97Shortest transaction:           0.52

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    137 hitsAvailability:                 100.00 %Elapsed time:                  29.04 secsData transferred:               0.93 MBResponse time:                  1.68 secsTransaction rate:               4.72 trans/secThroughput:                     0.03 MB/secConcurrency:                    7.94Successful transactions:         137Failed transactions:               0Longest transaction:            4.49Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    138 hitsAvailability:                 100.00 %Elapsed time:                  29.64 secsData transferred:               0.95 MBResponse time:                  3.96 secsTransaction rate:               4.66 trans/secThroughput:                     0.03 MB/secConcurrency:                   18.44Successful transactions:         138Failed transactions:               0Longest transaction:           12.19Shortest transaction:           0.05

2 HSM's (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hitsAvailability:                 100.00 %Elapsed time:                  29.34 secsData transferred:               0.00 MBResponse time:                  0.26 secsTransaction rate:               2.01 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.52Successful transactions:          59Failed transactions:               0Longest transaction:            0.62Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    109 hitsAvailability:                 100.00 %Elapsed time:                  29.04 secsData transferred:               0.00 MBResponse time:                  0.29 secsTransaction rate:               3.75 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.10Successful transactions:         109Failed transactions:               0Longest transaction:            1.13Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    211 hitsAvailability:                 100.00 %Elapsed time:                  29.36 secsData transferred:               0.00 MBResponse time:                  0.42 secsTransaction rate:               7.19 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.01Successful transactions:         211Failed transactions:               0Longest transaction:            2.45Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    253 hitsAvailability:                 100.00 %Elapsed time:                  29.04 secsData transferred:               0.00 MBResponse time:                  5.24 secsTransaction rate:               8.71 trans/secThroughput:                     0.00 MB/secConcurrency:                   45.62Successful transactions:         253Failed transactions:               0Longest transaction:           18.60Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    308 hitsAvailability:                 100.00 %Elapsed time:                  29.22 secsData transferred:               2.20 MBResponse time:                  0.61 secsTransaction rate:              10.54 trans/secThroughput:                     0.08 MB/secConcurrency:                    6.39Successful transactions:         308Failed transactions:               0Longest transaction:            2.29Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    318 hitsAvailability:                 100.00 %Elapsed time:                  29.94 secsData transferred:               2.21 MBResponse time:                  1.53 secsTransaction rate:              10.62 trans/secThroughput:                     0.07 MB/secConcurrency:                   16.23Successful transactions:         318Failed transactions:               0Longest transaction:            7.03Shortest transaction:           0.04
Benchmarking 2048 bit RSA key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     47 hitsAvailability:                 100.00 %Elapsed time:                  29.86 secsData transferred:               0.00 MBResponse time:                  0.82 secsTransaction rate:               1.57 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.29Successful transactions:          47Failed transactions:               0Longest transaction:            1.93Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     74 hitsAvailability:                 100.00 %Elapsed time:                  29.90 secsData transferred:               0.00 MBResponse time:                  1.50 secsTransaction rate:               2.47 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.71Successful transactions:          74Failed transactions:               0Longest transaction:            4.43Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     84 hitsAvailability:                 100.00 %Elapsed time:                  29.03 secsData transferred:               0.00 MBResponse time:                  4.29 secsTransaction rate:               2.89 trans/secThroughput:                     0.00 MB/secConcurrency:                   12.42Successful transactions:          84Failed transactions:               0Longest transaction:           14.70Shortest transaction:           0.69

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     72 hitsAvailability:                 100.00 %Elapsed time:                  29.87 secsData transferred:               0.00 MBResponse time:                 13.06 secsTransaction rate:               2.41 trans/secThroughput:                     0.00 MB/secConcurrency:                   31.48Successful transactions:          72Failed transactions:               0Longest transaction:           27.91Shortest transaction:           2.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     95 hitsAvailability:                 100.00 %Elapsed time:                  29.51 secsData transferred:               0.60 MBResponse time:                  2.75 secsTransaction rate:               3.22 trans/secThroughput:                     0.02 MB/secConcurrency:                    8.86Successful transactions:          95Failed transactions:               0Longest transaction:            8.09Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     87 hitsAvailability:                 100.00 %Elapsed time:                  29.89 secsData transferred:               0.41 MBResponse time:                  6.08 secsTransaction rate:               2.91 trans/secThroughput:                     0.01 MB/secConcurrency:                   17.70Successful transactions:          87Failed transactions:               0Longest transaction:           17.31Shortest transaction:           0.05
Benchmarking EC prime256v1 key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hitsAvailability:                 100.00 %Elapsed time:                  29.14 secsData transferred:               0.00 MBResponse time:                  0.29 secsTransaction rate:               2.02 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.59Successful transactions:          59Failed transactions:               0Longest transaction:            0.71Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     98 hitsAvailability:                 100.00 %Elapsed time:                  29.19 secsData transferred:               0.00 MBResponse time:                  0.34 secsTransaction rate:               3.36 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.13Successful transactions:          98Failed transactions:               0Longest transaction:            1.20Shortest transaction:           0.24

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    217 hitsAvailability:                 100.00 %Elapsed time:                  29.94 secsData transferred:               0.00 MBResponse time:                  0.52 secsTransaction rate:               7.25 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.78Successful transactions:         217Failed transactions:               0Longest transaction:            2.39Shortest transaction:           0.24

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    232 hitsAvailability:                 100.00 %Elapsed time:                  29.22 secsData transferred:               0.00 MBResponse time:                  5.62 secsTransaction rate:               7.94 trans/secThroughput:                     0.00 MB/secConcurrency:                   44.64Successful transactions:         232Failed transactions:               0Longest transaction:           20.16Shortest transaction:           0.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    276 hitsAvailability:                 100.00 %Elapsed time:                  29.35 secsData transferred:               1.92 MBResponse time:                  0.72 secsTransaction rate:               9.40 trans/secThroughput:                     0.07 MB/secConcurrency:                    6.74Successful transactions:         276Failed transactions:               0Longest transaction:            2.42Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    288 hitsAvailability:                 100.00 %Elapsed time:                  29.86 secsData transferred:               1.94 MBResponse time:                  1.77 secsTransaction rate:               9.65 trans/secThroughput:                     0.06 MB/secConcurrency:                   17.08Successful transactions:         288Failed transactions:               0Longest transaction:            8.33Shortest transaction:           0.04

3 HSM's (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     55 hitsAvailability:                 100.00 %Elapsed time:                  29.81 secsData transferred:               0.00 MBResponse time:                  0.24 secsTransaction rate:               1.85 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.45Successful transactions:          55Failed transactions:               0Longest transaction:            0.47Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    116 hitsAvailability:                 100.00 %Elapsed time:                  29.33 secsData transferred:               0.00 MBResponse time:                  0.26 secsTransaction rate:               3.95 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.05Successful transactions:         116Failed transactions:               0Longest transaction:            0.81Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    227 hitsAvailability:                 100.00 %Elapsed time:                  29.82 secsData transferred:               0.00 MBResponse time:                  0.31 secsTransaction rate:               7.61 trans/secThroughput:                     0.00 MB/secConcurrency:                    2.39Successful transactions:         227Failed transactions:               0Longest transaction:            1.58Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    420 hitsAvailability:                 100.00 %Elapsed time:                  29.90 secsData transferred:               0.01 MBResponse time:                  3.79 secsTransaction rate:              14.05 trans/secThroughput:                     0.00 MB/secConcurrency:                   53.27Successful transactions:         420Failed transactions:               0Longest transaction:           20.31Shortest transaction:           0.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    430 hitsAvailability:                 100.00 %Elapsed time:                  29.46 secsData transferred:               3.11 MBResponse time:                  0.37 secsTransaction rate:              14.60 trans/secThroughput:                     0.11 MB/secConcurrency:                    5.38Successful transactions:         430Failed transactions:               0Longest transaction:            1.96Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    494 hitsAvailability:                 100.00 %Elapsed time:                  29.72 secsData transferred:               3.49 MBResponse time:                  0.87 secsTransaction rate:              16.62 trans/secThroughput:                     0.12 MB/secConcurrency:                   14.43Successful transactions:         494Failed transactions:               0Longest transaction:            3.93Shortest transaction:           0.04
Benchmarking 2048 bit RSA key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     50 hitsAvailability:                 100.00 %Elapsed time:                  29.58 secsData transferred:               0.00 MBResponse time:                  0.73 secsTransaction rate:               1.69 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.23Successful transactions:          50Failed transactions:               0Longest transaction:            1.50Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     99 hitsAvailability:                 100.00 %Elapsed time:                  29.82 secsData transferred:               0.00 MBResponse time:                  0.91 secsTransaction rate:               3.32 trans/secThroughput:                     0.00 MB/secConcurrency:                    3.01Successful transactions:          99Failed transactions:               0Longest transaction:            2.97Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hitsAvailability:                 100.00 %Elapsed time:                  29.85 secsData transferred:               0.00 MBResponse time:                  2.22 secsTransaction rate:               4.22 trans/secThroughput:                     0.00 MB/secConcurrency:                    9.35Successful transactions:         126Failed transactions:               0Longest transaction:            7.04Shortest transaction:           0.66     

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    118 hitsAvailability:                 100.00 %Elapsed time:                  29.28 secsData transferred:               0.00 MBResponse time:                 10.28 secsTransaction rate:               4.03 trans/secThroughput:                     0.00 MB/secConcurrency:                   41.44Successful transactions:         118Failed transactions:               0Longest transaction:           29.13Shortest transaction:           0.66    

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    141 hitsAvailability:                 100.00 %Elapsed time:                  29.05 secsData transferred:               1.00 MBResponse time:                  1.74 secsTransaction rate:               4.85 trans/secThroughput:                     0.03 MB/secConcurrency:                    8.44Successful transactions:         141Failed transactions:               0Longest transaction:            4.63Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    145 hitsAvailability:                 100.00 %Elapsed time:                  29.27 secsData transferred:               1.01 MBResponse time:                  3.57 secsTransaction rate:               4.95 trans/secThroughput:                     0.03 MB/secConcurrency:                   17.70Successful transactions:         145Failed transactions:               0Longest transaction:           12.44Shortest transaction:           0.04
Benchmarking EC prime256v1 key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     57 hitsAvailability:                 100.00 %Elapsed time:                  29.41 secsData transferred:               0.00 MBResponse time:                  0.29 secsTransaction rate:               1.94 trans/secThroughput:                     0.00 MB/secConcurrency:                    0.56Successful transactions:          57Failed transactions:               0Longest transaction:            0.56Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    104 hitsAvailability:                 100.00 %Elapsed time:                  29.02 secsData transferred:               0.00 MBResponse time:                  0.31 secsTransaction rate:               3.58 trans/secThroughput:                     0.00 MB/secConcurrency:                    1.11Successful transactions:         104Failed transactions:               0Longest transaction:            0.94Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    220 hitsAvailability:                 100.00 %Elapsed time:                  29.82 secsData transferred:               0.00 MBResponse time:                  0.40 secsTransaction rate:               7.38 trans/secThroughput:                     0.00 MB/secConcurrency:                    2.97Successful transactions:         220Failed transactions:               0Longest transaction:            2.16Shortest transaction:           0.25

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    360 hitsAvailability:                 100.00 %Elapsed time:                  29.99 secsData transferred:               0.00 MBResponse time:                  4.34 secsTransaction rate:              12.00 trans/secThroughput:                     0.00 MB/secConcurrency:                   52.12Successful transactions:         360Failed transactions:               0Longest transaction:           18.87Shortest transaction:           0.30    

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    382 hitsAvailability:                 100.00 %Elapsed time:                  29.57 secsData transferred:               2.77 MBResponse time:                  0.45 secsTransaction rate:              12.92 trans/secThroughput:                     0.09 MB/secConcurrency:                    5.86Successful transactions:         382Failed transactions:               0Longest transaction:            2.32Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    427 hitsAvailability:                 100.00 %Elapsed time:                  29.96 secsData transferred:               2.90 MBResponse time:                  1.11 secsTransaction rate:              14.25 trans/secThroughput:                     0.10 MB/secConcurrency:                   15.83Successful transactions:         427Failed transactions:               0Longest transaction:            3.71Shortest transaction:           0.04
Tags: apache, articles, cluster, cryptoki, haproxy, hsm, mod_nss, nginx, nitrokey, nitrokey-hsm, openssl, pkcs11, safenet, smartcard, smartcard-hsm