Skip to main content Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Encrypt and decrypt files to public keys via the OpenSSL Command Line

Published: 25-10-2018 | Author: Remy van Elst | Text only version of this article

Table of Contents

This small tutorial will show you how to use the openssl command line to encryptand decrypt a file using a public key. We will first generate a random key,encrypt that random key against the public key of the other person and use thatrandom key to encrypt the actual file with using symmetric encryption.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

Because of how the RSA algorithm works it is not possible to encrypt largefiles. If you create a key of n bits, then the file you want to encrypt mustnot larger than (n minus 11) bits. The most effective use of RSA crypto is toencrypt a random generated password, then encrypt the file with the passwordusing symmetric crypto. If the file is larger then the key size the encryptioncommand will fail:

RSA operation error:  020:error:0406D06E:rsa routines:RSA_padding_add_PKCS1_type_2:data too large for key size:.\crypto\rsa\rsa_pk1.c:151:

We generate a random file and use that as the key to encrypt the large file withsymmetric crypto. That random file acts as the password so to say. We encryptthe large file with the small password file as password. Then we send theencrypted file and the encrypted key to the other party and then can decrypt thekey with their public key, the use that key to decrypt the large file.

The following commands are relevant when you work with RSA keys:

The key is just a string of random bytes. We use a base64 encoded string of 128bytes, which is 175 characters. Since 175 characters is 1400 bits, even a smallRSA key will be able to encrypt it.

Get the public key

Let the other party send you a certificate or their public key. If they send toa certificate you can extract the public key using this command:

openssl rsa -in certificate.pem -out publickey.pem -outform PEM -pubout

Generate the random password file

Use the following command to generate the random key:

openssl rand -hex 64 -out key.bin

Do this every time you encrypt a file. Use a new key every time!

Update 25-10-2018

The key format is HEX because the base64 format adds newlines. The -passargument later on only takes the first line of the file, so the full key is notused. (Thanks Ken Larson for pointing this to me)

Encrypt the file with the random key

Use the following command to encrypt the large file with the random key:

openssl enc -aes-256-cbc -salt -in largefile.pdf -out largefile.pdf.enc -pass file:./bin.key

The file size doesn't grows that much:

  $ ls -larth  -rw-r--r-- 1 user group  40M Nov  9 21:14 Linux-Voice-Issue-020.pdf  -rw-r--r-- 1 user group  40M Nov  9 22:03 Linux-Voice-Issue-020.pdf.enc

It's encrypted however:

$ file Linux-Voice-Issue-020.pdfLinux-Voice-Issue-020.pdf: PDF document, version 1.4$ file Linux-Voice-Issue-020.pdf.enc Linux-Voice-Issue-020.pdf.enc: data

Encrypt the random key with the public keyfile

Use the following command to encrypt the random keyfile with the other personspublic key:

openssl rsautl -encrypt -inkey publickey.pem -pubin -in key.bin -out key.bin.enc

You can safely send the key.bin.enc and the largefile.pdf.enc to the otherparty.

You might want to sign the two files with your public key as well.

Decrypt the random key with our private key file

If you want to decrypt a file encrypted with this setup, use the followingcommand with your privte key (beloning to the pubkey the random key was cryptedto) to decrypt the random key:

openssl rsautl -decrypt -inkey privatekey.pem -in key.bin.enc -out key.bin

This will result in the decrypted random key we encrypted the file in.

Decrypt the large file with the random key

Once you have the random key, you can decrypt the encrypted file with thedecrypted key:

openssl enc -d -aes-256-cbc -in largefile.pdf.enc -out largefile.pdf -pass file:./bin.key

This will result in the decrypted large file.

Tags: ca, certificate, decrypt, encrypt, openssl, pki, ssl, tls, tutorials