Encryption Padding

Question:
(This is a question about the amount of expansion that occurs when doing encryption.)

Here is a bit of a technical question for you, since I will be storing my data in MS SQL database I need to anticipate column size and types. If I have a column right now type nvarchar size 50, how much should I increase it by to prevent a possible overload?

The answer will depend on the maximum number of characters one character is before encryption for example lets say I encrypt the letter A, what’s the longest amount of characters it will become? Will it be 3 characters instead of 1, etc? Also please answer this question for all types of encryption you have available.

Answer:
I will answer this question for the symmetric encryption algorithms implemented by Chilkat. These answers can also be applied (in general) to other encryption implementations.

Expansion due to Padding:
The Chilkat encryption component/library implements the AES (also known as Rijndael), Blowfish, and Twofish symmetric encryption algorithms. (3DES will be coming soon) Symmetric means that the same key is used for both encrypting and decrypting. These algorithms are all block ciphers, meaning that encryption is performed in blocks. Input data must be padded to a multiple of the block size. The Blowfish block size is 64-bits (8 bytes). The AES and Twofish block sizes are 128-bits (16 bytes). Note: the block size is different than the key size. The key size is what indicates the strength of the encryption.

The amount of padding depends on a combination of the selected padding scheme and the encryption algorithm. The padding schemes implemented by Chilkat are listed below:

1. RFC 1423 padding scheme: Each padding byte is set to the number of padding bytes. If the data is already a multiple of the block size, additional bytes are added, each having the value 0×10 (or 0×08). (This is the default padding scheme).
   The maximum expansion due to padding is equal to the block size.
2. FIPS81 (Federal Information Processing Standards 81) where the last byte contains the number of padding bytes, including itself, and the other padding bytes are set to random values. If the data is already a multiple of the block size, additional bytes are added — all of them random except the last byte.
   The maximum expansion due to padding is equal to the block size.
3. Each padding byte is set to a random value. The decryptor must know how many bytes are in the original unencrypted data. If the input data is already a multiple of the block size, no padding is added.
   The maximum expansion due to padding is equal to the block size-1.
4. Pad with NULLs. (If already a multiple of 16, no padding is added).
   The maximum expansion due to padding is equal to the block size-1.
5. Pad with SPACE chars(0×20). (If already a multiple of 16, no padding is added).
   The maximum expansion due to padding is equal to the block size-1.

Expansion due to Encoding:
Encrypted data is binary. If you want to store the encrypted data in a string variable, it needs to be encoded using an algorithm such as Base64, Hex, URL-encoding, etc. The most common choices are either Base64 or Hex. Hex encoding doubles the size of your output. Each byte is represented by two characters (0-9A-F). Base64 encoding is more efficient because it uses a 64 character alphabet (whereas hex encoding uses a 16-character alphabet). Each 3 bytes of encrypted output get encoded as 4 characters in a base64 encoded string. If the encrypted output is not an exact multiple of 3, you’ll have up to 2 extra ‘=’ characters added. To summarize: Hex encoding doubles the size of your padded output, and Base64 encoding increases the size of your padded output by 4/3rds (+ 1 or two extra chars).

Note: Base64 output is not URL-safe. Use hex encoding if you need a string that is URL-safe.

It is fairly common to compress data first, then encrypt, and then hex-encode in cases where you want the encrypted data to be placed in a URL. This only makes sense if the clear-text (non-encrypted data) is large enough such that compression is meaningful, and small enough such that it can fit in a URL. Also note: Never try to compress after encrypting. Encrypted output has the same characteristics as random data, and compression algorithms depend on data having non-random repeated patterns. Always compress first, then encrypt.

Character Encoding:
Encryption algorithms work on bytes, not characters. If you are encrypting a string, what exactly are you encrypting? Is it a 2-byte per character Unicode string, a 1-byte per character iso-8859-1 string, or a string that might be encoded such that different characters have varying byte-length representations? If you are encrypting a string, and your encrypted data turns out to be roughly twice the size you expected, make sure you weren’t encrypting Unicode. You’ll probably want to convert the string to a byte array in an encoding (ANSI perhaps) and then encrypt. (The Chilkat encryption component provides a Charset property to make this easy.)

To put it all together, let’s work through an example. How much expansion occurs when AES encrypting a 19 character ANSI string using RFC 1423 padding and encoding the result as a Base64 string? On my computer, ANSI equates to Windows-1252, so my input is 1 byte/char — no problem there. AES is an 16-byte block cipher, so 19 bytes is padded to 32 bytes. The Base64 encoding expands the padded output by 4/3rds, so the output string is 42.667. Obviously we can’t have 2/3rds of a byte, so we round up. (This is the extra 1 or two bytes for non-multiples of 3) The result should be 43 bytes.