Netscape PKCS #11 Test Suite

Newsgroup: mozilla.dev.tech.crypto
Technical contact: Bob Relyea
Yell at the manager: Bob Lord

Purpose

The PKCS #11 Test Suite exercises cryptographic modules through Netscape's PKCS #11 interface layer. It is designed to exercise some of the PKCS #11 operations most commonly accessed from Netscape products. Running the suite against your PKCS #11 implementation will help expose incompatibilities that could otherwise be found by manual testing only.

Using the Suite

  1. Create a working directory. After you have built the suite, copy the following files into a directory on your hard drive:
    • regress executable for your platform
    • pk11test executable for your platform
    • pkcs11.reg, the regress specification file
    • internal, the sample configuration file

  2. If you don't already have security databases, create them with modutil. Running the test suite requires that you have a secmod.db file only; key3.db and cert7.db are not used by the suite. The databases should be in your working directory.

  3. Install the driver for your PKCS #11 implementation. Use modutil -add to add your PKCS #11 module to the security module database (secmod.db) in your working directory.

  4. Create a configuration file. A sample configuration file, called internal, is included in the package. The configuration file contains three entries, one per line. The first entry is the name of the token to be tested--this name can be acquired from modutil. The second line contains the name of the secmod.db file containing the security module database. This file may have been created with modutil, or it may have been created with Communicator. The third line is the PIN for the token, which will be used by the suite to login to the token.

    Here is another sample configuration file:

    My SuperCrypt Token
    dbdir/secmod.db
    myPIN
    

  5. Edit the regress specfile. In the "globalArgs" section of the regress specfile, put the name of the configuration file that should be used by the suite. For example, if the configuration file is called sprcrypt.txt, the globalArgs line would look like this:
    globalArgs=sprcrypt.txt
    
    Also, you may want to alter the location of test results. This is specified with the htmlout attribute. The default is NOW, which causes regress to construct a filename from the current date (e.g., 19980604.htm).

  6. Run the suite. Use the regress tool to run the test suite over all the cases in the specfile:
    regress specfile=pkcs11.reg
    
    Regress will write HTML results to the location specified with the htmlout attribute in the regress specfile.

Evaluating the Results

After the test suite completes, the results will be available in the HTML output file. There is one entry for each test. Each entry contains the test case number, the command line, a description of the test case, the start and end times of the test, and the result of the test. The result will be either PASS or FAIL. If the test fails, one of the following error codes will be returned:

Error Codes
Error CodeDescription
0no error
1 could not find token specified in configuration file
2key generation failed
3failed to create cryptographic context
4internal random number generation failed
5unknown mechanism
6error in configuration file
7failed to copy key to token
8cryptographic operation failed
9failed to finalize cryptographic operation
10ciphertext generated by token did not match ciphertext generated by Netscape security library
11failed to generate mechanism parameters
12recovered plaintext does not match original plaintext: Decrypt(Encrypt(x)) != x
13encryption is a null operation: Encrypt(x) == x
14failed to wrap private key
15failed to wrap symmetric key
16failed to unwrap symmetric key
17unwrapped key is different from original: Unwrap(Wrap(x)) != x
18failed to unwrap private key
19signature operation failed
20signature verification failed
21authentication failed with correct PIN
22authentication succeeded with incorrect PIN

Running the Test Program Manually

Individual tests can be run directly to assist in debugging. To run an individual test case, look up the command line from the HTML output file and type it at the command prompt. Alternatively, the program can be run in a debugger, which can trap on calls into your PKCS #11 driver. This is usually the most effective technique for debugging failed test cases.