Security Objects Tokenization Quickstart (Format_V2)

1.0 Introduction

This article describes how to implement security object tokenization using the Fortanix DSM REST API with the Format_V2 (format_v2) schema.

NOTE

Fortanix recommends using format_v2 for all new tokenization operations.

2.0 Using API Schema

2.1 Format Definition (Format_V2)

Fortanix DSM supports a format_v2 field, which provides an updated format-based definition for describing the structure of a token. The format_v2 model is the successor to the original format definition and should be used for new tokenization use cases.

When creating a security object for format-preserving encryption (FPE), the request includes an fpe field. The value of this field defines how the token is structured and how tokenization and detokenization are performed.

The following code block illustrates the value for the fpe field in a JSON format using the format_v2 schema:

fpe {
  "format_v2": <JSON object containing datatype format goes here!>,
  "description": "An optional string describing this data format"
}

The format_v2 field specifies the token format as a collection of one or more format variants, each describing a valid structural representation of the data. Each variant defines how the input is structured and processed, including which portions of the data are encrypted. Together, these variants describe the complete set of acceptable input formats for tokenization.

The following are the types of sub-parts of the format_v2 JSON tree structure:

• Variants: It represents a complete and valid format for a token.

  • Variants are represented by a JSON object contained within the variants array.

  • Each variant defines an ordered list of Sections.

  • A token is valid if it matches one complete variant.

  • If a variant has a variable length, prefix and suffix constraints may be applied.

• Section: It represents a logical portion of the token.

  • Sections are evaluated sequentially and may contain one or more Groups.

  • A section is represented by a JSON object contained within the sections array.

• Group: It represents a concatenation of multiple Codes that together form a structural unit within a section.

  • Groups are evaluated in sequence within a section.

  • A group is represented by a JSON object containing a codes array.

• Code: It represents the smallest unit of token generation and validation.

  • Codes are evaluated sequentially within a group.

  • Each code defines how a portion of the token is produced and may represent either an encrypted value or a literal value.

  • A code is represented by a JSON object containing a words definition.

• Words: It specifies a set of fixed-length strings. The following word types are supported:

  • Alphabet: It represents single characters.

  • Integer ranges: It represents a union of positive integer ranges.

  • Custom: It represents a custom set of fixed-length strings.

The leaf nodes of the Format V2 structure are code definitions that contain words.

The format_v2 field consists of a hierarchical structure where a data type is divided into variants, sections, groups, and codes. These components together define the complete structure of the token.

In addition to specifying the general format of a token section, Format V2 allows constraints to be applied by defining appropriate codes.

For example:

• Numeric values restricted to specific ranges

• Fixed-width numeric values using padding

• Exclusion of invalid values by omitting them from defined ranges

• Validation of structured numeric identifiers such as Social Security Numbers

The following code block illustrates the structure of the fpe field using an example:

"fpe": {
    "format_v2": {
      "mode": "preserve_length",
      "input_processing": "passthrough_unknown",
      "variants": [
        {
          "sections": [
            {
              "groups": [
                {
                  "codes": [
                    {
                      "words": {
                        "integer_ranges": {
                          "ranges": [[0, 9]],
                          "padding_required": true
                        }
                      },
                      "min_repetitions": 12,
                      "max_repetitions": 19
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  }

American Social Security Number:

The SSN is a nine-digit number assigned to American citizens, permanent residents, and eligible non-immigrant workers. This number is used to report wages to the government, track Social Security benefits, and for other identification purposes.

Some of the constraints are listed as follows:

• The format of this number is XXX-XX-XXXX.

  • The first section (XXX) must be between 001–665 or 667–899 (with 666 and 000 disallowed) and always zero‑padded to three digits.

  • The second section (XX) must be between 01–99 (00 not allowed) and zero‑padded to two digits.

  • The third section (XXXX) must be between 0001–9999 (0000 not allowed) and zero‑padded to four digits.

  • Hyphens remain fixed between sections and are preserved exactly during tokenization and detokenization.

• It is a hyphen-separated nine-digit number.

• Each group of digits is represented as an encrypted code defined using numeric ranges.

• The hyphen (-) separators are represented as literal codes.

• Fixed-length formatting is achieved through padding_required.

The following format_v2 schema defines a structured numeric pattern similar to an SSN format, with specific allowed numeric ranges for each section and a fixed separator (-):

 {
  "format_v2": {
    "variants": [
      {
        "sections": [
          {
            "groups": [
              {
                "codes": [
                  {
                    "words": {
                      "integer_ranges": {
                        "ranges": [
                          [
                            1,
                            665
                          ],
                          [
                            667,
                            899
                          ]
                        ],
                        "padding_required": true
                      }
                    }
                  },
                  {
                    "words": {
                      "custom": {
                        "list": [
                          "-"
                        ]
                      }
                    }
                  },
                  {
                    "words": {
                      "integer_ranges": {
                        "ranges": [
                          [
                            1,
                            99
                          ]
                        ],
                        "padding_required": true
                      }
                    }
                  },
                  {
                    "words": {
                      "custom": {
                        "list": [
                          "-"
                        ]
                      }
                    }
                  },
                  {
                    "words": {
                      "integer_ranges": {
                        "ranges": [
                          [
                            1,
                            9999
                          ]
                        ],
                        "padding_required": true
                      }
                    }
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}

2.2 Create AES Security Object with FPE Field

Perform the following steps to create a tokenization key:

1. Send a request to the Fortanix DSM endpoint with the fpe field in the security object request. The fpe field contains a JSON object that defines the token format and the behavior used for tokenization and detokenization.

   NOTE

   In the example below, the token format is defined using a hierarchical structure composed of one or more format variants. Each variant describes a valid structural representation of the token, including which portions are encrypted.

   The following is a sample JSON request for creating an AES key to tokenize a Social Security Number (SSN) using the format_v2 schema.

{
  "name": "<sobject name>",
  "description": "<sobject description>",
  "obj_type": "AES",
  "key_ops": [
    "ENCRYPT",
    "DECRYPT",
    "APPMANAGEABLE"
  ],
  "key_size": 256,
  "fpe": {
    "format_v2": {
      "variants": [
        {
          "sections": [
            {
              "groups": [
                {
                  "codes": [
                    {
                      "words": {
                        "integer_ranges": {
                          "ranges": [
                            [1, 665],
                            [667, 899]
                          ],
                          "padding_required": true
                        }
                      }
                    },
                    {
                      "words": {
                        "custom": {
                          "list": ["-"]
                        }
                      }
                    },
                    {
                      "words": {
                        "integer_ranges": {
                          "ranges": [
                            [1, 99]
                          ],
                          "padding_required": true
                        }
                      }
                    },
                    {
                      "words": {
                        "custom": {
                          "list": ["-"]
                        }
                      }
                    },
                    {
                      "words": {
                        "integer_ranges": {
                          "ranges": [
                            [1, 9999]
                          ],
                          "padding_required": true
                        }
                      }
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    "description": "<format description>"
  },
  "expirationDate": null,
  "enabled": true,
  "group_id": "<sobject group id>"
}

2.3 Tokenize by Calling Encrypt API

When the security object is created, the response contains the "kid" field. This field is required for calling the encrypt or decrypt APIs. The following example illustrates how to tokenize the number 123-12-1234.

The API to tokenize is /crypto/v1/encrypt.

JSON Request:

{
  "alg": "AES",
  "plain": "MTIzLTEyLTEyMzQ=","mode": "FPE",
  "key": {
    "name": "17-May-Key"
  }
}

Where, the value of the plain field is base64 encoded SSN. The result of decoding the "plain" field is “123-12-1234”, the original data provided.

Response:

{
 "kid": "<tokenization sobject kid>",
 "cipher": "NDU2LTU2LTQ1Njc="
}

The “cipher” is a base64 encoded value of the token. The result of decoding the"cipher" field is “456-45-4567”.

2.4 Detokenize by Calling Decrypt API

The API to de-tokenize is /crypto/v1/keys/{kid}/decrypt.

JSON Request:

{
  "alg": "AES",
  "cipher": "NDU2LTU2LTQ1Njc=","mode": "FPE"
}

Response:

{
 "kid": "<tokenization sobject kid>",
 "plain": "MTIzLTEyLTEyMzQ="
}

The "plain" field is the base64 encoded value of the original data. The result of decoding the "plain" field is “123-12-1234”, the original data provided.

NOTE

The /crypto/v1/keys/{kid}/decrypt API is not supported for Irreversible Tokenization key type.

3.0 References

• User's Guide: Tokenization

• Security Objects Tokenization Examples (Format_V2)

• Security Objects Tokenization Best Practices