Debugging

Introduction

Cryptography necessarily makes data unreadable, and as a result, this makes debugging especially challenging. MTE is no exception to this. It is very difficult to determine what might be wrong because the developer typically gets no sense of whether the data is "close" to correct or not. The result is either correct or not.

MTE does have some features that can assist the developer in determining what is wrong. Some of these features depend on which features of MTE you are using, but some apply to all MTE configurations. The following sections describe common problems and strategies for determining the cause and resolving the issue.

General Suggestions

Status

Almost all MTE functions return a status. It is extremely important that the status is always captured and checked. Ignoring the status, even for operations that may almost never fail, makes debugging very difficult because the error you get later may have been caused by an earlier error, which if checked, would be much easier to figure out. Once you get to the token does not exist error, there are many possible causes, some of which are caused by earlier unhandled errors.

Licensing

Always call the license init function and check the status, even if you do not require a license code (e.g., for the trial version). When a license code is not required, the arguments to the function are just ignored, but you will get a success status, and your code will be future-proofed in case you need a license code later.

MTE Options

The MTE library can be built with runtime options support or buildtime options. The buildtime options are used by most customers to reduce the library size and make it even more efficient for the exact options they need. If buildtime options are used, the MTE encoder and decoder must be initialized with the options that are actually built in; if mismatched options are passed an error or crash will occur. If your library has runtime options capability, any options may be used.

Most language interfaces contain constructor or initializer functions that are written to automatically use the correct set of buildtime options if that is how the library was built or a default set of options if runtime options are enabled. Always use these helpers where possible to avoid the possibility of passing the wrong options. This future-proofs your code to work against any set of buildtime options if you decide to change at some point.

The C language interface does not have this capability, so it is instead recommended to write the same logic in your code. Use the MTE functions that can query the built-in options to determine at runtime the appropriate options to use. Refer to the C++ language interface for an example, or look at the C demo applications in the SDK.

Token Does Not Exist

The token does not exist error is by far the most common problem. There can be many causes for this, including attacks or communications issues, but during development, the cause is almost always a programming error of some kind. Following are the most common errors.

Entropy Reuse

A common problem involves developers attempting to reuse entropy when creating both an encoder and decoder on an endpoint for two-way communication. In almost all language interfaces the entropy is zeroized when used, so you cannot pass the same entropy to two different MTE instances as the second one will get an all-zero entropy, not the intended entropy.

This will lead to token does not exist errors if the paired endpoint initialized in the opposite order. Or worse, it will lead to using all-zero entropy, which gives no security.

The correct solution to this problem is not to make a copy of the entropy. The correct solution is to have separate entropy for each communication direction.

Communication Issues

Unlike basic encryption, the MTE technology will generally fail completely if the decoder receives something other than exactly what was encoded. This could be changed data or truncated data.

A useful technique is to print out the encoded version immediately after encoding and before transmission and printing it again immediately before attempting to decode. If there is any difference, this must be resolved.

If you are using the Managed-Key Encryption Add-On in chunk mode, make sure you are sending all the chunks. Make sure you are sending the chunks in the correct order. Finally, make sure you are sending the results of the finalize function as well, not just the encrypted data.

Use Save State

The MTE save state feature is a very useful debugging feature as well as serving its main purpose of allowing the state to be saved and restored later. The saved state actually contains every bit of information MTE needs to encode or decode a message. As an aside, it is important to stress again that the saved state must be protected as if it was the entropy or an encryption key or other similar piece of critically sensitive data.

The internal state of the encoder and decoder will match when they are in sync and will not match when they are not in sync. If the encoder and decoder are not in sync, the decoder will not recognize the encoded message, resulting in the token does not exist error.

A useful technique with saved state is to put calls to save state before/after certain operations on both the encoder and decoder and compare the saved states at each point. Where the saved state mismatches you will start to have problems, and you can then see why. The key points to check the state are:

• After instantiation. The instantiation call uses the entropy, nonce, and personalization string to create the initial state. A call to save state after the instantiation gives you that initial internal state. The encoder and decoder must have the same initial state. If they do not, the most likely problem is that they did not both get exactly the same entropy, nonce, and/or personalization string.
• Before encode/decode. A call to save state before encode or decode shows the current internal state that will be used to perform the encode or decode. As with instantiation, the states must be the same, meaning the encoder and decoder are in sync.

There is one caveat to the saved state being identical. If you are using async sequencing and some async messages have been seen, the decoder's saved state includes information about that in the 5th through 12th bytes, and the encoder will have all zeros in those positions. However, all other bytes must be the same.

Use A Debug Decoder

It is sometimes helpful to create a decoder just for debug purposes and simply decode immediately after encoding to make sure that works. To make this work you will have to copy your entropy to avoid the [entropy reuse](#entropy Reuse) issue. Make sure you remove this debug decoder once the problem has been fixed.

If the decode immediately after encode fails, this almost always means there is a status not being properly checked, resulting in an error earlier somewhere that is the actual cause of the problem you are seeing later as token does not exist.

Invalid Input

The invalid input error has a couple different meanings, depending on the context.

Initialization

In an initialization function call, the invalid input error means the wrong options were passed. Use the techniques described to avoid initialization errors.

Encode/Decode

In an encode or decode call, the invalid input error typically means that MTE was able to determine the data was cut off. Check for communications issues.

Sequencing

If sequencing is enabled, some additional debugging techniques are possible. Sequencing also has its own set of errors. It is also important to note that when using sequencing with saved state, the accesses to the state must be serialized since the state contains the sequencing information. Attempting to decode multiple messages in parallel will lose the sequencing information in the saved state, leading to very difficult to find problems.

Sequence Mismatch

The sequence mismatch is the most common error and tells you that things are not in sync. There are a couple techniques you can use to debug this.

Using the save state technique, you can check the sync state. The saved state also contains the next sequence number in the first four bytes, so that can be easily compared.

The encoded message includes the sequence number as the last four bytes in cleartext, so it can be examined after encoding and before decoding. In some cases, communications errors like truncation cause a mismatch because MTE just reads the last four bytes as the sequence number, so if it's random data or part of a token it will cause the sequence window to be exceeded. Check for communications issues.

Message Length

Sequencing requires all messages to be the same length in order to recover from a skipped message. A common problem is not ensuring all messages are the same length. There are three ways to accomplish this:

• Use the Fixed-Length Add-On. It will ensure all messages are the same length. This is practical if all messages are fairly short, and can be easily padded unambiguously.
• Use the Managed-Key Encryption Add-On. This add-on allows variable-length messages that are fixed-length from MTE's point of view, because the only part of the message that is tokenized is the authentication hash, which is a fixed length. This is the best option for larger data.
• Design your messages or otherwise write your code in such a way that the messages are always the same length. This can be difficult for many applications, but for some it is easy and offers the least overhead, which may be important in resource-constrained environments.

The sequence mismatch error is often seen when the message length was not constant.

Hash Digest Mismatch

When using using a decoder, the meaning of the mte_status_digest_mismatch error is that the MTE has detected that the decrypted data does not match the data originally encrypted by the MKE encoder.

The most likely causes of this error are:

1. Missing some part of the encrypted data
2. Chunks of encrypted data processed out of order
3. Corruption of the encrypted data

To debug this problem, it is recommended that the encrypted data is saved to a file on the encoding side immediately after encoding, and the encrypted data is saved to another file on the decoding side immediately before decoding. If those two files differ, you know there is a communication problem causing the encrypted data to not be presented completely, in order, or correctly to the decoder. Fixing the communication problem should resolve the MKE problem.

NOTE

This error means the authentication tag, which is MTE encoded, was successfully decoded. If there was a problem with the authentication tag, it is more likely to receive a mte_status_token_does_not_exist error, as is typically seen in the core decoder. The digest mismatch instead is talking about the encrypted data part of the payload.