How File Verification Works

Applies to TestComplete 15.47, last modified on January 20, 2023

When testing applications, you may need to compare various files. For example, you may want to make sure that your tested application generates the expected output data (for example, a list of orders). With TestComplete, you can verify files by using file checkpoints that perform a byte-to-byte comparison of the actual file contents and the baseline file stored in your project.

File checkpoints take the following parameters:

  • FileName - Specifies the name of the file to be verified.

  • AllowedDifference - Specifies the allowed amount of dissimilarity between two files.

File checkpoints can operate in the following two modes:

  • Strict verification - Verification is successful only if the files are identical. To perform verification in the strict manner, assign 0 to the Allowed Difference parameter.

  • Verification with a permissible variation - Verification is successful if the files are identical or if one file differs from the other and this level of difference is allowed.

Verifying Files in a Strict Manner

Run a checkpoint with the AllowedDifference parameter value set to 0 (the default value). The checkpoint will pass only if the actual and the expected contents are absolutely the same.

The following code shows how to strictly verify a file against its baseline copy stored in the Stores > Files collection:

JavaScript, JScript

Files.OrdersList_old.Check("C:\\Work\\OrdersList.txt");

Python

Files.OrdersList_old.Check("C:\\Work\\OrdersList.txt");

VBScript

Files.OrdersList_old.Check("C:\Work\OrdersList.txt")

DelphiScript

Files.OrdersList_old.Check('C:\Work\OrdersList.txt');

C++Script, C#Script

Files["OrdersList_old"]["Check"]("C:\\Work\\OrdersList.txt");

Verifying Files Taking Into Account the Allowed Difference

There can be situations where the verified and the expected files are not absolutely identical, but they still should be considered equal. For example, this can be necessary if the application under test generates a report that slightly differs from the reports generated during the previous runs. In this case, you can specify a value of the allowed difference and verify the files.

To do this, use the AllowedDifference parameter of your file checkpoints. Unlike the conventional allowed difference that is the “fingerprints” of a single file, the AllowedDifference parameter specifies the amount of dissimilarity between two files.

When verifying files, TestComplete calculates the difference, in bytes, and compares the resulting value with the AllowedDifference parameter value. If the actual difference is not greater than the expected one, TestComplete will report the files are identical.

To get the AllowedDeference value to use in tests:

  1. Run your file checkpoint in the strict comparison mode (the AllowedDifference parameter is set to 0).

  2. If the verification fails, TestComplete will calculate the number of bytes that differ and post that value to the test log:

    The number of bytes different in files reported by TestComplete

    Click the image to enlarge it.

  3. Use the value to set the AllowedDifference parameter for the further test runs.

The following code snippet demonstrates how to use the parameter:

JavaScript, JScript

  …
  var PrecalculatedDifference = "Value_From_the_Additional_Information_Panel";
  Files.OrdersList_old.Check("C:\\Work\\OrdersList.txt", PrecalculatedDifference);
  …

Python

...
PrecalculatedDifference = "Value_From_the_Additional_Information_Panel"
Files.OrdersList_old.Check("C:\\Work\\OrdersList.txt", PrecalculatedDifference)
...

VBScript

  …
  PrecalculatedDifference = "Value_From_the_Additional_Information_Panel"
  Files.OrdersList_old.Check "C:\Work\OrdersList.txt", PrecalculatedDifference
  …

DelphiScript

var PrecalculatedDifference : integer;
begin
   
  PrecalculatedDifference := 'Value_From_the_Additional_Information_Panel';
  Files.OrdersList_old.Check('C:\Work\OrdersList.txt', PrecalculatedDifference);
   
end;

C++Script, C#Script

  …
  var PrecalculatedDifference = "Value_From_the_Additional_Information_Panel";
  Files["OrdersList_old"]["Check"]("C:\\Work\\OrdersList.txt", PrecalculatedDifference);
  …

Verifying Small and Large Files

When the file is smaller than or equal to 50 Mb, TestComplete performs elaborate comparison (ignoring whitespaces, character displacement, and so on) and reports the first 200 000 lines in the log. When the file is larger than 50 Mb, TestComplete uses a more simplified comparison mechanism and reports only the first mismatch.

Verifying Files by Using Third-Party Tools

  1. Open the Tools > Options dialog and select the Engines > Stores item in the tree in the left pane.

  2. Select the Active check box in the Files diff utility group of options.

  3. In the Diff name box, specify the full path to the diff utility you want to use.

  4. If needed, in the Command line box, specify command-line parameters to be passed to the utility.

When you run your checkpoint, it will use the specified diff utility to compare file contents. The report generated by the diff utility will be posted to the test log:

File comparison report generated by a third-party tool in TestComplete

Click the image to enlarge it.

See Also

File Checkpoints
About File Checkpoints
Verifying Files With Variable Parts
Files

Highlight search results