3.3+59
Details about the algorithm for image comparison
Introduction
The classic Check image node is only minimally tolerant towards deviations. Using the default algorithm of comparing pixel by pixel it is not possible to check images that are generated in a not-quite-deterministic way or differ in size.
By using the attribute Algorithm for image comparison it is possible to define a special algorithm which is tolerant towards certain image deviations. The attribute must start with the algorithm definition in the form algorithm=<algorithm> followed by all required parameters, separated by semicolons. Parameters may be defined in any order, and use of variables is possible, e.g.:
algorithm=<algo>;parameter1=value1;parameter2=value2;expected=$(expected)
3.5.1+
Since QF-Test 3.5.1 the definition does not need so start with algorithm=<algorithm>
but can simply begin with <algorithm>.
It is also no longer necessary to define the parameter 'expected'. QF-Test uses a default value
if it is not set. Please see below for more information.
A detailed description of the available algorithms and their parameters is provided in the following section. For illustration, explanations are based on their effects on the following image:

In the related run log
(see section 7.1)
of a failed check you have the opportunity to analyze the results of the algorithm
as well as the result of probability calculation.
If you activate the option Log successful advanced image checks all tolerant
image checks will be logged for further analysis.
Description of algorithms
Classic image check
- Description
- The classic - or default - image check compares the color value of every single expected and actual pixel. If at least one expected pixel differs from the actual pixel the check fails. The option Tolerance for checking images defines the tolerance for comparing pixel values.
- Purpose
- This pixel-based check is suitable if you expect an exact image match with minimal tolerances or any deviations. Whenever your application renders the component not fully deterministically, this algorithm is not suitable.
- Example
- The classic image check doesn't transform the image, thus the result looks identical to the original image.

The classic image check is used when the Algorithm for image comparison attribute is empty.
Pixel-based identity check
- Description
- This algorithm is similar to the classic algorithm, but accepts an amount of unexpected pixels. It splits every pixel in it's three sub-pixels red, green and blue. Afterwards it checks every actual color value against the expected color value. The final result is the amount of identical pixels divided by the total amount of pixels. The calculated result is checked against an expected value.
- Purpose
-
If your images are not rendered fully deterministic but you accept a certain
percentage of unexpected pixels, this algorithm may be useful.
But it is not suitable, if the actual images are used to have shifts or distortions. - Example
-
The result image of the exemplary algorithm
algorithm=identity;expected=0.95
looks identical to the original image because this algorithm does not manipulate the image.

- Parameters
-
- algorithm=identity
- The 'Pixel-based identity check' should be used.
- expected (optional, but recommended)
-
Defines which probability you expect.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Pixel-based similarity check
- Description
- This algorithm splits every pixel in it's three sub-pixels red, green and blue. Afterwards it checks every actual color value against the expected color value to calculate a percental similarity. All percental deviations are added up and used for calculation. The final result is the average deviation over all color values and all pixels. The calculated result is checked against an expected value.
- Purpose
-
If your images are not rendered fully deterministic but you accept a certain
deviation, this algorithm is a possible candidate for your purpose.
If you accept deviations for some pixels, but the average deviation all over the image is small, this algorithm is also suitable.
But it is not suitable, if the actual images are used to have shifts or distortions. - Example
-
The result image of the exemplary algorithm
algorithm=similarity;expected=0.95
looks identical to the original image because this algorithm does not manipulate the image.

- Parameters
-
- algorithm=similarity
- The 'Pixel-based similarity check' should be used.
- expected (optional, but recommended)
-
Defines which probability you expect.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Block-based identity check
- Description
-
This algorithm partitions the image into quadratic blocks with a selectable size.
The color value of each of these blocks is calculated as the average of the color
values of the pixels the block contains.
If the width or height of the image is not a multiple of the block size, the blocks at
the right and bottom edge are cropped and weighted accordingly.
The actual blocks are checked against the expected blocks. The final result is the amount of identical blocks divided by the total amount of blocks. - Purpose
- This algorithm's purpose is to check an image which only differs at some parts but is identical at the remaining parts.
- Example
-
The exemplary algorithm
algorithm=block;size=10;expected=0.95
results in the following image:

- Parameters
-
- algorithm=block
- The algorithm 'Block-based identity check' should be used.
- size
-
Defines the size of each block.
Valid values are between 1 and the image size. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Block-based similarity check
- Description
-
This algorithm also partitions the image in quadratic blocks with a selectable size.
The color value of each of these blocks is calculated as the average of the color
values of the pixels the block contains.
If the width or height of the image is not a multiple of the block size, the blocks at
the right and bottom edge are cropped and weighted accordingly.
The color value of each expected block is checked against the actual block. Their color values are analyzed for percental similarity. The final result is the average similarity of all blocks with their weight taken into account. - Purpose
- This algorithm is suitable for checking images with similar color variances.
- Example
-
The exemplary algorithm
algorithm=blocksimilarity;size=5;expected=0.95
results in the following image:

- Parameters
-
- algorithm=blocksimilarity
- The algorithm 'Block-based similarity check' should be used.
- size
-
Defines the size of each block.
Valid values are between 1 and the image size. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Histogram check
- Description
- To create a histogram, an image is first broken into its three base colors red, green and blue. Then the color values for each pixel are analyzed to partition them into a definable amount of categories (known as buckets when talking about histograms). The actual fill level of each bucket is compared to the expected level. The result of the algorithm is a comparison of the relative frequencies of color categories.
- Purpose
-
Histograms are used for many scenarios. For example it is possible to check for
color tendencies or do brightness analyses.
However, histograms are not suitable for checking rather plain-colored images. - Example
-
The exemplary algorithm
algorithm=histogram;buckets=64;expected=0.95
results in the following image:

- Parameters
-
- algorithm=histogram
- A 'Histogram' should be used for this image check.
- buckets
-
Defines how many buckets to use.
Valid values are a power of 2 between 2 and 256. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Analysis with Discrete Cosine Transformation
- Description
-
The Discrete Cosine Transformation is a real-valued, discrete, linear, orthogonal
transformation which transforms the discrete signal from local range to frequency range.
After transforming an image, you can eliminate low-order (fast oscillating) frequencies. The remaining high-order (slow oscillating) frequencies with the steady component (zeroth frequency = 0*cos(x) + y) can now be analyzed. You can define how many frequencies per basic color should be used for this image check. You can also specify a tolerance to accept cosine-oscillations as identical which actually differ. Low-order frequencies get weighted less than high-order frequencies when calculating the result. - Purpose
- The Discrete Cosine Transformation is suitable for many kinds of image checks, which require certain tolerances. The more frequencies are used for analysis the sharper the image check is.
- Example
-
The exemplary algorithm
algorithm=dct;frequencies=20;tolerance=0.1;expected=0.95
results in the following image:

- Parameters
-
- algorithm=dct
- 'Analysis with Discrete Cosine Transformation' should be used for this image check.
- frequencies
-
Defines how many frequencies to analyze.
Valid values are between 0 (steady component only) and the area of the image.
The less frequencies are analyzed the more tolerant the check is. The tolerance is also dependent on the size of the image. - tolerance
-
Defines the (non-linear) tolerance for accepting different cosine-oscillations as
identical.
Valid values are between 0.0 and 1.0.
The value 1.0 means every image matches every other image, because the maximum difference of each frequency is tolerated. A value of 0.0 means frequencies only match if they are exactly the same. A value of 0.1 is a good starting point because only quite similar frequencies are accepted as identical. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Block-based analysis with Discrete Cosine Transformation
- Description
- When using this algorithm the image is first partitioned into quadratic blocks with a selectable size (see subsection 59.2.4). Afterwards every partition is analyzed using a Discrete Cosine Transformation (see subsection 59.2.7). The final result is the average of all results of these Discrete Cosine Transformations with consideration of the blocks and their weight.
- Purpose
- The Discrete Cosine Transformation used on the whole image deviates strongly in case of significant brightness differences occurring in the middle of the image because then the steady component (zeroth frequency), which is the highest weighted part, varies strongly. The partitioning circumvents this behavior because now only the affected partitions result in intense deviations while the other partitions stay untouched.
- Example
-
The exemplary algorithm
algorithm=dctblock;size=32;frequencies=4;tolerance=0.1;expected=0.95
results in the following image:

- Parameters
-
- algorithm=dctblock
- 'Blocks for analysis with Discrete Cosine Transformation' should be used for this image check.
- size
-
Defines the size of each block.
Valid values are between 1 and the image size. - frequencies
-
Defines how many frequencies to analyze.
Valid values are between 0 (steady component only) and the area of a block.
The less frequencies are analyzed the more tolerant the check is. The tolerance is also dependent on the size of the image. - tolerance
-
Defines the (non-linear) tolerance for accepting different cosine-oscillations as
identical.
Valid values are between 0.0 and 1.0.
The value 1.0 means every image matches every other image, because the maximum difference of each frequency is tolerated. A value of 0.0 means frequencies only match if they are exactly the same. A value of 0.1 is a good starting point because only quite similar frequencies are accepted as identical. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Bilinear Filter
- Description
-
This algorithm shrinks the image to a chooseable percental size.
Afterwards the image gets resized to its original size by use of a bilinear filter.
This filter effects a blurring, because every color value is calculated by use
of neighbor pixels.
The final result is the average deviation over all color values and all pixels of this transformed images. - Purpose
- Depending on the chosen sharpness the images loose any desired image information. Thus this algorithm is valuable for nearly any scenario.
- Example
-
The exemplary algorithm
algorithm=bilinear;sharpness=0.2;expected=0.95
results in the following image:

- Parameters
-
- algorithm=bilinear
- A 'Bilinear Filter' should be used for this image check.
- sharpness
-
Defines the sharpness of this bilinear filter.
Valid values are between 0.0 (complete loss of information) and 1.0 (no loss of information).
The sharpness is a linear parameter. This means a value of 0.5 eliminates exactly half (plus minus rounding to entire pixels) of information. - expected (optional, but recommended)
-
Defines the minimal match probability for the check to succeed.
Valid values are between 0.0 and 1.0. If not defined, use 0.98. - resize (optional)
-
Defines, if the actual image should be resized before calculation to match
the size of the expected image.
Valid values are "true" and "false". - find (optional)
-
Defines an image-in-image search.
A detailed description of this parameter can be found in subsection 59.3.1.
Description of special functions
Image-in-image search
- Description
-
The image-in-image search allows to find an expected image within a (larger) image.
The check is successful when the expected image can be found anywhere using the defined algorithm.
Furthermore, you can determine the position of the match.
The following combination of parameters are valid: - Purpose
- The image-in-image search allows to compare images if you don't know the exact position and thus cannot define an offset. The search can be combined with any algorithm and is thus valuable for any purpose.
- Example
-
The exemplary algorithm
algorithm=similarity;expected=0.95;find=best(resultX,resultY)
uses pixel-based similarity check (see subsection 59.2.3) to find an image of Q as part of the full image. The got image with highlighted region can be found within the run log. Besides, the variables resultX and resultY are set to the location of the found image.
find=best
or find=anywhere
find=best(resultX, resultY)
or find=anywhere(resultX, resultY)


- Parameters
-
- find=best
- Defines that the best match should be used.
- find=anyhwere
-
Defines that the first match which exceeds the expected match probability should be used.
The image-in-image search uses multiple threads and thus finding anywhere is non-deterministic. - resultX
-
resultX
is the name of a QF-Test variable which holds the x-position of the found image.
If a variable for the x-position is defined, a variable for the y-position has to be defined as well (see syntax above). - resultY
-
resultY
is the name of a QF-Test variable which holds the y-position of the found image. If a variable for the y-position is defined, a variable for the x-position has to be defined as well (see syntax above).