File-AID Compare in Topaz Workbench
November 1, 2016 Data

How Shall I Compare Thee? An Overview of File-AID Compare in Topaz Workbench (part 1)

Compare, Search and Sort are base capabilities every programmer needs in their Swiss Army knife of development skills. Adept use of these core capabilities speeds up the development process and, more importantly, ensures code quality. Let’s take a closer look at how to get the most out of comparing data, specifically with Compuware File-AID Compare in Topaz Workbench.

Compare Basics

Compare is the quirky sibling in this triumvirate. For one thing the beauty is in the eye of the comparer—sometimes the user expects a complete match and sometimes they expect differences.  Neither is good nor bad except in how they meet or don’t meet the user’s expectations. And secondly, the user can influence the results of the compare by providing some background information on the data being compared.

file-aid compare algorithm

Figure 1: Compare Algorithm

Let’s spend a moment on the virtual compare engine shown in Figure 1.  When records match, life is good. However, a mismatch instantly complicates things: Does it represent a record that was changed? Is it an inserted record? Is it an indication that record(s) were deleted?

Influencing the Compare

This is where you can influence the compare by telling it the characteristics of the data files being compared. This occurs in the Record Matching tab of the File Compare dialog.

file-aid record matching

Figure 2: Record matching methods

The first choice is “Match records using a sort field or key field.” You can use this if you know your records are in a specific order based on a field or field(s). This is the default setting when a KSDS file is involved and can also be specified if you have flat files that you know to be in a specific order. As you can see in Figure 2, if your files are in a specific order we can detect changes, inserts and deletes immediately by comparing the keys of the mismatch (if the new record’s key is lower, it’s an insert; if the new record’s key is the same, it’s a change; if the new record’s key is higher, it’s a delete).

The second choice is a slight variation; the files are likely in order. This is a very common situation in flat files—they are generally in order but it is not mandatory that they be in order. In this case when we hit a mismatch, we will read ahead in the file in an attempt to find a matching record in the new file. If we don’t find a matching record, the mismatch is assumed to be a delete; if we find a matching record during the read ahead then these records in between are inserts. Since these files are not keyed (or to put it another way, the entire record is the key), this technique does not allow for detecting changed records. This is indicated in the dialog shown in Figure 2. How far does it read ahead? That is controlled via preferences (Windows → Preferences → Compuware → File-AID Compare).

The third choice causes the compare to do even more work on your behalf. In this case, you tell compare that the files are not in any specific order. When a mismatch is encountered, we will read both forward and backward to look for a match. If we can’t find a match, it is assumed to be a delete. Finding a match will mark the records in between as tentative inserts. Once the compare is completed, the tentative inserts that remain will be converted to inserts. Again, how far we read ahead/behind is controlled via preferences, and changes are not detected.

The fourth choice is a 1-1 compare. Records are compared and they either match or they don’t. All differences are marked as changes (except for the obvious case where there are a different number of records in the files).

Which Technique Is Best for You?

In most cases File-AID Compare in Topaz will select the appropriate technique: if a keyed VSAM file is involved we will use the key field method; if one isn’t involved we will do the fourth option, the 1-1 compare method. This should work in a vast majority of cases—for instance, the classic pass/fail type compare answering the question, “Do the files match?” For more nuanced cases where you are aware of the file tendencies, you may opt to use the one of the two middle choices. And if you are doing similar compares repetitively, you can set your preference as desired (or save and reuse the compares!).

Further Refinements

The other tabs in the File-AID Compare dialog give you even more control over the compare. The Fields tab allows you to specify (either formatted against a copybook or unformatted) the areas of the records you want compared. You can specify formatted fields based on a copybook or unformatted fields using length, offset and type. This is useful in cases where you are only concerned with the match/mismatch of a subset of the record.

The Selection Criteria tab allows you to control which records are to be compared. This is the same dialog used by Edit/Browse/Search. This is useful in cases where you are only interested in the match/mismatch of a subset of the records. Together, the Fields and Selection Criteria tabs can be used to control the breadth and depth of your compare.

Finally, the Limits tab allows you to add some throttles on your compare to improve the performance. If your compare hits 100 differences, do you really need to see if there is a 101st?

File-AID Compare in Topaz Workbench was designed to be as sophisticated as necessary while still being as simple as possible. The simplicity of interpreting the Compare results naturally follows suit. In a follow-up blog, we’ll cover how to make the most of the compare results you get.[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]