Elements of Sampling: The Population, the Frame, and the Sampling Unit

By Neal B. Hitzig

E-mail Story Print Story

The Auditing Standards Board’s (ASB) “Approved Highlights” from its April 8–9, 2003, conference call meeting recommended that SAS No. 39, Audit Sampling, be revised to clarify whether the auditor should consider a sample customer as consisting of:

• The balance in accounts receivable for that particular customer, or
• All of the individual invoices that comprise the customer’s balance.

In making this recommendation, the ASB members demonstrated the broad ignorance of some of the most basic elements of sampling procedures that has resulted from the lack of professional training. In contrast, during the 1970s and ’80s all of the large firms conducted week-long seminars in statistical sampling for their audit staffs. Similar programs were available through the AICPA, state societies, universities, and other providers. Those seminars no longer exist. This article provides some of the basics of statistical sampling for auditors.

SAS 39 requires that “sample items should be selected in such a way that the sample can be expected to be representative of the population.” In order to properly apply this requirement, an auditor must understand what is meant by the terms population, frame, and sampling unit. If an auditor understands these terms and how they relate to a test of details of an account or a class of transactions, the auditor can properly execute an audit test of details and draw valid, defensible conclusions.

The Population and the Frame

The population is the set of all accounts or transactions about which an auditor wishes to draw a conclusion. Defining the population should follow immediately from defining the test objective, and is not a complicated task for the auditor. Nevertheless: “the auditor should determine that the population from which he draws the sample is appropriate for the specific audit objective” (AU 350.17). Although AU 350 repeatedly refers to the “population” when discussing sample planning and execution, it also states that “the auditor should project … results of the sample to the items from which the sample was selected” (AU 319.26; emphasis added). The text does not state that the results are to be projected to the population, but to the items. The reason for this subtle distinction is that auditors do not select a sample directly from the population itself, but from a representation of the population. That representation, known as the frame, provides the auditor with a convenient basis for identifying items to be included in a sample.

An accounting population usually is represented in the form of a list, such as the entries in a cash disbursements journal, a sales journal, accounts receivable detail, or a payroll file. The availability of a list or frame frequently simplifies the sample selection process. The choice of a frame is often based on convenience and accessibility. The most convenient frames are often in the form of computer data files, which lend themselves to the application of computer-assisted audit techniques and data retrieval.

An accounting population’s sampling frame need not be a list. Physical locations provided by floor plans, or other population identifiers, such as index cards, may also provide suitable frames. Physical inventory is an example of an accounting population for which the frame might consist of the warehouse locations of inventory items.

Occasionally, a sampling frame must be constructed by the auditor because no convenient representation exists. For example, an auditor may wish to select an audit sample from a population of sales orders. The orders are grouped by date and by sales location, and not by invoice or order number. The frame might consist of days and sales locations. If the population consists of 250 business days at 20 locations, the frame would comprise 5,000 location-days.

To rephrase AU 350.26 in light of the foregoing, “ … the auditor should project … results of the sample to the frame from which the sample was selected.” A sample can be projected only to the frame from which it was selected. The basis for that projection is the sampling unit.

The Sampling Unit

A population is composed of elementary units that are clustered into sampling units. The auditor’s choice of a frame leads to a decision about the sampling unit. A sampling unit in an audit application is the item that the auditor actually subjects to examination, whether by inspection, observation, or confirmation. The sampling unit is the item that is vouched or traced.

Statistical sampling procedures are extraordinarily flexible regarding the auditor’s choice of sampling unit. If the total recorded amount of the sampling units equals the total recorded amount of the population under examination, no technical issue exists regarding the validity of results, provided that selection and evaluation are performed properly. In substantive tests of details for which the auditor’s objective is to draw a conclusion about a total monetary amount, many auditors view a dollar (or other monetary unit) as the population’s elementary unit, an especially convenient viewpoint if the auditor is employing dollar-unit sampling. Auditors should be aware, however, that the auditing literature often mistakenly identifies the monetary unit as the sampling unit. In fact, the monetary unit is the elementary unit of such a population. The sampling unit is the account or the transaction, which is selected with probability proportional to its recorded amount.

The ASB’s stated issue regarding a possible revision to SAS 39 is basically a question about the choice of sampling unit (incorrectly referred to as “sample customer” in the cited meeting minutes). In an audit test involving receivables, the sampling unit may be an account, an open invoice, or even a line item on an invoice. The choice of sampling unit—and it is a choice—depends on considerations of practicability. As Leslie, Teitlebaum, and Anderson stated in Dollar Unit Sampling (Copp, Clark, Pitman, 1979): “Generally, the auditor should choose the (sampling) unit that can be most economically verified to yield persuasive evidence” (p. 154). The basic decision in a test of accounts receivable is whether to sample at the customer account level or at the open invoice level. The following are key considerations in arriving at a choice of sampling unit for a test of the existence assertion.

Accounts. Choosing the account to be the sampling unit is preferable for consumer accounts, such as credit cards. In this case, the auditor is directly confirming the net balance in a selected account. Commercial accounts, on the other hand, tend to be maintained on vouchers payable systems. Commercial customers may have difficulty confirming account balances. In its recommendation, the ASB raised the following questions:

If an unanswered confirmation request has an accounts receivable balance consisting of ten unpaid invoices, and the auditor performs satisfactory alternative procedures on only two of the ten invoices, how should the results be evaluated? Must the auditor reach a conclusion regarding every invoice in the sample? If they do not, may they assume that the entire balance is incorrect? (Auditing Standards Board Approved Highlights, April 8–9, 2003, Conference Call Meeting)

An auditor need not reach a conclusion regarding every invoice. If an auditor performs procedures on invoices within accounts, then she is performing tests on a two-stage sample, which should be evaluated as such. The problem with two-stage sampling is computational, especially if no misstatement is found in the second (invoice-level) stage sampling units. Failure to properly consider the contribution of the second stage to sampling risk understates the risk associated with the projected misstatement. For this reason, an auditor should strongly consider choosing the open invoice as the sampling unit.

Open invoices. This is the preferred approach if a company has a file of open invoices that can be directly accessed. Because open invoices comprise only debits to accounts receivable, the auditor must apply a separate test procedure for credits to accounts, especially a test for the completeness of the credits.

Because many businesses record their purchases in a vouchers payable system, they find it easier to confirm individual invoices than account balances, thus raising the likelihood of a response to a confirmation request. This choice of sampling unit may be applied with either equal probability (such as simple random sampling) or with probability proportional to size (dollar-unit sampling). Taking this approach renders moot the questions raised by the ASB’s recommendation above, because the sampling unit is the invoice.

Invoice line items. Dollar-unit sampling enables an auditor to choose an invoice line item as the sampling unit. This is an approach Leslie, Teitlebaum, and Anderson called subsampling. Typically, a computer program identifies the invoice and the dollar within the invoice in which the selected line item is located. The auditor generally identifies the line item by manually footing the invoice until he locates the selected item. The auditor only has to vouch that item, and every other selected line item in the sample. In dollar-unit sampling, the auditor projects the results associated with the selected line items by using the total book value of the frame as the representation of the frame size.

Because the frame is a representation of the population, it should be complete. That is, every sampling unit in the population should be included in the frame. The converse need not be true, however, as discussed below.

Overspecified Frames

If there are units in the frame that do not contain members of the population, they are not applicable to the conclusion that the auditor wishes to draw; they are irrelevant. Suppose, for example, selection of a random sample of sales invoices by invoice number in a substantive test yielded some invoices that had been voided or never used. If the auditor were testing for misstatement, then those voided invoices would have no misstatement. That is, the misstatement amount in each of the voided vouchers would be $0 for the purpose of obtaining a projection of the total amount of misstatement in the population of cash disbursements. Similarly, in the previously described frame of location-days, if there were no transactions at some location-days, no misstatement could exist therein.

To see why this is true, it is convenient to think in terms of performing a census, instead of a sample. The auditor views the test procedure as though every item in the frame were to be examined. In a census, the auditor simply totals the test results; there is no projection to be performed. By assigning a misstatement of zero to those frame members that are not population members, the correct population total misstatement is obtained. The oval area in Exhibit 1 represents the relevant portion of the frame, that is, the population. The figure shows that every member of the population is included in the frame.

A random sample selected from the frame will also yield a random sample of the population. The expected number of selections that belong to the population will be proportional to the population size vis-à-vis the frame size. An auditor will define a frame to be larger than, and inclusive of, the population when the exact size of the population is unknown (for example, a printed journal in which the number of entries per page varies). Even if the selection results in extraneous items, the sample projections to the frame will yield valid and unbiased projections of its subsumed population, although some efficiency may be lost as a consequence of the auditor’s selecting more items than are relevant to the examination. The converse, however, is not true, as discussed below.

Underspecified or Incomplete Frames

As stated above, when planning a sampling procedure, it is important to establish that every item in the population is included in the frame. If a frame is not complete, then some members of the population of interest to the auditor will have no chance of being included in the sample. This is a violation of AU 350’s requirement for representativeness, which requires that every item in the population under examination must have a chance of being selected. An incomplete frame can lead to biased estimates of the population value that is under examination if the auditor is not careful to distinguish between the size of the population and the size of the frame on which the selection of the sample was performed.

To see why this is true, consider again the taking of a census from an incomplete frame. Only those members of the population that are included in the frame will be totaled, understating the true population total. In Exhibit 2, the population is indicated by the oval, but only the rectangular portion is included in the frame. In this case, a random sample of the frame will not yield a random sample of the population, because elements of the population with no sampling units in the frame will have no chance of being selected.

When an auditor projects the results of an audit sample, it is the frame size that forms the basis for that projection, not the population size. Testing for completeness of a frame is problematic, and frequently depends on the auditor’s assessment of the company’s ability to maintain adequate documentation.

A Need for Improvement

Several modifications to SAS 39 are long overdue, but the ASB’s decision at its April 2003 meeting is the wrong one. Tinkering with questions about the choice of sampling unit suggests that the ASB does not understand the sampling basis for the issue it has raised. Such a misunderstanding is likely to result only in poor and confusing guidance.

The best way to improve practice in auditing testing is for the AICPA to revise its audit practice guide, Audit Sampling, which does not address the ASB’s current concerns, to include extensive material on this aspect of sampling. In addition, auditors should restore education and training in audit sampling techniques, especially statistical sampling.