The Use of Control Self-Assessment by Independent Auditors

By Gilbert W. Joseph and Terry J. Engle

E-mail Story Print Story

CSA activities also have the potential to improve the efficiency and effectiveness of independent financial statement audits in response to changing demands on independent auditors. While independent auditors can benefit from CSA activities, little evidence indicates the extent to which independent auditors avail themselves of these benefits. The authors investigated the uses of CSA by independent auditors, as well as the perceptions about the value of independent-auditor involvement with CSA activities.

Approaches to CSA

The Institute of Internal Auditors (IIA) defines CSA as a process through which internal control effectiveness is examined with the objective of providing reasonable assurance that all business objectives are met. The employees performing CSA work are in the functional area being examined rather than upper-level managers that are above the system of internal controls. These employees have a wealth of information about internal controls and fraud (if it exists). While internal (or independent) auditors can be involved with CSA initiatives, auditors do not “own” the process and do not make the assessments and evaluations. The most common approaches to performing CSA activities are facilitated team meetings and CSA surveys.

• Facilitated team meetings are the most popular form of CSA. The facilitated sessions consist of six to 15 employees who are subject on a day-to-day basis to the internal controls being evaluated. A trained facilitator guides the meeting, and another individual records the activity. Anonymity can be promoted by using “groupware” software.
• The survey approach uses questionnaires to elicit data about controls, risks, and processes. It differs from traditional internal control questionnaires used by auditors because the operational employees (not the auditors) use the survey results to self-evaluate the controls or processes.

Relevant Experiences of the Internal Auditing Profession

The internal auditing profession has widely embraced the use of CSA. The IIA supports internal auditors who use CSA to achieve internal auditing objectives and recognized the importance of CSA by creating a Control Self-Assessment Center. The IIA does not prohibit internal-auditor participation in the CSA activities of auditees due to independence concerns, and in practice, organizations have not had independence issues when internal auditors have participated in a variety of ways (e.g., as facilitators of CSA meetings).

Members of the internal auditing profession have considerable experience in successfully using CSA in the internal auditing process. These experiences are relevant to external auditors because they face many of the same challenges. For example, both external and internal auditors must effectively evaluate internal control systems, effectively make fraud risk assessments, understand their auditees’ operations and business, and focus auditing resources based on risk. In addition, both types of auditors are going to be increasingly responsible for assessing enterprise risk management (ERM) systems under the new Committee of Sponsoring Organizations of the Treadway Commission (COSO) Enterprise Risk Management–Integrated Framework. John Flaherty, 2004 COSO chairman, and Tony Maki, COSO Advisory Council chair, noted: “[C]ompanies may decide to look to this enterprise risk management framework both to satisfy their internal control needs and to move toward a fuller risk management process.” The COSO ERM framework explicitly recognizes that it is intended to assist organizations in achieving their objectives, including the reporting objectives that are so relevant to independent auditors.

Examples. The internal auditing profession has aggressively promoted the use of CSA, and internal auditors at many different organizations have used CSA to improve the internal auditing process in ways that are relevant to external auditors. For example, the internal auditors at Cargill Inc. have integrated CSA into the auditing process for more than six years and have successfully used it to assess auditee risks at the front end of an audit, to better learn the auditee’s business, to uncover ethics violations, and to improve the evaluation of essential “soft” controls (e.g., quality of communications, and the ability to discuss sensitive issues with the next level of management). (For more information, see Christina Brune and Diane Sears Campbell, “Integrating CSA as Another Audit Tool,” CSA Sentinel Online, IIA Control Self-Assessment Center, October 2002.)

The internal auditors at Pennsylvania State Employees Credit Union have demonstrated that CSA can be successfully used with that organization’s ERM system. A senior internal auditor reported that: “The auditors’ knowledge of risks and controls throughout each business unit has also increased, which has improved the efficiency and effectiveness of audits and with developing the audit schedule.” (See T.L. Heimbaugh, “CSA—An Integral Part of the Process,” CSA Sentinel Online, IIA Control Self-Assessment Center, February 2004.)

Walter Stachnik, as Inspector General for the SEC, extensively used CSA to achieve a variety of internal auditing objectives. In describing his experiences, Stachnik stated: “CSA is not necessarily a faster or easier tool to use than traditional auditing. On the other hand, we get a much deeper understanding of the critical factors involved when we use CSA to evaluate soft controls. The quantitative results of traditional auditing are easier to defend sometimes, but the qualitative understanding of issues supplied by CSA generally adds significantly more value to the control environment.” In describing an audit that focused on a formal communication process at the SEC, he said: “Traditional auditing methodology can be used to assess soft controls like communication, but the results are frequently off-the-mark. This CSA on communication delivered a significantly different, but much more useful result than what we previously attained with traditional methods” (Jonathan Figg, “The Power of CSA,” Internal Auditor, August 1999). While external auditors must perform limited testing to corroborate CSA-generated evidence, the experiences of these internal auditors are obviously relevant to achieving many auditing objectives.

The Value of CSA to Financial Statement Audits

Independent auditors face a changing environment and higher expectations, particularly in the areas of internal control evaluations and fraud detection. After several amendments, AU 319, Consideration of Internal Control in a Financial Statement Audit, now requires auditors to gain an adequate understanding of all five components of control to adequately plan the audit. The AICPA has also promulgated Statement on Auditing Standards (SAS) 99, Consideration of Fraud in a Financial Statement Audit, which requires financial statement auditors to evaluate the potential for fraud. SAS 99 clearly recognizes the importance of effective internal control evaluations and calls for the development of new auditing approaches to help fulfill auditors’ expanded responsibilities. Significant new control responsibilities are also part of the Public Company Accounting Oversight Board (PCAOB)’s Auditing Standard (AS) 2, An Audit of Internal Control Over Financial Reporting Performed in Conjunction with an Audit of Financial Statements, which requires that financial statement auditors audit and attest to the fairness of management’s assessment of their internal control system over financial reporting.

In this environment, auditors need both hard and soft controls. Both types are found in all five components of the COSO control framework (i.e., control environment, risk assessment, control activities, information and communication, and monitoring). Auditors can effectively evaluate hard controls (e.g., bank reconciliations, credit approvals) by traditional auditing procedures such as reperformance, confirmations, inspections, and physical observation. These procedures are far less effective in evaluating critical soft controls such as management’s integrity and ethical values, management’s commitment to competence, or management’s philosophy and operating style. For example, when evaluating client management’s integrity and ethical values, there may be no documents to examine, no confirmations available from third parties, and no recalculations to perform. Soft controls are often reflected in employees’ perceptions and impressions about management’s attitudes and intentions that can only be investigated using the “inquiry” audit procedure.

CSA expands the reliability of the inquiry audit procedure to supplement traditional tests of controls. With CSA, the auditor is not obtaining the impressions of only a few employees, but rather the anonymous, composite impressions of at least six to 15 operational employees or lower-level operational managers, which reflect repeatable attitudes over time. When many knowledgeable employees agree on an issue, the evidence is highly corroborated and typically superior to evidence gathered from selected individuals while completing traditional internal control questionnaires, narratives, or flowcharts.

The language from paragraph 97 of AS 2 demonstrates this premise: A “signature on a voucher package to indicate that the signer approved it does not necessarily mean that the person carefully reviewed the package before signing it.” AS 2 recommends that the auditor test the control by reperforming it, and that the auditor perform inquiries of the person responsible for approving voucher packages and that person’s supervisor regarding what they look for and any history of errors made in these judgments. A better approach would be to use the results of a CSA evaluation of the procedures surrounding voucher package approval and processing, and to involve all employees affected by those procedures. The CSA evaluation would identify changes or consistencies in procedures over the period under audit, assess procedural problems, identify errors and irregularities experienced, and recommend control and procedural improvements. This would require less effort on the part of the independent auditor (i.e., improve audit efficiency), because the auditor would use the work of others (company personnel in addition to internal auditors), which AS 2 allows. Because the individuals performing the CSA evaluation are directly involved in approving and processing voucher packages, they have high competence. The composite impressions of this larger group of directly involved individuals would provide more-objective results than limited inquiries of people who may be motivated to protect their personal judgments and actions. AS 2 (paragraph 117) states that the higher the degree of “competence and objectivity” reflected in the work of others, the greater the auditor may make use of that work. In fact, AS 2 (paragraph 53) specifically groups “self-assessment programs” with the activities of the internal audit function and the audit committee, as controls designed to monitor and evaluate other internal controls.

In addition to evaluating many aspects of the internal control system, external auditors can use CSA to gain a better understanding of a company’s business and industry, to document an understanding of the internal control system, and to assess all types of risks (e.g., control risk, inherent risk).

• Understanding the business and the industry. The CSA facilitator could direct the discussion by marketing, engineering, product development, and production employees toward an in-depth analysis of the industry and specific client operations.
• Understanding the internal control system. The CSA facilitator could elicit information about issues such as the integrity and ethical values of management, management’s commitment to competency, the effectiveness of communications with the board of directors and the audit committee, management’s philosophy and operating style, and human resources policies and practices. Interaction between CSA participants provides insight often not available with traditional tools (e.g., internal control questionnaires).
• Assessing risk. In addition to letting the auditor assess control risk, CSA can also identify the consistency of controls during the period under audit by addressing issues such as unusual events, employee turnover, employee absenteeism, and the quality of training. The CSA facilitator can direct discussions to issues relevant to inherent risk. Employees can evaluate soft issues, including the complexity of transactions, the susceptibility of inventory to theft or damage, the degree to which estimates are used to record accounting information, the extent to which employees must perform tasks without necessary information, and factors that affect the obsolescence of assets. Employee agreement on these issues gives the auditor evidence on which to base inherent risk levels, and thereby to more efficiently and effectively plan substantive testing.

As previously stated, independent auditors and internal auditors face many of the same challenges in using CSA; they can reap similar benefits as well. For example, independent auditors could similarly participate in audit-relevant CSA activities. While the independent auditor must not assume the role of management or employee, in order to protect its independence, it could provide input during CSA planning, serve as the CSA meeting facilitator, attend CSA meetings in a nonfacilitator capacity, or simply use data already developed by CSA activities. Nothing in Generally Accepted Auditing Standards (GAAS), SOA, or the PCAOB auditing standards prohibits, these types of involvements.

CSA Survey Questionnaires

The lack of information about the extent of CSA use during the independent auditing process motivated this research project. Data were gathered via two questionnaires. The first questionnaire was sent to 430 individuals working for U.S. or Canadian organizations that were listed as members in the IIA Control Self-Assessment Center 2001 Membership Directory. Individuals employed by public accounting or professional services firms were excluded, as were multiple members employed by the same organization.

One hundred and thirteen respondents answered questions about the specific uses of CSA at their organization, communications between their organization and their independent auditors about CSA, and their sentiments about auditor involvement in CSA activities. Sixty-seven respondents forwarded an enclosed second questionnaire to their independent auditor. Thirty-one independent auditors responded to questions about how often the audit firm used CSA to accomplish auditing objectives and to specific questions about the prior year’s financial statement audit of the client that forwarded the survey. Tests concluded that results were not materially affected by nonresponse biasing.

Responses from Auditors

Most respondents were evenly split between being audit partners and audit managers, with a few identifying themselves as audit seniors. Twenty-six of the 31 auditors were employed by the (then) Big Five firms.

General use of CSA. The auditors were first asked to indicate the approximate percentage of the independent audits performed out of their office in the previous year that used evidence from client CSA activities to help achieve independent auditing objectives. This question was about the general use of CSA, not use specific to the client organization that forwarded the survey. The results clearly indicate that CPAs were not commonly using CSA to achieve independent auditing objectives:

• Ten respondents (35.7%) indicated that none of their audits involved the use of CSA to achieve auditing objectives.
• Of the 18 respondents who indicated that CSA was used, 13 (72.2%) said that CSA was used in less than half of the audits.
• On average, CSA was used in only 21.6% of the audits.

The remaining questions on the survey pertained to the use of CSA during the financial statement audit of the company that forwarded the survey. Only nine of the 31 respondents used CSA on this audit. This low CSA utilization rate is consistent with the findings pertaining to the overall CSA usage rates, and it sends a pointed message.

Reasons for not using CSA. Exhibit 1 presents the reasons why CSA was not used on the independent audit. The two most common were the belief that doing so was inefficient (54.5%) and the fact that the independent auditors lacked training in its use (50.0%). Whether CSA would be inefficient is a matter of opinion. Lack of training is factual and uncontestable, but can be corrected. The third most common response (40.9%) was “other.” The most commonly cited reasons for not using CSA were that the client was not using CSA much, the client had not developed an adequate CSA program, or the auditors were unaware of how (or if) the client was using CSA. It appears that independent auditors are not taking the initiative to request audit-relevant CSA activities, and management is not communicating with their independent auditors about CSA activities. Logically, this lack of communication, and lack of initiative, is contributing to low CSA utilization levels during independent audits.

Uses of CSA during the audit. Exhibit 2 reveals the attitudes of the nine independent auditors who used CSA during the previous audit of the referring company. A comparison with Exhibit 1 reveals interesting differences of opinion between auditors that did not use CSA and auditors with first-hand experience of CSA. Exhibit 2 shows data about how CSA was used and its perceived value.

A high percentage of the respondents used CSA to understand the company’s business and industry, to document the required understanding of all five components of internal control, and to supplement traditional tests of controls. A majority (55.6%) of these respondents were not using CSA to assess fraud risk, which is surprising because fraud risk assessments typically require auditors to evaluate soft controls (e.g., management’s ethics and integrity). Exhibit 2 also reveals that auditors commonly found CSA either “very” or “somewhat” useful in all areas except substantive testing (few auditors used CSA for this purpose, and their opinions were widely divergent).

Exhibit 3 presents the overall sentiments about the value of CSA. A majority of the auditors “strongly agreed” that CSA resulted in a more efficient and effective audit, the opposite of the expectations of auditors that did not actually use CSA (as shown in Exhibit 1).

Independent auditors’ direct participation in CSA activities. Only a small subset of the nine CPA firms that used CSA were actively involved in those CSA activities. Exhibit 4 shows that only five participated in planning CSA activities, and even fewer were actively involved with their client’s facilitated team meetings.

The absence of auditors’ involvement in their clients’ CSA activities is particularly interesting when related to one finding from the first questionnaire. Respondents from many companies thought that auditor involvement would reduce the value of CSA to their organization. The data in Exhibit 4 suggest that these negative sentiments were based not on direct negative experiences, but rather on preconceptions. It appears that many companies and their auditors are forming their opinions about CSA without firsthand information.

Matching of responses. The authors matched the responses of 31 independent auditors to their clients’ response to obtain further insights into the very low level of CSA utilization by independent auditors. Eight of the 31 companies said that they did not use CSA during the audit period under study, leaving 23 client organizations that did use CSA.

A word of caution about interpreting the meaning of the following matched responses: CSA is a very robust tool, and different parties can use the same CSA-generated data for different purposes. For example, an auditor can use information from CSA activities to evaluate the strength of the control environment, and to determine control risk and fraud risk. The company under audit can use information from the exact same CSA activities for other purposes (e.g., assessing the efficiency and effectiveness of operations).

Underutilization of available evidence. The following three sets of comparisons reveal specific instances where auditors probably underutilized available audit-relevant information generated from their client’s CSA activities:

• Six companies used CSA to assess management ethics (important to the control environment), but their auditors did not use CSA data to gain an understanding of the control environment.
• Four companies used CSA to assess the risk of material fraud within their organization, but their auditors did not use CSA data for their fraud risk assessments.
• Four companies used CSA to evaluate internal controls promoting the reliability of the firm’s financial statements. Three of the four auditors did not use CSA data to assess control risk.

Low CSA utilization by auditors, and the possible effects. Earlier, it was suggested that low CSA use during independent audits was due to auditors’ rarely requesting CSA information from their clients. The following comparisons are very revealing.

• The company had CSA data and the auditor did not request it. Twenty-three of the 31 organizations used CSA in some fashion, whereas only three confirmed that their independent auditors requested CSA data.
• The use of CSA data for independent auditing objectives may be lower than the amount of requests. Three companies said their auditors requested CSA data, but their auditors said that CSA data were not used. This suggests that the actual use of CSA data to achieve independent auditing objectives is lower than the number of auditor requests for client CSA data.
• The lack of auditor initiative and communication may have led to inaccurate perceptions by the independent auditors. Nine auditors said they did not use CSA because the client did not have sufficiently developed CSA activities to be useful, yet in six of these cases, the companies used CSA in areas relevant to the independent audit. Clearly, the independent auditors had inaccurate perceptions about their client’s CSA involvement.

More Communication Needed

Although auditors can use CSA to improve the effectiveness of their internal control evaluations, this study revealed very low CSA utilization levels by independent auditors. Companies and their independent auditors both have a role to play in increasing CSA usage. The data strongly suggest that enhanced auditor training and effective two-way communication between companies and their independent auditors could lead to increased CSA use by independent auditors. Such measures would likely result in more-effective and -efficient audits that would benefit all parties.