DHS Inspector General Withdraws 13 Reports Not Compliant With Federal Auditing Standards

By:
Chris Gaetano
Published Date:
Jun 7, 2019
FEMA-GettyImages-483743566

The Inspector General of the Department of Homeland Security, following a review of its own director, withdrew 13 reports that it concluded were not conducted in accordance with federal auditing standards, characterizing them instead as "feel good" reports intended to flatter DHS staff, according to the Washington Post. The Emergency Management Oversight Team (EMOT) reports generally concerned the Federal Emergency Management Agency's (FEMA) performance in the wake of natural disasters such as hurricanes or tornadoes. The review was promoted by an EMOT report from the inspector general's office about recent flooding in Louisiana that praised FEMA as "remarkable," "resilient and mission focused," and "[providing] hope and a way forward," despite 13 deaths as a result of the disaster. The House Committee on Oversight and Government Reform (HOGR) raised concerns about the accuracy of the report’s findings and conclusions, prompting the report's removal. After conducting a preliminary quality assurance review of similar EMOT reports, the DHS OIG retracted and removed from its website an additional 12 reports in March 2018 due to similar concerns. 

The IG's special report cited several factors that led to this state of affairs. 

First, following a leadership change in 2011, it was determined that EMOT deployments would always result in the issuance of a report to FEMA, which previously had not been the case.

Second, while Acting Inspector General John V. Kelly determined that these reports would be compliant with Yellow Book auditing standards, teams were not immediately aware of this decision when they were deployed, and so did not know they were expected to conduct the work according to those requirements. Further, the reports were not later revised to comply with the standards.

Third, the goal of EMOTs shifted from promoting effectiveness and efficiency to evaluating whether initial disaster response was effective and efficient. 

"Notwithstanding this change in the objective, DHS OIG did not identify criteria that would have allowed it to adequately evaluate FEMA’s effectiveness and efficiency," said the report. 

Finally, the report said that the attitudes and actions of leadership impaired the objectivity of staff. Kelly, as well as another senior employee, were particularly impressed by FEMA's dedication and work ethic, which was expressed repeatedly to staff, creating a "preconceived notion" among the staff that whatever FEMA did was probably effective and efficient. 

Beyond these attitudes, the report also pointed out a change in how the reports were drafted. Kelly instructed staff, in the event they identified significant systemic issues, to carve their findings out of the report so it could be discussed in further depth in a separate "spinoff" report. Asked why he instructed staff to do so, Kelly said that it was better to issue a "clean report" for EMOTs because the messaging would be clearer and less likely to get lost or ignored by FEMA. "However," said the special report, "it does not appear that he explained his rationale to EMO staff, who did not understand why they were being instructed not to include some of the more significant findings in their EMOT reports." 

These sorts of changes also affected an attitude shift in the inspector general's office about the nature of EMOT reports. The review said they came to be commonly understood throughout the division as “feel good” reports. Their common elements included the following: (1) the overall conclusion of an EMOT report would be that FEMA’s disaster response was “effective and efficient”; (2) the report would not include recommendations; (3) if the report identified challenges, it would note how FEMA overcame them; and (4) other negative information, if systemic, would be removed from the report and placed in a separate report. 

This was, in fact, explicitly stated as the intention by a senior employee in an email on which Kelly had been copied. 

"These are feel-good reports to tell the public how well FEMA initially responds to disasters. Yes, we will identify some problems, but the point is – how did FEMA overcome them? ... EMOT deployments present wonderful opportunities to identify systemic problems, but the EMOT report is not the place to discuss the problems in detail. These opportunities allow us to write quick, meaningful reports that actually help FEMA, and FEMA usually loves the reports,” said the email. 

Kelly, said the special report, did not respond to the email to clarify or correct any of the statements in it. 

This was not a one-off communication. The report cites another email from the same senior employee saying that Kelly did not want serious issues discussed in the EMOT report, and once again used the phrase "feel-good report" to characterize what was expected. When the staff member pushed back against this, the senior employee said that the report's priority is to "commend FEMA on a job well done" and "identify various problems and how FEMA overcame those challenges (which also establishes opportunities for future reports). There will be no recommendations.” 

On the Louisiana flooding report that prompted the inquiry, the report said that the EMOT director made material changes to the draft without ensuring that they were supported by sufficient and appropriate evidence. This included actions such as adding superlative language like “aggressively,” “remarkable,” and “creatively” to describe FEMA’s disaster response efforts, and changing the overall conclusion of the report from “generally efficient and well organized” to “effective.” He also added language discounting complaints about FEMA’s disaster response. None of the DHS OIG senior leaders who reviewed and approved this report corrected these issues. 

When the report was published, HOGR staff members, who also happened to be in Louisiana to observe FEMA’s response to the disaster, could not reconcile the report’s conclusion about the effectiveness of FEMA’s response with their own observations. Upon learning of their concerns, DHS OIG conducted a quality assurance review of the EMOT report in question and determined that it did not comply with Yellow Book auditing standards. The report was retracted and withdrawn from DHS OIG’s website on July 19, 2017. 

The inspector general's special report said that much of the problem could be placed on poor internal controls. It said that the EMO's management and supervision were deficient in ensuring adequate planning that would have established achievable audit objectives. EMOTs generally stated that their objective was to evaluate the effectiveness of FEMA’s response to the disaster, but there were no defined criteria or steps for impartially conducting this assessment. 

The special report said management and supervision were also deficient in ensuring that teams adequately assessed the evidence obtained during fieldwork. For example, not only did auditors not evaluate FEMA's own internal control systems, it almost exclusively relied on testimonial evidence (most of which was provided by FEMA personnel) without taking steps to independently confirm assertions made and to protect against bias. 

It also faulted EMO's internal quality control procedures due to weaknesses in its Independent Reference Reviewer (IRR) process, where employees not part of the audit team are meant to confirm that a draft report's statements are accurate and its findings supported by evidence. The process failed to flag deficiencies, instead being more focused on confirming basic facts and figures, rather than on ensuring that supporting conclusions and auditor opinions were adequately supported. 

Beyond this, the report also said that the Inspector General's Supervisory Review Checklist, through which leadership certifies that they reviewed the audit work and that it met Yellow Book standards, was not used properly, and failed to flag deficiencies. For example, in the Louisiana flooding report, the checklist indicated that the team did indeed verify the validity and reliability of FEMA's data while, in fact, the team only verified the data from the FEMA data system without checking whether the data contained in the system was valid and reliable. 

The report recommended the following steps: 

1. Confirm that all DHS OIG reports bearing the four signature traits of a “feel good” report have been retracted and removed from DHS OIG’s website. 
2. Ensure that all future early deployment work conducted by DHS OIG, if any, is performed in accordance with applicable standards. 
3. Design and implement improvements to DHS OIG’s quality assurance processes to resolve the internal controls deficiencies identified by the internal review. 
4. Close all open recommendations outlined in the EPA OIG peer review. 
5. Refer this report to the Council of Inspectors General on Integrity and Efficiency for whatever action it deems appropriate

Leadership at the Office of the Inspector General concurred with the recommendations. 

Click here to see more of the latest news from the NYSSCPA.