Final Report

Audit, Evaluation, and Risk Branch

October 2016

Table of Contents

• Executive Summary
• Introduction
• Focus of the Audit
• Findings, Recommendations and Action Plans
• Program Management
• Monitoring and Reporting
• Conclusion
• Acknowledgement

Executive Summary

Background

Canadians expect that the Canada Revenue Agency (CRA) is well managed, information is safeguarded, and resources are used effectively and efficiently, while ensuring that taxpayers meet their compliance obligations. They also expect integrity and fairness in CRA processes, which promote the consistent treatment of all taxpayers.

The Audit Quality Review (AQR) program is the result of the Domestic Compliance Programs Branch’s (DCPB) effort to further enhance audit quality and internal controls, and foster a culture of integrity. The AQR program, launched spring 2013, is based on seven compliance program control measures developed by the DCPB and the International, Large Business, and Investigations Branch (ILBIB). These control measures were introduced to update and promote rigour in audit activities, enhance the effectiveness of audit processes, and provide assurance of the integrity in which compliance programs are undertaken.

Objective

This audit is the first phase of a two-phased internal audit of the AQR program. The objective of this phase is to provide assurance that the seven compliance program control measures have been integrated into the AQR program and that these control measures are in place and working as intended to support the program’s mandate and achieve its objectives. The second phase of this audit will look at the governance and internal controls in place to support the achievement of the program’s goals and objectives.

Conclusion

Overall the seven control measures are reflected in the AQR program. Any implementation issues may be due to the relative infancy of the AQR program and the challenge of coordinating the support provided to the audit programs. Strengths of the AQR program include the audit file review process and supporting the risk-based audit process. A comprehensive monitoring system will aid in addressing the various roles of the AQR program. The monitoring and feedback mechanisms within the AQR program require clarification of accountabilities and responsibilities at both the regional and national levels.

Action Plan

In response to the conclusions presented, DCPB has agreed with the internal audit recommendations and developed action plans to address the outstanding issues outlined in the audit findings and further strengthen the AQR program.

These action plans include:

• developing a framework for the enhanced monitoring of audit activities which identifies the expectations for AQR;
• documenting roles and responsibilities for addressing AQR results and identified training needs related to the performance of the audit programs;
• documenting the monitoring plan of the National Quality Assurance program; and
• documenting the rationale that supports the coverage rates and business rules employed by AQR.

Introduction

Canadians expect that the Canada Revenue Agency (CRA) is well managed, information is safeguarded, and resources are used effectively and efficiently, while ensuring that taxpayers meet their compliance obligations. They also expect integrity and fairness in our processes, which promote the consistent treatment of all taxpayers.

The CRA has established and maintained broad systems of internal control to help mitigate the risks related to the achievement of program goals. Internal control activities are designed to provide reasonable assurance that particular objectives are achieved as intended and may be categorized by the type or nature of activity. It is important to understand that no one control measure alone can provide adequate assurance; rather it is their combination that provides strength in the internal controls.

The Audit Quality Review (AQR) program is the result of the Domestic Compliance Programs Branch’s (DCPB) goal to further enhance audit quality and internal controls, and foster a culture of integrity. The AQR program is based on seven compliance program control measures developed by the DCPB and the International, Large Business and Investigations Branch (ILBIB). The control measures were introduced to update and promote rigour in audit activities, enhance the effectiveness of audit processes and provide assurance of the integrity in which compliance programs are undertaken.

The AQR program, a real-time audit file review process, was implemented in the spring of 2013. The program’s objective is to support the completion of timely, accurate, and independent audit quality reviews and provide effective feedback regarding audit file quality. The AQR functional responsibilities, located in Headquarters (HQ), are divided between income tax and GST/HST. The income tax component is located in the Quality Section of the Business Intelligence Division in the Small and Medium Enterprise Directorate. The GST/HST component is in the Quality Assurance Section of the Small and Medium Business Audit Division in the GST/HST Directorate, all within the DCPB.

Regionally, where the audit files are reviewed, the AQR program is paired with the Business Intelligence program under a director who reports to the Regional Assistant Commissioner. Each region has a Business Intelligence and Quality Assurance (BIQA) program. The BIQA director in each region oversees the delivery of the AQR program for the following income tax and GST/HST compliance programs:

Small and Medium Enterprises Directorate:

• Office Audit (centralized in the Atlantic Region)
• Small Business Audit
• Medium Business Audit

GST/HST Directorate:

• GST/HST Small and Medium Business Audit
• GST/HST Refund Integrity
• GST/HST Large Business Audit (centralized in the Ontario Region)
• GST/HST Complex Non-resident Audit
• Public Service Bodies
• Aggressive GST/HST Planning (centralized in the Pacific Region)

Description of Audit and Audit Review Processes

In general, work on audit files begins with a risk assessment of the taxpayer’s or GST/HST registrant’s return and filing history by the regional Business Intelligence (BI). If chosen for audit, the file is assigned to the auditors by the BI. The auditor addresses the issues identified by the BI and proposes a reassessment where warranted. The completed audit file is uploaded into the case management system after a review by the team leader.

At this point, the audit file is either closed or randomly selected for an audit quality review. The compliance programs have determined the coverage rates and business rules which define the percentage and nature of the audit files to be reviewed for their specific workload. The AQR officer reviews the quality elements of the file including the planning process, legislative authority, and applicable CRA policies and procedures of the completed audit file. If changes are required, the audit file is returned to the audit area. Once the required changes are made, and approved by AQR, the file is closed. In certain situations audit files selected for review may be closed out immediately and the review by AQR will be deferred; for example, where there are resource constraints.

The AQR program’s audit file reviews, along with the quarterly regional and national results, are part of the compliance programs’ monitoring process. This information is used by the compliance programs to identify opportunities for improvement.

The National Quality Assurance Program reviews regional AQR files to verify consistency in the application of the audit quality standards. These reviews are the responsibility of the audit quality review sections in DCPB. The regional and national AQR results are distributed to the compliance program managers and the Agency senior management.

Figure 1: Audit and Audit Quality Review Processes

Focus of the Audit

This audit is the first phase of a two-phased internal audit of the Audit Quality Review program. The objective of this phase is to provide assurance that the seven compliance program control measures have been integrated into the AQR program and that these control measures are in place and working as intended to support the program’s mandate and achieve its objectives. The second phase of this audit will look at the governance and internal controls in place to support the achievement of the program’s goals and objectives. Phase 2 of the audit has been included in the Audit, Evaluation, and Risk Branch’s (AERB) Risk-Based Audit and Evaluation Plan as part of the 2017-2019 Inventory of Engagements for Consideration.

The seven control measures to be reviewed are:

1. All workload decisions pertaining to workload screening and selection, including risk assessment, are captured electronically.
2. Restricted scope workloads are limited to nationally approved projects.
3. Risk analysis, planning, and execution of case files are documented.
4. There is mandatory central electronic storage of case files.
5. There is mandatory review of files for conformity with legislation, policies and procedures.
6. Procedures for Taxpayer Relief and/or other similar considerations are established and consistent with national guidelines, documented, and authorized.
7. There is program level monitoring of activities.

During the examination phase we reviewed and analysed the AQR results, interviewed HQ and selected regions, and performed a review of AQR audit file review reports.

The audit team reviewed a statistically valid random sample of 121 income tax review reports and 102 GST/HST review reports based on a confidence level of 90% and margin of error of 7%.

The examination phase took place from March 2016 to June 2016.

The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing.

Findings, Recommendations and Action Plans

Integration of the control measures is an important element to the achievement of the AQR program’s goals and objectives. They are evident in the program’s contribution to the effectiveness of the audit process. The degree of integration and implementation can be observed in the program management (control measures 1-6), and monitoring and reporting processes (control measure 7) of the AQR program.

Program Management

The AQR program’s role is to review completed audit files for the various compliance workloads within the DCPB. Internal Audit Directorate (IAD) found that audit quality reviews supported the branch’s approach of auditing to the risks identified and maintained the effectiveness of the audit process. There was, however, a weakness in the guidelines to decide whether files selected for review are to be deferred until after the file has been closed. These findings are discussed below.

Risk-Based Auditing

The DCPB risk-based auditing approach is the selection of audit files based on tax compliance risk as assessed by the screeners in the Business Intelligence program prior to being assigned to the auditor (control measure 1). This process attempts to identify and assign the highest risk files. The issues identified by the screeners are to be examined by the auditors. The assigned audit file is further risk-assessed by the auditor to identify any additional audit issues (control measure 3). This internal audit concluded, based on the file review, that the AQR program supports the branch’s risk-based auditing processes by confirming that all risks identified by the screener and auditor are addressed as part of the audit. This verifies the quality and completeness of the audit file.

Audit Process Effectiveness

An objective of the AQR program is to improve the effectiveness of the audit process. AQR can contribute to the achievement of this objective through the identification of changes to be made in an audit file and communicating the required changes in an effective manner (control measure 5).

The requirement for detailed documentation was seen by some GST/HST and income tax auditors as creating additional work (control measure 1). However, this can be viewed as positive since it improves the quality of the supporting audit file and is a direct result of the introduction of the AQR program.

Based on IAD’s review of AQR files, when audit files required changes, the AQR reviewers provided detailed explanations of the changes to be made to the file. The detailed explanation of the required changes resulted in audit file corrections being made and processed without unnecessary delays. Additionally, internal audit concluded that AQR reviews consistently apply the audit quality standards and elements to the audit files in accordance with Branch and Agency policies and procedures (control measures 5 and 6).

Deferred files

Deferred files are completed audits, selected by AQR, that are not reviewed until after the reassessment (if applicable) has been sent to the taxpayer or registrant and the audit file closed. Files are deferred by AQR team leaders where it is determined that an audit quality review prior to reassessment may add an unreasonable delay to the completion of the audit file (for example, in situations where a region encounters volumes of intake which cannot be handled by the available resources). The procedures in place to decide whether to defer a file focus primarily on the volume of the intake and the available resources. Files are deferred without risk-assessing the potential for errors within the file. This may result in the file being re-opened and reassessed. It is policy to re-open files only for errors that may benefit the taxpayer; therefore, without a cursory review before closing and deferring the files, deferred files that may have resulted in higher reassessment after the audit quality review will remain closed.

Recommendation:

The DCPB should identify risk factors to consider when deciding to defer an AQR file.

Action Plan

The concept of risk assessing files when making deferral decisions was envisaged during the design and initial implementation phases of AQR, but in practice this proved unachievable and was hence abandoned.

Given the low number of deferred files, and more specifically, the even lower number of deferred files with over or under-assessments, DCPB is prepared to accept the minimal risk of having to re-open audit files reviewed on a deferred basis. Over-assessments found on deferred files are corrected and hence present little risk to agency reputation.

Notwithstanding the minimal risk, DCPB will establish guidelines to formalize the current practice of deferring known low risk projects files first before deferring regular audit files.

Responsible area: Quality sections in Small and Medium Enterprise Directorate (SMED) and GST/HST

Completion Date: Quarter (Q) 3 2016

Monitoring and Reporting

The objective of control measure 7 is the establishment of an enhanced monitoring process, as outlined in the document “Compliance Programs Branch Audit Controls”. This control measure is aimed at providing adequate functional oversight on a continual basis for the purpose of ensuring:

• conformity with compliance programs’ objectives in terms of qualitative and quantitative elements;
• a balanced focus between qualitative and quantitative criteria;
• effective key performance indicators are enhanced and tracked on a regular basis; and
• an effective follow-up (feedback) mechanism with those accountable for program delivery.

Though there are some elements of the enhanced monitoring within the AQR process, controls have been identified that require strengthening. These are outlined below.

AQR Roles and Responsibilities

An enhanced monitoring process requires clearly defined, documented, and accepted roles, responsibilities, and accountabilities. These responsibilities aid in achieving the function’s mandate of promoting and monitoring the consistent application of the established quality standards. Compliance programs rely on the AQR program as part of their program monitoring process.

Compliance programs expect that AQR will identify and, where possible, correct deficiencies in the conducting and reporting of audits. AQR is expected to ensure that auditors follow the audit quality standards and report any deficiencies to the regions and the compliance programs in HQ. The compliance programs may identify controls to be strengthened within their processes utilizing the monitoring information provided by AQR. There are no documents that clearly define the accountabilities and responsibilities that confirm the compliance programs’ expectation of AQR’s service to their areas.

There is no protocol or memorandum of understanding that documents the accountabilities and responsibilities at the national or regional level. Assigning accountability and responsibility will clarify roles within DCPB and the regions in addressing AQR results, recommendations, and identified training and learning needs.

Recommendation:

The DCPB should clarify responsibilities and accountabilities for servicing the functional area, as well as addressing AQR results, recommendations, and AQR identified compliance programs’ training and learning needs.

Action Plan

DCPB will document roles and responsibilities for addressing AQR results and identified training needs related to the performance of the audit programs.

Responsible area: Quality sections, Training sections and Audit program sections in SMED and GST/HST. BIQA directors and Regional program directors.

Completion Date: Q1 2017

Documentation

Where the roles and responsibilities of the compliance programs and the AQR program overlap, the absence of documentation may hinder the decision-making process or affect the quality of the information reported by AQR. For example, a gap exists in the documentation of the rationale and impact analysis for the coverage rates and business rules used in determining the AQR-eligible workload for each of the applicable audit programs.

The compliance programs determine the percentage and nature of the audit files that are selected by the system based on the coverage rates and the business rules. For example, it may be determined that T1 audit files which are re-assessed as a consequence of GST/HST audit are not eligible for selection by AQR. The rationale or analysis used in these types of determinations would provide a historical record of how decisions were made and additional perspective on the focus of the compliance program areas.

The DCPB should document the rationale to determine the coverage rate and business rules being employed in the AQR process.

Action Plan

DCPB will document the rationale that supports the coverage rates and business rules employed by AQR.

Responsible area: Quality sections in SMED and GST/HST.

Completion Date: Q3 2016

Monitoring Guidelines

As noted earlier, the AQR sections in DCPB are responsible for the monitoring of the AQR program. The sections monitor the regional AQR activities as well as provide functional support to the regional BIQA organizations. Part of the monitoring process is the National Quality Assurance Program whose mandate is to review AQR files to verify consistency of the application of the audit quality standards in the regions. The National Quality Assurance Program reports on the regional and national AQR results.

IAD acknowledges that these monitoring activities take place within the AQR program; however, there are no comprehensive monitoring guidelines that detail the program’s objectives, methodology, and performance measures and expectations in supporting the various compliance programs.

Similarly, the National Quality Assurance Program requires detailed monitoring guidelines defining the AQR program’s key performance indicators and program’s outcomes and outputs which are to be monitored. Such guidelines would better identify the contribution and expectations of AQR in the monitoring on behalf of compliance programs.

Recommendation:

The DCPB should create comprehensive guidelines detailing the goals and objectives, roles and responsibilities, methodology, reporting and performance expectations of the AQR Program.

Action Plan

DCPB will develop a framework for the enhanced monitoring of audit activities which identifies the expectations for AQR. The framework will be developed in conjunction with the recommendation on the clarification of roles and responsibilities.

Responsible area: Audit sections and Quality sections in SMED and GST/HST.

Completion Date: Q1 2017

Recommendation:

The DCPB should create a comprehensive monitoring program outlining for example, the frequency of monitoring visits, methodology, reporting, and performance expectations for regional AQR activities.

Action Plan

DCPB will document the monitoring plan of the National Quality Assurance program.

Responsible area: Quality sections in SMED and GST/HST.

Completion Date: Q3 2016

Feedback

An effective feedback or follow up mechanism is important for achieving many of the objectives of the seven compliance program control measures. For example, the following control measures’ objectives require an effective feedback mechanism:

• The enhancement and tracking of key performance indicators.
• Addressing of any identified irregularities, or unusual trends through the monitoring program.
• Achieving the objectives of structuring and directing risk assessment activities, prioritization of audit effort to the most pertinent issues.
• Limiting restricted audit files to nationally approved projects for specific purposes.
• Ensuring that the exercise of Taxpayer Relief as it relates to waivers of interest is undertaken in a manner consistent with national guidelines.

Feedback mechanisms are essential for the achievement of many of the control measures’ objectives and the strengthening of the AQR program. It is noted from interviews with the regions that the AQR program has had an informal process for feedback that has addressed various concerns and led to changes in the AQR policies and processes.

The effectiveness of the feedback mechanism is dependent on the accuracy and completeness of the data being collected and analyzed. Internal audit found that there are controls in place to support the accuracy, and completeness of the AQR data (control measure 4).

Feedback from Disputed Files

A potential avenue for feedback is the information from the files which have been reviewed by AQR but the changes required are disputed by the auditor and the audit team leader. Locally and at the regional level, this information could be used to improve audit quality by identifying similarities in disputed files and providing a source of reference for future interpretations of branch policy or procedures. The discussions and decisions made as part of the dispute resolution process are not consistently documented as required by the policies and procedures.

Recommendation:

The DCPB should document and track disputed file resolutions, and create a feedback loop to aid in strengthening the AQR program.

Action Plan

DCPB will ensure the regional BIQA directors record and track disputed files consistently between regions, as well as strengthen HQ monitoring.

Responsible area: Quality sections in SMED and GST/HST, BIQA directors in all regions

Completion Date: Q3 2016

Regional Compliance Projects

Regional Compliance Projects provide guidance that may be beneficial to decisions made at the regional and national program levels. These projects are undertaken in each region to address non-compliance issues in sectors that may be specific to that region. An objective of control measure 2 is to limit restricted audit files to nationally approved projects for specific purposes. Restricted audit files are usually limited to addressing one specific issue from the outset; therefore a full risk assessment or screening process may not take place. The audit or review may not have a full planning phase; therefore, not all material areas would be identified or addressed.

In each region the BIQA director is responsible for approving regional compliance projects, with limited involvement from HQ in regards to the AQR program. Currently, there is no policy or guideline for the inclusion or exclusion of the AQR program in new regional or national compliance projects. Without a policy or guideline, it is possible that the AQR functional areas may not be aware of ongoing regional projects, resulting in their omission in the AQR program workload.

Recommendation:

The DCPB should include all national and regional projects in the AQR process, with criteria developed to aid in determining when projects are to be excluded from the AQR process.

Action Plan

DCPB has developed guidelines for the reporting and approval of projects which requires prior consultation with AQR. Guidelines will be issued shortly to clarify and delineate the circumstances whereby projects can be excluded from AQR.

Responsible area: Quality sections and BI sections in SMED and GST/HST.

Completion Date: Q3 2016

Conclusion

The focus of this internal audit was to provide assurance that the seven compliance program control measures have been integrated into the AQR program and these control measures are in place and working as intended to support the program’s mandate and achieve its objectives. Overall the seven control measures are reflected in the AQR program. Any implementation issues may be due to the relative infancy of the AQR program and the challenge of coordinating the support provided to the audit programs. Strengths of the AQR program include the audit file review process and supporting the risk-based audit process. A comprehensive monitoring system will aid in addressing the various roles of the AQR program. The monitoring and feedback mechanisms within the AQR program require clarification of accountabilities and responsibilities at both the regional and national levels.

Acknowledgement

In closing, we would like to acknowledge, recognize and thank the Quality Section and the GST/HST Quality Assurance Section in DCPB, as well as the BIQA organizations within the Atlantic, Quebec, Ontario, Prairie, and Pacific Regions, for the time dedicated and the information provided during the course of this engagement.