Final report

To print a copy of this report, use the printable version (PDF, 3587 KB).

Audit, Evaluation, and Risk Branch

September 2014

Table of Contents

• Executive Summary
• 1. Purpose
• 2. Background
• 3. Evaluation scope and methodology
  • 3.1 Scope
  • 3.2 Methodologies
  • 3.3 Limitations and clarifications
• 4. Findings
  • 4.1 Is the EAC sent to Canadians by the CRA understandable?
  • 4.2 How well is the CRA administering EAC?
• 5. Conclusion
• 6. Recommendations
• 7. Management response
• 8. Acknowledgements
  • Appendix A: EAC Reviewed by Siegelvision and TNS
  • Appendix B: Larger Versions of Figure 1
  • Appendix C: Larger Versions of EAC Found at Figure 2
  • Appendix D: Larger Versions of IRS EAC Found at Figure 3

Executive Summary

Purpose

The evaluation of external administrative correspondence (EAC) was approved by the Management Audit and Evaluation Committee in October 2012 to explore the following two issues:

• Is the EAC sent to Canadians by the Canada Revenue Agency (CRA) understandable?
• How well is the CRA administering EAC?

The Program Evaluation Division of the Audit, Evaluation, and Risk Branch used a series of methodologies to answer these issues including interviews with staff, subject matter experts, and stakeholder groups; surveys; data analysis; literature and document reviews; consulting foreign and domestic tax administrations; and engaging third party research contractors.

Background

EAC can be described as personalized correspondence that is intended for a specific recipient or their authorized representative. The CRA sends out approximately 129 million pieces of EAC each year.

Public Affairs Branch (PAB) is responsible for developing and implementing CRA policies, guidelines, standards, training, and tools related to communications.

Findings

Messaging, structure, tone, and branding of EAC could be improved.

Other tax administrations have recently made significant changes to the way they administer EAC.

PAB has made efforts to improve the administration of EAC, however, internal processes and systems create challenges to the effective administration of EAC.

Recommendations

We recommend that CRA senior management:

1. Consider enterprise-wide options in moving forward to address the shortcomings of EAC.
2. Consider including EAC in the CRA Corporate Business Plan and Service Strategy.
3. Seek opportunities to position EAC to take advantage of technology (electronic forms of communication) and emerging strategies e.g. behavioural economics.

Management Response

A plan is being developed to address the issues raised in the evaluation report.

The plan includes the following:

1. The CRA will seek the views of businesses, tax intermediaries, and their associations on how the Agency can improve its notices and letters, through its Red Tape Reduction consultations scheduled for the fall.
2. A further engagement opportunity will be launched in the new year to seek the views of additional stakeholders, including charities and benefit recipients.
3. A summary of the results of these consultations will be made public in 2015, and will form the basis of an action plan, with a particular focus on delivering more correspondence online and in a manner that is clear and can be tailored so that the information needs of recipients, whether simple or more complex, are met.
4. The CRA anticipates that, beginning in February, 2015 and continuing through to early 2016, its top volume letters and notices will be available online to Canadians in simplified, easier-to-understand formats.
5. The CRA will also review the best practices of other tax administrations, and engage a third party consultant to provide expert advice on how to simplify language on complex or technical topics, visually present information in a way that encourages understanding and action where required, and use emerging technologies effectively.

1. Purpose

The External Administrative Correspondence (EAC) Steering Committee, chaired by the Public Affairs Branch (PAB) from 2007-2012, requested an evaluation of the personalized correspondence sent by the Canada Revenue Agency (CRA) to external recipients. The Management Audit and Evaluation Committee (MAEC) approved the EAC evaluation framework in October 2012.

The purpose of the evaluation was to examine two issues:

• Is the EAC sent to Canadians by the CRA understandable?
• How well is the CRA administering EAC?

This document provides the findings and recommendations for this evaluation.

2. Background

2.1 What is EAC?

EAC is defined as personalized correspondence generated by the CRA that is intended for a specific recipient or their authorized representative. It can be classified as either a notice or a letter but does not include returns or forms. ^{Footnote 1}

EAC produced by the CRA varies considerably in terms of content, complexity, and the quantity generated. In 2011-2012, the CRA sent out just under 129 million pieces of EAC, comprised of approximately 430 specific letter or notice types (for example, individual income tax notice of assessment, Canada child tax benefit notice of redetermination, etc.). Roughly 8% (n=35) of the different types of EAC is responsible for 90% of the total volume. The Assessment and Benefit Services Branch is responsible for generating approximately 331 (77%) of these different types of EAC. Table 1 provides a volume breakdown of the types and amount of EAC generated by branch.

Table 1: Types and Volume of EAC by Branch 2011-2012

Branch	Different Types of EAC	Volume of EAC	% of Total Volume
Assessment and Benefit Services (ABSB)	331	118,344,621	91.93%
Taxpayer Services and Debt Management (TSDMB)	23	9,072,819	7.05%
Compliance Programs (CPB)	35	693,037	0.54%
Appeals (AB)	4	313,709	0.24%
Legislative Policy and Regulatory Affairs (LPRAB)	39	310,927	0.24%
Total	432	128,735,113	100.00%

Source: CRA program branches

2.2 Why is EAC important?

The CRA Taxpayer Bill of Rights states that "all Canadians have the right to complete, accurate, clear, and timely information". The Bill of Rights also commits the CRA to minimizing the costs of compliance for small businesses. This includes minimizing the time, effort, and costs that small businesses incur by making interactions with the CRA as straightforward and convenient as possible.

The CRA Public Affairs and Communications Policy requires that communications, including EAC, be effective, well-coordinated, and responsive to the needs of the public. It also requires that information provided to the public must be in plain language (more specifically, information must be clear, well-organized, and easy to understand and use).

EAC is one of the ways the CRA ensures that taxpayers ^{Footnote 2} have the information they need to comply with Canada's tax laws and access the benefits available to them. EAC is sent predominantly for one of two purposes:

• to elicit action (file, pay, register, provide missing information, etc.); or
• to provide the recipient with information (acknowledgement of receipt, confirmation of information, etc.).

Table 2 provides a breakdown of EAC by purpose (action and information) both at the Agency and branch level. Note that overall approximately 36% of EAC required action on the part of the recipient and 64% provided information.

Table 2: 2011-2012 Action and Information Correspondence

	ABSB	TSDMB	CPB	AB	LPRAB	Agency
Action	37,621,577	7,542,276	412,806	108,006	86,951	45,771,616
Information	80,723,044	1,530,543	280,231	205,703	223,976	82,963,497

Source: CRA program branches

2.3 Who is responsible for EAC?

There is no one area within the CRA that has exclusive accountability or responsibility for EAC.

Branches and regions, as subject matter experts in administering and delivering CRA programs, are responsible for the content of and day to day activities related to generating EAC. Employees are responsible for ensuring that the EAC they generate is in plain language or, more specifically, clear, well-organized, easy to understand, and easy to use. ^{Footnote 3}

Most operational programs generate EAC. Some program activities, such as the processing of returns, may result in only one piece of EAC a year (for example, an individual income tax notice of assessment). Other program activities, such as conducting reviews, audits and examinations, require more interaction and may result in multiple pieces of EAC.

PAB is responsible for developing and implementing CRA policies, guidelines, standards, training, and tools related to communications. ^{Footnote 4}

PAB is also responsible for assisting program areas to ensure CRA communications products are of the highest quality; and supporting employees with tools and resources to enhance their writing. PAB chaired the EAC Steering Committee until it was dissolved in April 2012. It has since been replaced by the Plain Language Implementation Committee.

2.4 How is EAC generated and sent to recipients?

EAC is generated in three different ways:

• Automated EAC is created, based on system algorithms, by CRA information technology systems that are used to register businesses, assess or process returns, or perform accounting, collections, and other compliance functions.
• Employee-customized EAC is predominantly created by selecting standardized verses or templates from system applications, such as the Electronic Letter Creation System, that can be manually modified by employees.
• Employee-authored EAC is typically free-form letters created using word processing software without the aid of templates or standardized verses.

The vast majority of EAC generated by the CRA is automated (96% or 123.5 million), followed by just over 5 million (almost 4%) that is employee-customized and just over 0.1% that is employee-authored (177,313). Table 3 provides a breakdown of action and information EAC by how it is generated.

Table 3: Action and Information EAC by Generation Method (2011-2012)

	Action Volume	Action %	Information Volume	Information %	Total Volume	Total %
Automated	41,847,570	91.43%	81,606,220	98.36%	123,453,790	95.90%
Employee-customized	3,790,071	8.28%	1,313,939	1.58%	5,104,010	3.96%
Employee-authored	133,975	0.29%	43,338	0.05%	177,313	0.14%
Total	45,771,616	35.55%	82,963,497	64.45%	128,735,113	100.00%

Source: CRA program branches

Most EAC is sent to external recipients through regular mail. A very small volume is sent by registered mail or electronically. About 1% of the volume generated by ABSB is in electronic form (mainly pdf) available to subscribers of My Account and My Business Account.^{Footnote 5}

3. Evaluation scope and methodology

3.1 Scope

The two issues the evaluation focused on were:

1. Is the EAC sent to Canadians by the CRA understandable?

The elements involved in determining understandability can be difficult to isolate and in fact may include elements that on their own may not be synonymous with understanding but assist in optimizing understanding. Examples include: use of white space, organization of thoughts, use of headings, bold text, italics, etc.

Plain language, clarity, readability, and understandability as it pertains to written correspondence are dependent on the following basic input elements ^{Footnote 6}:

• Content: the selection of information to be communicated
• Structure: how the information is organised, sequenced and linked
• Language: how the information is expressed in words including tone
• Design: the typography, layout and graphic design of the document

Readability must also consider the reader's response to the document:

• Their attitude and emotional response
• What they know from reading it
• What they are able to do as a result

All the above stated elements have gone into our assessment of understandability of EAC.

2. How well is the CRA administering EAC?

The focus was on activities related to administering EAC in the CRA (e.g., governance and accountability provisions, roles and responsibilities, policies, procedures, processes). This included an examination of the work completed by the EAC Steering Committee and Plain Language Implementation Committee and how the CRA is organized to deliver EAC.

3.2 Methodologies

The following methodologies were used to examine the evaluation issues:

• Volumetric data from all program areas responsible for EAC was requested, collected, and analyzed.
• A review of files and documentation associated with the management and administration of EAC in the CRA was conducted.
• A total of 67 interviews (many of which were group interviews) with approximately 400 interviewees representing headquarters and the regions were conducted.
• A review of literature related to literacy and financial literacy, plain language and written communications was conducted.
• A total of 10 interviews were held with stakeholder groups and subject matter experts in literacy and plain language.
• We attended two conferences, one on plain language and the other on literacy, and had the opportunity to speak to and learn from subject matter experts.
• We contracted with a third party to independently assess the clarity ^{Footnote 7} of 23 unique pieces of EAC (representing 42% of EAC volumes).
• We contracted with a third party to conduct an online survey of 4 different target populations: individuals, businesses, benefit recipients, and charities to assess the clarity, comprehension, and ease of understanding of 13 pieces of EAC.
• An online survey was sent to 15,586 employees responsible for generating EAC in headquarters and the regions (6,274 respondents).
• An online survey was distributed to all CRA telephone call centre agents across the country (327 respondents).
• We met with representatives of the Office of Taxpayer Correspondence, United States Internal Revenue Service (IRS), and conducted teleconferences with representatives of the Australia Tax Office (ATO), United Kingdom Her Majesty's Revenue and Customs (HMRC), and New Zealand Inland Revenue Department (IRD).
• Information was exchanged with Revenu Québec.

3.3 Limitations and clarifications

The following limitations and clarifications should be considered when reviewing this report:

• An external survey using online panels^{Footnote 8} was carried out by TNS Canada for this evaluation. Although online panels are the preferred method of the Government of Canada to survey Canadians endorsed by Public Works and Government Services Canada, they are not ideal for all circumstances. Research suggests that they cannot be construed as representative and they are known to be less accurate than probability samples using random digital dialing telephone surveys.^{Footnote 9}
• As indicated in the evaluation framework, the sheer volume and various types of EAC generated by the CRA prevented an in-depth examination of some aspects of EAC (for example, quality review and assurance processes).
• The evaluation team was dependent on many different program areas to provide volumetric data on the EAC they generate. This proved to be a difficult exercise and it was not possible to test the reliability or integrity of the data. However, the total volume of EAC provided by the branches was assessed against the total volumes identified from online sources with a variance of just over 0.2%. We acknowledge the intrusive nature of our requests and the efforts of the implicated branches to obtain this data, in particular, the Assessment and Benefit Services Branch.
• Accuracy and timeliness of EAC were not included in the scope of the evaluation.
• Lastly, no single source of data is sufficient to support conclusions or findings unless the data is quantitative and proven irrefutable. All findings and conclusions in this report are supported by multiple sources of data.

4. Findings

4.1 Is the EAC sent to Canadians by the CRA understandable?

As indicated earlier, understandability of EAC must factor in a number of elements – those that are within and outside the CRA's control.

4.1.1 There are external factors that impact understanding

Regardless of age, education, or cultural background, everyone needs information from the CRA that they can read and clearly understand. For EAC to be effective it must be easily understood so that the reader can, if necessary, take the appropriate action. Ideally, it should not place an unnecessary burden on the reader or the author. Research and subject matter experts in plain language and literacy interviewed for this evaluation emphasized that for EAC to be most effective, it first has to meet the needs of the reader (as opposed to the author).

Research^{Footnote 10} has shown that people:

• Read with different degrees of literacy.
• Read in a hurry or without full attention.
• Read in poor reading conditions.
• May not know (or read) the language well.
• May have a cognitive or learning disability.
• May have a visual disability that can affect reading.
• May have a physical disability that affects interaction.

Research also suggests that reading ability is flexible. It shifts depending on task, context, and motivation.^{Footnote 11} The CRA relies on Canadians voluntarily participating in the tax and benefit system for it to be successful. The time and effort required to comply with tax obligations and apply and continue to receive benefits is often viewed as an inconvenience (if not a burden). In 2011, the Red Tape Reduction Commission (RTRC) identified over 1,100 irritants directed at the CRA^{Footnote 12} many of which were related to access to readily available and clear information.

Aside from time and effort, the results of the Organization for Economic Cooperation and Development adult skills survey^{Footnote 13} found that many Canadians are not performing well in literacy and numeracy - skills necessary to read and understand EAC.

This survey found that almost one out of every two Canadians (49%) between 16 and 65 have literacy rates below the minimum level necessary to function well in society. These Canadians would find it challenging to understand the content of a newspaper article or the essence of this paragraph. Similarly, 55% of Canadians have levels of numeracy proficiency below a level that allows them to function well in Canadian society. Understanding EAC with more than one mathematical computation would cause confusion for these individuals.

Immigrants make up almost 21% of the total Canadian population and the majority (almost 73%) do not have English or French as their mother tongue, and 6.5% speak neither English nor French.^{Footnote 14} An estimated 65% of Canada's immigrant population has low literacy skills in English or French. More generally, 60% of recent and established immigrants, compared to 37% of the Canadian-born population, were below the average in prose literacy.^{Footnote 15}

Conclusion:

To meet the needs of all Canadians and specifically the large population of Canadians with literacy challenges it is incumbent on the CRA to ensure its correspondence is clear and understandable.

4.1.2 The results of third party assessments of the understandability of EAC were mixed but clearly indicate that improvements can be made

We contracted two third parties to independently assess whether the EAC sent to Canadians is understandable. Siegelvision, a company that specializes in the clarity of communications, was hired to undertake an assessment of 23 distinct English and corresponding French letters selected from a variety of program areas within the CRA. Siegelvision was uniquely qualified to conduct this assessment based on their internationally renowned expertise in the field of clear communications. Siegelvision's team worked with the United States IRS within the last 5 years and did work for Revenu Quebec in the 1980's to assist them in improving their EAC.

The EAC reviewed by Siegelvision ranged from 1 to 14 pages in length and represented a cross-section of the correspondence the CRA sends to businesses, individuals, benefit recipients, and charities every year. Most were selected based on their high volumes whereas others were selected because they were considered complex or represented a distinct workload. The 23 types of correspondence represented 42% of the overall volume of EAC sent in 2011-2012.

Siegelvision assessed each piece of EAC and rated it against 11 different criteria. These criteria are widely recognized as the industry standards by subject matter experts in a variety of fields related to communications^{Footnote 16}, including those in the CRA.

Additionally, 7 pieces of EAC (6 English and 1 French) were subject to a detailed critique in which Siegelvision highlighted specific issues that they believed contributed to confusion and undermined readability and effectiveness. These pieces made up 31% of the volume of EAC and contain issues common to all 23 letter types tested.

Overall, Siegelvision concluded that while the EAC had certain strengths such as being relatively short in length, using limited legal disclosures and fine print, and often using personal pronouns, their main criticisms were that the:

• information was not well organized;
• presentation of information did not inspire confidence; and
• tone used lacked empathy.

Through a contract with TNS Canada Ltd., 13 of the 23 pieces of EAC^{Footnote 17} reviewed by Siegelvision were also reviewed by 4 target populations - individuals, businesses, benefits recipients, and charities through the use of online panels.

TNS is an international marketing and social research company that specializes in the use of Highlighter^{Footnote 18} technology. Using Highlighter and a series of closed-ended questions, participants assessed the clarity, comprehension, and ease of understanding of CRA EAC. Refer to Appendix A for a list of the EAC reviewed by Siegelvision and TNS.

Results of the TNS Canada survey are more difficult to interpret as they are not generalizable to the Canadian population as a whole and exclude Canadians having low computer proficiency and/or literacy rates. Therefore, while respondent results may appear more favourable in terms of understandability and other measures, we caution that these results represent very optimal circumstances.

We have used the results from Siegelvision and TNS as well as other available research to explore the results in further detail.

Information could be better organized to optimize understanding

Siegelvision found that the majority of the EAC examined was not organized in a way that allowed the reader to quickly and easily identify its main purpose or important pieces of information such as dates, amounts, phone numbers, next steps, etc. Often the main purpose of the documents was not readily apparent and other important information was scattered throughout the document or embedded in dense paragraphs. When looking at the EAC as a whole, Siegelvision found that there was no underlying grid to ensure that EAC is written and designed in a consistent manner and common elements are reused.

To illustrate these points see Figure 1, the Canada child tax benefit (CCTB) notice of redetermination (larger versions can be found in Appendix B). Note that all of the first page and most of the second is devoted to the calculations the CRA uses to arrive at an amount owing. The reader must work their way through this data, which the CRA uses to validate its request, until the bottom of the second page where the request for payment is made.

Figure 1: Canada Child Tax Benefit Notice of Redetermination

Source: Assessment of external correspondence for the Canada Revenue Agency (Siegelvision)

Explanation of points

1. Top-most message is "keep for your records", but urgent issues are relegated to bottom page 2 (you owe money; CCTB entitlement will be reduced by 50%).
2. Four calculations across two pages (vertical/horizontal/indented) make it difficult to trace relationship of original benefit, subsequent credits, payments, and current amount owed.
3. Reference "Information used to calculate" is disconnected from resulting "entitlement" amount shown and the relationship to CCTB entitlement is unstated.
4. Same amount ($186.74) described two ways: "Amount direct deposited" (page 1) and "payment issued" (page 2).
5. Tone is authoritarian and overly formal (we will reduce … until the amount owing is paid in full).

TNS surveyed an online panel of benefits recipients^{Footnote 19}(n=225 English and n=115 French) about the CCTB notice of redetermination and findings similar to those identified by Siegelvision emerged. When asked specifically what the EAC requires them to do, 53% of English respondents and 48% of French respondents answered correctly "to make a payment". The remaining English and French respondents answered incorrectly or did not know.

When asked the extent to which they agreed with a list of statements about the CCTB notice, survey participants responded as shown in Table 4. Interestingly, respondents gave themselves greater credit for being able to determine the main message (58% English and 55% French) than they were actually able to do (see above paragraph).

Table 4: Evaluation of the CCTB redetermination notice in English and French

	The document is confusing	Organized correspondence	Able to immediately determine main purpose	Appropriate amount of information	Clear words, terms & language
English version (n= 225)	27%	57%	58%	65%	65%
French version (n= 115)	31%	50%	55%	62%	50%

Source: CRA Taxpayer Communications Clarity Testing (TNS)

Table 5 shows that of the 13 English pieces of EAC reviewed by TNS panels, 6 (46%) had self-assessed ratings of being able to immediately determine the main message ranging from 80% to 91% and confusion rates from 8% to 18%.

Table 5: Evaluation of notices and letters

	I was able to immediately determine the main purpose of the document.	The correspondence was organized or laid out in a way that helped me understand why I needed to take action.	I found the document confusing.
Individual-Soft collection letter	91%	85%	9%
Reminder to charities to file a return (TX11D)	88%	82%	10%
Notice of intention to revoke a charity's registration	87%	81%	18%
Individual - Request to file a return	86%	82%	9%
Benefit recipients - Missing information	85%	81%	11%
Benefit recipients - GSTC marital status validation	80%	83%	8%

Source: CRA Taxpayer Communications Clarity Testing (TNS)

The remaining 7 pieces of EAC (54%) had higher scores on confusion (13% to 37%) and generally lower self-assessed ratings of being able to immediately determine the main message (56% to 77%). These 7 pieces of correspondence represent 39% of all action oriented EAC (see Table 6).

Table 6: Evaluation of notices and letters with lower understandability results

	I was able to immediately determine the main purpose of the document	The correspondence was organized or laid out in a way that helped me understand why I needed to take action.	I found the document confusing
Businesses - Request to file a return	77%	66%	21%
Businesses - CPP/ROE	73%	68%	13%
Businesses - Corporation NOA	73%	63%	22%
Individuals - Validation Letter	70%	66%	37%
Individuals - Income tax and Benefit NOA	68%	58%	31%
Benefit recipients - CCTB redetermination notice	58%	57%	27%
Businesses - SOA for current source deductions	56%	56%	25%

Source: CRA Taxpayer Communications Clarity Testing (TNS)

When asked to recall whether they found EAC they received from the CRA easy to understand 68% French and 75% English Charity representatives responded affirmatively whereas approximately 57% of business representatives agreed. See Table 7 for more details.

Table 7: Percentage of respondents within recipient groups that found CRA correspondence easy to understand

	Charities	Benefit Recipients	Individuals	Businesses
English	75%	67%	61%	58%
French	68%	71%	64%	56%

Source: CRA Taxpayer Communications Clarity Testing - TNS survey results

Research conducted for the CRA^{Footnote 20} by the Walker Consulting Group also identified issues with EAC:

"Prevailing sentiment about this channel of communication (written communication) tended to be more negative than positive:

• Participants indicated that they often do not understand much of the information expressed in the letters they receive. Many indicated a degree of frustration about not understanding various pieces of information contained in letters from the CRA.
• Many felt that letters from the CRA often do not fully explain the reasons why the CRA had come to certain conclusions impacting their taxes payable and/or tax credits."

Presentation of information may not inspire confidence

As previously noted, within the CRA, communications is a shared responsibility. This includes ensuring high quality EAC that will not only communicate information, but will help to reinforce the CRA's brand, and build and maintain public trust and confidence in the CRA's integrity, impartiality, and commitment to service.^{Footnote 21}

One of Siegelvision's main findings of the EAC they reviewed was that its presentation did not inspire confidence. More specifically, they felt the EAC did not meet best practices in today's marketplace utilized by organizations in other industries serving the same recipients (banking, insurance, cable, utilities). The EAC didn't use standard formatting conventions to organize and highlight important information. For example, the EAC did not make use of different font sizes, bold, headings, subheadings, column titles, or colour to allow the reader to quickly and easily find or navigate through the document.

Siegelvision also noted inconsistencies across notices and different formats within notices that suggest impersonal, antiquated output processes. They indicated that the EAC uses an outdated practice where the first page contains information specific to the purpose of the EAC yet the back of the same page contains information of a general nature (how to contact the CRA, how to make a payment, etc.) then on the third page, marked as page 2 of 2, the remainder of the message is continued. Without following the best practice of true duplex printing, the reader must skip over boilerplate content to finish reading important information about their situation.

Siegelvision also expressed concern that the lack of unifying CRA brand elements may suggest that the EAC, which may request payment or confidential information, are fraudulent and could impede action being taken on the part of the recipient.

Even the majority of employees interviewed agreed that the EAC generated by the CRA, especially when compared with EAC produced by the private sector, does not look professional. They commented that many of the information technology systems used by the CRA are legacy systems and that this is reflected in the EAC they generate.

The examples of EAC in Figure 2 show a range of formats, style and branding elements (larger versions can be found at Appendix C). They make up 26% of the total volume of EAC sent by the CRA in 2011-2012.

Figure 2: Examples of the different EAC generated by the CRA

Source: CRA program branches

The tone of EAC could be more aligned to the circumstances

In their report Siegelvision felt that the tone of the EAC was bureaucratic and one-sided and has a punitive, demanding nature to it. They felt that the language used does not convey a partnership between the CRA and the recipient.

Corroboration of Siegelvision's comments on tone can be found in other research conducted for the CRA^{Footnote 22}:

"There was a broadly held belief that the tone of the language in letters from the CRA is often unduly severe, even when the information does not relate to an audit or like issue. In a few groups, participants juxtaposed the friendly, helpful nature of the telephone service to the less friendly and less helpful tone of the CRA EAC, suggesting a notable and puzzling difference in tone and approach."

"As was found in the general public groups, SME focus group participants found that they often felt uncertain about the language and tone of elements of paper EAC they received from the CRA."

Interviews with CRA staff also suggest that tone of EAC may be too severe especially with first contact correspondence. Telephone enquiries staff commented that many calls originate from distressed taxpayers calling after receiving CRA EAC because they are fearful of the repercussions of not doing things right.

TNS respondents were less harsh with respect to the tone of EAC. When asked to what extent you agree that the tone of the EAC is respectful, responses ranged from 57% to 85% for the different types of EAC.

Conclusion:

A professional look, hierarchical structure, clear messaging and engaging tone are essential elements that significantly contribute to understandability especially considering that Canadians read at differing levels of literacy proficiency. These are elements that could contribute to reinforcing the CRA's brand, and building and maintaining public trust and confidence in the CRA's integrity, impartiality, and commitment to service.

4.1.3 Most recipients will contact the CRA by telephone if they do not understand the EAC they receive

An unintended impact of not writing clear EAC is a higher rate of telephone or written enquiries. TNS survey respondents indicated that their first choice would be to telephone the CRA if they could not understand EAC they received from the CRA (see Table 8).

Table 8: Percentage of respondents within recipient groups who said they would contact CRA by telephone when correspondence is not understood

	Charities	Benefit Recipients	Individuals	Businesses
English	52%	59%	52%	51%
French	66%	66%	62%	44%

Source: CRA Taxpayer Communications Clarity Testing (TNS)

In 2011-2012, the CRA telephone enquiries services answered over 22 million taxpayer calls.^{Footnote 23} While enquiries staff acknowledged that their main call drivers were not EAC specific, they did indicate that the number of EAC related calls were still substantial. When asked to what extent their workload consisted of answering questions related to CRA EAC, a majority of enquiries telephone staff surveyed said 25% or more of their calls related to EAC. See Table 9 for more details.

Table 9: CRA Enquiries Agents Survey Responses
Of the calls answered over the last 12 months, approximately what percentage related to letters or notices sent by the CRA?

	Less than 25%	25% - 50%	51% - 75%	More than 75%
Individual Income tax enquiries (N=229)	16%	48%	28%	8%
Benefit recipients enquiries (N=229)	7%	31%	42%	20%
Business enquiries (N=98)	10%	37%	30%	23%

Source: PED survey of Agency Staff

Of the calls relating to EAC, a majority of enquiries staff estimated that 25% or more were about not understanding the correspondence. See Table 10 for more details.

Table 10: CRA Enquiries Agents Survey Responses
Over the last 12 months, approximately what percentage of enquiries calls relating to correspondence were a result of the recipients not understanding the content of the letters/notices?

	Less than 25%	25% - 50%	51% - 75%	More than 75%
Individual Income tax enquiries (N=229)	21%	41%	26%	12%
Benefit recipients enquiries (N=229)	10%	27%	29%	34%
Business enquiries (N=98)	18%	33%	32%	17%

Source: PED survey of Agency Staff

Unintended Impacts

Other implications, although not measured as a part of this evaluation, relate to compliance and hardship. Benefits recipients who expect to receive and are dependent on a monthly benefit may be negatively impacted by the cessation of receiving that benefit as a result of failing to comprehend and respond to the EAC. Similarly, the CRA will need to pursue other means to secure compliance (another letter, a telephone call, an in person arrangement) should the recipient not respond to a demand or request as a result of not understanding the EAC.

4.1.4 Benchmarking of other tax administrations reveal that they have recently enacted changes to improve the understandability of their EAC

United States

In July 2008, the IRS embarked on a major initiative to improve the clarity, accuracy, and effectiveness of its EAC. The IRS recognized its EAC appeared dated (see Figure 3 for examples^{Footnote 24}) and wanted to, in part, simplify and clarify the language used in its EAC. The objectives were to improve taxpayer understanding and reduce some of the burden placed on taxpayers by EAC.

The IRS, with the aid of the staff at Siegel+Gale (now with Siegelvision), conducted a review of its EAC. Similar to the findings in this evaluation they found that the differences among many letters reflected internal IRS structure, as opposed to taxpayer needs. Yet, despite the systems-driven structure, letter production did not take advantage of existing technological capabilities. And from a communications perspective, many letters lacked a logical framework and a consistent voice.^{Footnote 25}

Figure 3: Examples of IRS EAC prior to the initiative to change its EAC

Source: Making a Noticeable Difference (IRS presentation)

In response, the IRS developed a framework based on best practices and the input of both employees and stakeholders. The new design framework addressed consistency across different notices, a logical flow of information, a layout and design that aids readability and comprehension, and a tone that aligns to the circumstances. Each notice clearly explains: why it was sent, how the recipient should respond, what the consequences would be should the recipient not respond, and where the recipient could find additional resources, publications, and services. Refer to Figure 4 for an example of an IRS notice designed using the framework.

Figure 4: Overview of the IRS Design Framework for EAC (first page)

Source: Making a Noticeable Difference (IRS presentation)

As a point of comparison, Figure 5 shows a CRA notice of taxpayer requested reassessment and the above notice.

Figure 5: CRA and IRS comparison – Notice of Reassessment

Source: Making a Noticeable Difference (IRS presentation)

United Kingdom (UK)

An objective of the UK Her Majesty's Revenue and Customs (HMRC) is to ensure that correspondence is read, understood, and acted upon. Approximately 10 correspondence types make up 59% of the HMRC's EAC. The HMRC has undertaken extensive testing on these 10 pieces of correspondence over time. Testing involves focus groups, individual interviews, and discussions with stakeholders which allows them to ask questions about understanding, layout, user-friendliness, and whether they perceive their letters as a call to action (to file, pay, provide information, etc.).

The HMRC also collects quantitative data using online testing panels and a communications lab (COMLAB) developed specifically for HMRC. This allows them to use "hot spot" technology (similar to highlighter) to identify key areas where people have difficulty understanding the EAC.

The HMRC is able to store the results of the data from its testing and compile a history of what does and what does not work. The results from COMLAB can be used when considering any future modifications to correspondence.

Some of the lessons learned by the United Kingdom are consistent with what plain language experts recommend to improve understandability (for example, make it easy, highlight key messages, and use personal language). They found that there is a minute window of opportunity to get a reader's attention, the average reader has an attention span of 2 minutes, and readers generally focus on headings, boxes, and images.^{Footnote 26}

Australia

The Australia Tax Office (ATO) redesigned some of their EAC based on UK research emphasising fairness, reciprocity, and social norms. Figure 6 shows a before and after version of their overdue payment reminder notice.^{Footnote 27}

Figure 6: Changes made to Overdue Payment Reminder by the Australian Tax Office

Source: ATO

To assess the impact of the changes on taxpayer behaviour, the ATO set up a randomized experimental design involving the revised and original overdue payment reminder. They sent different versions to 100 random debt cases. The cases were tracked for 2 months after the reminder was sent. The responses to the "after" version was compared with responses to the "before" version. The results showed a positive effect on compliance amongst recipients of the "after" version (see Table 11).^{Footnote 28} Basically, it resulted in more arrangements, increase in payments, fewer unpaid accounts, and a lessening of overall debt.

Table 11: Results of the ATO's Overdue Payment Reminder Pilot

	% change
Enter Arrangement	4.3%
Paid in part or in full	6.8%
Unpaid	-2.6%
Increased Debt	-4.2%

Source: Applying behavioural insights to reduce fraud, error and debt, Cabinet Office Behavioural Insights Team (February 2012).

The ATO also emphasizes the importance of using the correct tone in its EAC. They group their EAC into four categories depending on the stage of contact. First contact, they adopt a tone that portrays the professional advisor and educator. As things escalate (i.e. they do not obtain the required response) their tone changes from fair advisor to trusted authority and finally to firm and forceful administrator.

4.2 How well is the CRA administering EAC?

The focus of this section will be on internal processes, structure, governance, accountability, training, systems, tools, support, etc. This includes an examination of how the CRA is organized to deliver EAC. Best practices of other tax administrations are also presented.

4.2.1 The CRA, led by PAB, has made a concerted effort to improve EAC over the past 9 years

Since 2005, PAB has worked in conjunction with the branches and regions to improve the quality of EAC. In 2007, PAB implemented an EAC framework to:

• formalize the structure and encourage consistency in EAC activities;
• define roles and responsibilities;
• assist the CRA in moving towards greater consistency in messaging; and
• develop common tools and guidelines.

PAB established a working group and then a steering committee that focused specifically on EAC. All program branches^{Footnote 29}, as members of the steering committee, were asked to conduct a review of their EAC and EAC-related activities. As a result, all program branches developed and implemented action plans to improve the quality of their EAC. Although there were concerns expressed over the subjective nature of some aspects of quality, the reviews identified errors and inconsistencies in EAC.

Branches continue to work on improving the quality of their EAC. For example, the Assessment and Benefit Services Branch and Information Technology Branch launched Enterprise Correspondence (EC) to address the format and archival limitations of existing system applications. The Taxpayer Services and Debt Management Branch implemented an ongoing review of its accounts receivable EAC to ensure that it is clear, concise, and accurate. The Compliance Programs Branch has implemented a new audit quality review program and online learning product to address the most common written communication issues in auditor EAC. In 2012, PAB replaced the EAC Steering Committee with the Plain Language Implementation Committee.

Since 2009, PAB has devoted time and effort to promoting the use of plain language. In 2009, PAB developed and implemented a 2 day course on plain language. As of November 2013, 6,225 employees in headquarters and the regions attended the course and 150 plain language instructors had been trained to deliver the course. Approximately 59% (n = 3,694) of participants submitted feedback after completing the course and 83% rated it as relevant to their job and 87% rated it as useful. PAB is currently working on developing an online plain language course.

PAB also developed and maintains a Writer's Toolbox which is available to all employees on the InfoZone. The Writer's Toolbox was created to collect, in one location, the tools, resources, and information employees need when preparing written communications, including EAC. The different elements of the toolbox are intended to guide employees in using standard terminology, writing more clearly and simply, and applying a consistent style. The toolbox has had over 400,000 visits in the last two years.

No one area within the CRA is ultimately accountable for the administration of EAC

PAB is responsible for the communications policy but ultimately it is Agency management that must support and enforce its adoption. In 2007, an Agency wide steering committee, led by PAB, implemented a framework with the objective of formalizing the structure of and encouraging consistency in EAC activities.

The CRA does not identify EAC as a service delivery channel despite the almost 129 million pieces that are sent out every year

The corporate business plan clearly identifies service as one of the three pillars of an effective tax administration.^{Footnote 30} The plan recognizes that the information needs of Canadians vary widely and, as a result, the CRA will provide current information and respond to enquiries through several service channels. The plan makes multiple references to different channels such as the website, enquiries services, and outreach, throughout the document. Although EAC is one of the primary ways in which the CRA provides Canadians with information, the plan does not make reference to EAC as a service channel.

4.2.2 Despite the CRA's efforts to improve its EAC there are major obstacles in place that limit what can actually be achieved

As indicated earlier, the CRA has invested a significant amount of time and effort to improve its EAC over the past 9 years. Considerable efforts have been made on the part of PAB, the program areas responsible for administering EAC, and the regions. However, our research has indicated that the strategic direction and strategic initiatives do not always align with what the program areas and regions can, in reality, implement given their other priorities and the resources available to them.

Barriers to Implementation

The evaluation found that, when viewed as a whole, the CRA's ability to successfully administer EAC was impeded by:

• complex tax and benefit laws that mandated what information needs to be included in EAC (or the sending of EAC);
• fragmented business processes both within and across many branches;
• approximately 15 letter creation systems and over 40 input systems;
• high volumes of EAC and high volumes of verses that are used to generate EAC (some of which may be obsolete or unnecessary);
• variability in content, style, format, and writing style; and
• non-existent, incomplete, or inconsistent measures of effectiveness.

The information technology systems and infrastructure currently in place make it challenging to administer EAC

The CRA relies heavily on its information technology systems and infrastructure to conduct EAC-related activities. We estimate that about a third of CRA employees are involved in automated, employee-customized, or employee-authored EAC.

Automated EAC is typically in response to information provided or not provided by Canadians. Many of the systems used to automatically generate EAC are legacy systems. The age and limited functionality of these legacy systems do not allow for much flexibility in making changes to the content, format, or design of the EAC they generate. Many of the flaws identified by Siegelvision can be traced back to the limited functionality of these systems. In the area of effective design and inviting graphics, where Siegelvision scored all 23 pieces of EAC as poor, the feasibility of making these changes with existing functionality may be limited.

The Business Client Communications System (BCCS), for example, may pull information from several systems to create a single piece of EAC. BCCS uses 112 templates and 10,177 verses to create communication items. The selection of verses from these systems to compose a letter may directly contribute to some of the understandability issues identified in this study.

Most customized correspondence is also generated using legacy systems such as the Electronic Letter Creation System (ELCS). These systems were designed to automate the creation, review, printing, and storage of letters. The objectives of these systems were to improve efficiency, ensure quality and address consistency of EAC through the use of pre-approved templates and verses (ELCS has 1 template and 5,503 verses).

ELCS was the system most commonly used by the employees we interviewed and surveyed (47% of respondents) to generate customized EAC. While there are benefits associated with ELCS, many employees expressed frustration working with ELCS and the quality of the EAC it generated. One quarter of those surveyed who use ELCS (26%) rated it as "hard" or "extremely hard" to use when looking for verses.

The CRA has recognized the shortcomings of the systems it uses to generate EAC. As mentioned earlier, the Assessment and Benefit Services Branch, in partnership with the Information Technology Branch launched Enterprise Correspondence (EC). It is being offered to branches as an option to address the format and archival limitations of existing systems. However, while many program areas have migrated (or plan to migrate) to EC, its use is not mandatory. We did not have the opportunity to interview many employees in the regions that used EC. The few employees that we did speak to who used EC were very positive.

4.2.3 The CRA could learn from what other tax administrations have done to improve the administration of their EAC

United States

Much of what we found as part of this evaluation was similar to what the IRS found when they embarked on a major initiative to improve the clarity, accuracy and effectiveness of their EAC. To achieve their objectives the IRS implemented a centralized approach. In July 2008, the IRS created the Taxpayer Communications Taskgroup (TACT), which examined, not only how EAC was written and designed, but:

• how EAC was governed;
• how EAC was integrated into the IRS service delivery strategy;
• how management and business processes could be streamlined;
• how EAC was measured in terms of is impact; and
• how enabling technology and tools could be improved.

In January 2010, the IRS replaced TACT with the permanent Office of Taxpayer Correspondence (OTC). It serves as an enterprise-wide focal point for EAC from design and development to effectiveness and downstream impact. Its responsibilities include:

• creating and revising EAC;
• forecasting and measuring the downstream impact of new and changes to existing EAC;
• collecting data on EAC;
• evaluating the effectiveness of EAC;
• implementing standards, procedures, and policies for EAC; and
• being the point-of-contact to report the risk of, or actual issuance of, erroneous taxpayer EAC.

Program areas in the IRS are still recognized as the subject matter experts; however, the OTC is ultimately responsible for IRS EAC.

Australia

The ATO also takes a more centralized approach to EAC. While business areas are responsible for managing their own EAC, the ATO has established a corporate gatekeeper for all outbound EAC (paper or electronic).

The gatekeeper role is to undertake the end-to-end development and maintenance of all ATO outbound communications in consultation with the requesting business area. They champion and promote the move from traditional to electronic channels and offer solutions to assist in the management of inbound responses to outbound messages. Each business area has an EAC coordinator who meets weekly with the gatekeeper. The ATO established a corporate senior executive steering committee for EAC. They also established an enterprise EAC forum that is responsible for undertaking the practical tasks related to decisions made by the steering committee such as prioritising of EAC into the information technology systems.

Revenu Quebec

In the early 1990's, an increasing volume of communications led the organization to design and implement a standardized correspondence creation system. They launched the Generalized System of Standardized Communications (GSSC) as part of an overhaul of all its administrative systems. The integration of all administrative communications systems was completed in the fall of 2005. Since communications were standardized and integrated into an overall management communications system (both paper and electronic versions), Revenu Quebec has greatly enhanced its services to produce personalized communications while ensuring a better quality control of its correspondence items.

5. Conclusion

Overall, the understandability of Agency EAC and its administration could be improved. Improvements to design and administration would likely result in greater efficiencies and, potentially, better outcomes achievement (i.e. improved service, reduction of red tape, improved compliance).

6. Recommendations

The CRA has an opportunity to benefit from the considerable research and findings of this evaluation to move forward to address its shortcomings in EAC. It is noteworthy that the IRS has recently undergone the modernization of its correspondence and has established a model from which the CRA could learn. We recommend that CRA senior management:

1. Consider enterprise-wide options in moving forward to address the shortcomings of EAC.
2. Consider including EAC in the CRA Corporate Business Plan and Service Strategy.
3. Seek opportunities to position EAC to take advantage of technology (electronic forms of communication) and emerging strategies e.g. behavioural economics.

7. Management response

The EAC evaluation was undertaken as part of the CRA's continuing efforts to enhance its service to Canadians. The CRA welcomes the findings of the evaluation, which reinforce the need for a continued and concerted effort to improve CRA correspondence, and agrees with its recommendations.

The CRA has a strong track record of continuous improvement to its communications with Canadians. This includes improved plain language in its tax forms and guides, internal training and tools for employees on the use of plain language, and more recently, its focus on providing simple-to-use online services, including online mail.

The CRA is committed to improving its correspondence as a critical part of its efforts to reduce red tape and to support taxpayers in complying with their tax obligations and accessing the benefits to which they may be entitled.

A plan is being developed to address the issues raised in the evaluation report. In recognition that Canadians increasingly want to receive their correspondence electronically, and that the cost of changing paper-based mailing systems are costly and time consuming, a central focus of CRA's activities under the plan will be on electronic correspondence.

The plan includes the following:

1. The CRA will seek the views of businesses, tax intermediaries, and their associations on how the Agency can improve its notices and letters, through its Red Tape Reduction consultations scheduled for the fall.
2. A further engagement opportunity will be launched in the new year to seek the views of additional stakeholders, including charities and benefit recipients.
3. A summary of the results of these consultations will be made public in 2015, and will form the basis of an action plan, with a particular focus on delivering more correspondence online and in a manner that is clear and can be tailored so that the information needs of recipients, whether simple or more complex, are met.
4. The CRA anticipates that, beginning in February, 2015 and continuing through to early 2016, its top volume letters and notices will be available online to Canadians in simplified, easier-to-understand formats.
5. The CRA will also review the best practices of other tax administrations, and engage a third party consultant to provide expert advice on how to simplify language on complex or technical topics, visually present information in a way that encourages understanding and action where required, and use emerging technologies effectively.

8. Acknowledgements

Program Evaluation Division wishes to acknowledge the assistance provided by management and staff of all implicated branches of the CRA, especially those who took time out of their busy schedules to complete our surveys, meet with us and/or provide us with information. We also extend a special thanks to:

• representatives from the United States IRS, the ATO, the New Zealand Inland Revenue Department, the United Kingdom HMRC, and Revenu Quebec for providing us with insightful and thought provoking information on their tax administrations;
• representatives from the British Columbia Securities Commission, Canadian Payroll Association, Chartered Professional Accountants Canada, ABC Life Literacy Canada, Decoda Literacy Solutions, Literacy Partners of Manitoba, and Employment and Social Development Canada; and
• Cheryl Stephens (plainlanguage.com) and Brigid Hayes (Brigid Hayes Consulting). Both are renowned experts in their fields (plain language and literacy respectively) who gave freely of their advice, experience and knowledge.

Appendix A: EAC Reviewed by Siegelvision and TNS

No.	Type of EAC	Reviewed by Siegelvision	Reviewed by TNS
1	CCTB and GST/HST Credit Missing information letter	√
2	CCTB Missing Information notice	√	√
3	CCTB Notice of redetermination	√ Detailed	√
4	GST/HST Credit Validation and control - marital status - initial contact letter	√	√
5	CCTB Validation and control - overlimit - initial contact letter	√
6	Corporate Income Tax Notice of assessment	√	√
7	Corporate Income Tax Notice of reassessment	√
8	Payroll Source deductions - statement of account	√ Detailed	√
9	GST/HST Pending cancellation notice	√
10	Individual Income Tax Processing review - additional information validation letter	√ Detailed	√
11	Individual Income Tax Processing - matching - additional information letter	√
12	Individual Income Tax Notice of assessment	√ Detailed	√
13	Individual Income Tax Notice of reassessment	√
14	Canada Pension Plan Record of earnings - discrepancy letter	√ Detailed	√
15	Information Returns Notice of assessment	√
16	GST/HST Notional assessment	√
17	Payroll Request to file letter	√
18	Individual Income Tax Request to file T1 return letter	√	√
19	Corporate Income Tax Request to file T2 return letter	√	√
20	Individual Income Tax Soft collection letter	√	√
21	GST/HST Minor balance letter	√
22	Charities Notice of intention to revoke	√ Detailed	√
23	Charities Reminder to file return	√	√

Appendix B: Larger Versions of Figure 1

1. Top-most message is "keep for your records", but urgent issues are relegated to bottom page 2 (you owe money; CCTB entitlement will be reduced by 50%).
2. Four calculations across two pages (vertical/horizontal/indented) make it difficult to trace relationship of original benefit, subsequent credits, payments, and current amount owed. Difficult to follow, especially for vulnerable CCTB audience.
3. Reference "Information used to calculate" is disconnected from resulting "entitlement" amount shown; relationship of income entitlement is unstated).
4. Same amount ($186.74) described three ways: "Amount direct deposited" (page 1) and "credits applied to the amount owing" and "credits applied" (page 2).
5. Tone is authoritarian and overly formal (We will).

Appendix C: Larger Versions of EAC Found at Figure 2

Appendix D: Larger Versions of IRS EAC Found at Figure 3

Footnotes

Footnotes

Footnote 1

Given that it is a specialized workload, executive EAC (responses from the Minister, Commissioner, and Assistant Commissioners) was not included in the scope of this evaluation.

Return to footnote 1 referrer

Footnote 2

Taxpayers include all persons having reason to interact with or be engaged by the CRA.

Return to footnote 2 referrer

Footnote 3

CRA Public Affairs and Communications Policy (July 2012)

Return to footnote 3 referrer

Footnote 4

CRA EAC Framework, Public Affairs Branch (January 2007)

Return to footnote 4 referrer

Footnote 5

Subscribers to these electronic accounts may opt to receive correspondence such as the Notice of Assessment within their accounts and be notified by way of e-mail.

Return to footnote 5 referrer

Footnote 6

University of Reading, Simplification Centre, Criteria for clear documents: a survey, April 2011

Return to footnote 6 referrer

Footnote 7

All elements used to assess understandability

Return to footnote 7 referrer

Footnote 8

Panels are the single largest sampling source for online studies in market research. An online panel is a pre-recruited group of individuals or households who have agreed to take part in online market research surveys. People with lower incomes, with less education, having lower literacy levels, living in rural areas or age 65 and older are typically underrepresented on online panels.

Return to footnote 8 referrer

Footnote 9

American Association of Public Opinion Research Report on On-line Panels, March 2010

Return to footnote 9 referrer

Footnote 10

Content for Everyone (making information accessible), Whitney Quesenbery, Center for Civic Design (May 2013)

Return to footnote 10 referrer

Footnote 11

Plain by design: evidence-based plain language, Karen Schriver, KSA Communication Design and Research, PLAIN 2013 (October 2013)

Return to footnote 11 referrer

Footnote 12

Leaders in red tape reduction, questions and answers, CRA Website (January 2012)

Return to footnote 12 referrer

Footnote 13

Survey of Adult Skills or Programme for the International Assessment of Adult Competencies, Organization for Economic Cooperation and Development (October 2013).

Return to footnote 13 referrer

Footnote 14

Statistics Canada - Immigration and Ethnocultural Diversity in Canada, National Household Survey, 2011

Return to footnote 14 referrer

Footnote 15

Literacy and Immigration – Saskatchewan Literacy Network

Return to footnote 15 referrer

Footnote 16

Fields such as reading, visual design, information design, typographic design, literacy, cognitive psychology, educational psychology, technical communications, impact of technology, etc.

Return to footnote 16 referrer

Footnote 17

Resource limitations did not provide us with the opportunity to have TNS survey all of the EAC reviewed by Siegelvision.

Return to footnote 17 referrer

Footnote 18

The highlighter tool allowed survey participants to access a sample of the EAC online and click on text and images in the document that they did not understand or found confusing.

Return to footnote 18 referrer

Footnote 19

These responses cannot be generalized across the Canadian population.

Return to footnote 19 referrer

Footnote 20

CRA Corporate Research – Walker Consulting Group, March 2012 and May 2013

Return to footnote 20 referrer

Footnote 21

CRA External Administrative Correspondence Framework, Public Affairs Branch (January 2007)

Return to footnote 21 referrer

Footnote 22

CRA Corporate Research - March 2012 and May 2014

Return to footnote 22 referrer

Footnote 23

Commissioner's Annual Reports to the Governments of the Provinces and Territories, 2011-2012

Return to footnote 23 referrer

Footnote 24

Larger versions are available at Appendix D

Return to footnote 24 referrer

Footnote 25

Case study | Internal Revenue Service, CarryOn, Siegel+Gale

Return to footnote 25 referrer

Footnote 26

Behavioural Economics in the ATO, International Debt Management Committee, Australian Tax Office (July 2013)

Return to footnote 26 referrer

Footnote 27

Applying behavioural insights to reduce fraud, error and debt, Cabinet Office Behavioural Insights Team (February 2012).

Return to footnote 27 referrer

Footnote 28

Applying behavioural insights to reduce fraud, error and debt, Cabinet Office Behavioural Insights Team (February 2012), ATO.

Return to footnote 28 referrer

Footnote 29

Program branches include the Appeals, Assessment and Benefit Services, Compliance Programs, Legislative Policy and Regulatory Affairs, and Taxpayer Services and Debt Management Branches.

Return to footnote 29 referrer

Footnote 30

Summary of the Corporate Business Plan 2014-2015 to 2016-2017

Return to footnote 30 referrer