Qualitative and Quantitative Research Outreach to the Charities Sector

Prepared for the:
Canada Revenue Agency
January, 2015
Contract Number: 46558-159854/001/CY
Contract Award Date: March 4, 2014
Project Cost: $97,618.55 (HST included)
POR Number: 075-13

Prepared by:
Environics Research Group

For further information:

Media Relations
Canada Revenue Agency
4th Floor – 555 MacKenzie Avenue
Ottawa, ON K1A 0L5
Media.relations@cra-arc.gc.ca

Executive summary

Background and Understanding

Environics Research Group is pleased to present Canada Revenue Agency with this report on findings from a series of focus groups on enhancing and developing communications and educational tools for registered charities. Environics also organized the data collection for the quantitative component of this research, for which we were responsible solely for the fieldwork and tabulation. This report was prepared by Environics, who was contracted by Canada Revenue Agency (contract number 46558-159854/001/CY, awarded 2014-03-04).

Background and objectives

The Charities Directorate of the Canada Revenue Agency is the primary regulator of registered Canadian charities. Its mission is to promote compliance with the income tax legislation and regulations relating to charities through education, quality service, and responsible enforcement, thereby contributing to the integrity of the charitable sector and the social well-being of Canadians.

To ensure that its programs and activities keep pace with the changing needs and interests of registered charities, the Charities Directorate has launched a strategic review of its outreach program. As part of this review, the Directorate will examine how the next generation of internet applications and mobile devices are changing how Canadians learn and seek to determine how best to develop and promote its online tools to meet changing service expectations.

The purpose of conducting charitable sector research was to provide an understanding of the charitable sector's awareness and use of the Agency's informational and educational tools and services, identify the need for new or updated products, and identify areas for improvement in ongoing stakeholder engagement efforts.

The charitable sector research took place in three parts:

• a telephone survey of registered charities;
• a telephone survey of sector umbrella groups; and
• focus groups.

Methodology

Quantitative phases

Survey of charities. A telephone survey conducted to obtain quantifiable data on the target audience of registered charities. The survey was conducted by Environics Research Group and is based on 815 telephone interviews conducted from April 7 to May 2, 2014. The survey data were weighted to the known regional and size distributions of registered charities in Canada. The margin of error for a sample of 815 is +/- 3.4 percentage points, in 19 out of 20 samples (margin of error is greater for subgroups).

Survey of umbrella groups. A telephone survey was conducted to obtain quantifiable data on the target audience of umbrella groups serving charities. The survey was conducted by Environics Research Group and is based on 93 telephone interviews conducted from April 14 to May 6, 2014. The CRA provided a contact list of 312 umbrella organizations across the country.

Qualitative phase

Two in-person focus groups were conducted in Ottawa on July 28, 2014. These groups lasted approximately two hours, and consisted of seven (7) participants in the first group and five (5) in the second (out of 10 people recruited for each group). In addition, 4 telephone focus groups were conducted from July 29 to July 30, 2014. These groups lasted approximately one hour and 30 minutes, and consisted of between three (3) and five (5) participants (out of 6 people recruited for each group). All participants were recruited from a list of registered charities provided by the CRA.

Statement of limitations: Qualitative research provides insight into the range of opinions held within a population, rather than the weights of the opinions held, as would be measured in a quantitative study. While the results of the focus groups cannot be projected to the full population of registered charities, it can be used as directional information in developing educational tools.

Cost of research

The cost of this research was $97,618.55 (HST included).

Use of findings of the qualitative research

The research will be used to improve the format of the CRA webinars for charities and how the CRA promotes them. In addition, the information will be used to determine if and how the CRA should proceed with the development of a mobile application for charities.

Overview of qualitative findings

Usage of CRA webinars

• Very few participants were aware of the webinars the CRA offers. Webinars were typically found while navigating the CRA site (not purposely looking for it), or hearing about them through the CRA's email updates.

Developing the "Ideal" webinar

• Participants were asked to describe what they feel constitutes an ideal webinar:
• Participants seem to like both the "live" and the recorded webinars. Those that expressed a preference for the "live" sessions stated that they valued the ability to ask questions. Those who indicated a preference for the recorded sessions valued the ability to access them whenever, wherever, and as often as they want. They can be paused, fast-forwarded (to skip over certain sections) or replay a segment they may not have completely understood. A few participants mentioned the ideal scenario is to broadcast the webinar live, and offer the recording right after to get the best of both worlds.
• Participants felt strongly about having a presenter who uses a plain-language, conversa¬tional style when presenting, and who is engaging, experienced, prepared and knowledgeable.
• Several participants noted that they experienced technical glitches when watching previous webinars, which discouraged them to use them again.
• An ideal webinar is respectful of the scheduled time and strikes a balance between explaining enough of the content while remaining clear and concise - the ideal maximum length for a webinar is from 45 minutes to one hour.
• Having access to an agenda, a summary or an abstract of the webinar ahead of time allows them to quickly assess relevance and, when considering a recorded webinar, the agenda or table of contents allows viewers to "fast forward" to the most interesting parts. A recap or quick summary at the end of each chapter and at the end of the webinar is also appreciated.
• From a visual perspective, participants prefer slides that are clean and light on text, they like how bullets can help break information down and how strong visuals (videos, frameworks, charts, etc.) can effectively and succinctly convey a lot of information.
• Finally, participants like being able to download content such as the entire presentation deck, parts of the presentation or secondary resources.
• Respondents hope that, if the CRA is planning on developing more webinars, the Agency particularly consider their suggestions related to interactivity and audience engagement. Participants understand that the topics that the CRA would cover could be tedious and occasionally complex which underscores the need to make their webinars engaging, clear and understandable for the average person.

Evaluation of the CRA High-Impact Webinar

• All participants were shown a high-impact webinar on Gifting and Receipting that was recently developed by the CRA. Overall, first impressions of the webinar were positive, however participants highlighted a number of opportunities for improvement, notably:
• The voice of the presenter(s) should be more natural and conversational.
• For a few participants, the graphics, animations, and overall design seemed dated and of low quality. Many also felt that the faceless stickmen were seen as adding to the impersonal nature of the webinar and should be removed, or at least modified to include eyes and a mouth.
• Participants stressed that technical terms, irrespective of the subject matter presented, should be avoided or at least clearly defined.
• Participants also liked certain aspects of the high-impact webinar they saw :
• The majority of participants liked the quizzes and the use of role play.
• The graphics and the charts were intuitive and effective - in fact, several participants would have liked access to downloadable versions.
• Participants appreciated that phone numbers and a website were provided at the end of the webinar.
• Notwithstanding a few technical terms, the presentation of the information was considered clear and simple, the pace was good and the 20-minute length was appreciated.
• The idea of producing an overview webinar in combination with mini-webinars (roughly 5 minutes long) more focused on a specific sub-topic was very well received by participants.

Suggestions on how the CRA should promote its webinars

• Most participants felt comfortable with the idea of learning about new CRA webinars via email, such as through the Charities Directorate's "What's New" e-mail. Several participants suggested the CRA should partner with umbrella organizations that can distribute the information to other charities. The CRA website was seen as another good channel, since most participants access the website to ask questions or find out information when running their organization.

Reactions to a potential CRA mobile application

• Despite familiarity with mobile applications, barely anyone uses an application specifically to help them run their charitable organization.
• Most participants had a hard time understanding how a CRA mobile application would work or how it could help them, even when examples of features and content were provided by the moderator.
• The key barrier seemed to be that participants do not see themselves using a smartphone or a tablet at any time of the day to access anything having to do with the CRA. This is done at their computer during regular business hours.
• Some of the ideas that had some traction with participants included: using the application as an educational tool (e.g. to stream CRA webinars); to receive notifications when documents are due or when regulations change; to provide a shortcut to CRA information that is presented in a more user-friendly fashion than the CRA website; and, as a tool to update company information electronically.

In general, participants felt CRA is going in the right direction when thinking about creating high-impact webinars and a mobile application for charitable organizations. Their general feeling is that anything the CRA can do to improve accessibility to and user-friendliness of their information would be appreciated.

Political neutrality statement and contact information

I hereby certify as a Senior Officer of Environics Research Group that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Brenda J. Sharpe
Senior Research Associate, Public Affairs
Environics Research Group
brenda.sharpe@environics.ca
(613) 230-5089

Supplier name: Environics Research Group
PWGSC contract number: 46558-159854/001/CY
Contract award date: 2014-03-04
For more information, contact Canada Revenue Agency at media.relations@cra-arc.gc.ca

Date modified:

2015-02-02