2010 CRA Individual Web Site User Survey

Prepared for the:
Canada Revenue Agency
February 2011
POR# 049-10
Contract #46558-110973-001-CY

Prepared by:
Phoenix Strategic Perspectives Inc.

Le rapport complet en français sera fourni sur demande.

To request a full copy of this report, please contact Library and Archives Canada at:
613-996-5115 or 1-866-578-7777 or www.porr-rrop.gc.ca/index-e.html

Media Enquiries:
Media Relations
Canada Revenue Agency
4th Floor 555 MacKenzie Avenue
Ottawa ON K1A 0L5
media.relations@cra-arc.gc.ca

Executive Summary

The Canada Revenue Agency (CRA) commissioned Phoenix SPI to conduct research with members of the general public who visited the CRA's Web site in 2010. The main purpose of the 2010 CRA Individual Web Site User Survey was to track user perceptions and satisfaction with the Individuals section of the CRA Web site. Similar research was conducted in 2003, 2004, and 2005. The CRA intends to use these results, together with other Web site performance measurement data, to make modifications to the site, increase its efficiency for individual users, and increase their level of satisfaction with the site.

Methodology

A mixed-mode research design was used, one that included telephone and online surveys. The telephone interviews were completed between November 18th and December 12th, 2010 with 1,008 members of the general public who had visited the CRA Web site since January 1st, 2010 for personal reasons (as opposed to business reasons). Based on a sample of this size, the overall findings can be considered accurate to within ±3.1%, 19 times out of 20. The online survey was completed by 1,311 visitors of the CRA Web site from December 7th to 8th, 2010. Visitors accessed the online intercept survey through links that were strategically placed on four pages of the CRA Web site. The extent to which the results can be projected to the full population of CRA Web site visitors cannot be determined with precision due to sampling issues related to Web site surveys. The data, however, do offer an excellent profile of Web site visitors that can be used by the CRA as directional information.

The results of the telephone survey are presented in the main body of the executive summary, with the results of the online survey presented in shaded boxes.

Visitor Profile

Approximately half (49%) of the visitors to the CRA Web site were between the ages of 35 and 54 years, and most (82%) had at least some post-secondary education (44% completed a university degree). Just over half (55%) reported an annual personal income of $40,000 or more, with 34% earning $60,000 or more a year. The regional distribution of visitors closely matched that of the general population: Atlantic Canada (7%), Quebec (24%), Ontario (38%), the Prairies (17%), and British Columbia (13%). In addition, 75% said they use the Internet daily (17% do so weekly), and close to half (46%) reported that they filled out their 2009 tax return by themselves. With the exception of Internet use, which is higher now than in previous years, and age, this visitor profile is largely the same as it was in the past. Regarding age, the proportion of respondents under 35 years has steadily declined since 2003, from 34% to 17% in 2010.

Overall, the profile of CRA Web site visitors that emerged from the phone survey was corroborated by the findings of the online survey. Exactly half the CRA Web site visitors surveyed were between the ages of 35 and 54, and 82% had at least some post-secondary education, with 42% having a university degree. In terms of Internet use, 77% reported going online daily and 19% weekly. Notable differences were found when it came to income (44% said they earned at least $40,000 a year compared to 55% of phone respondents) and region (a greater proportion of visitors were from Ontario, 46%, and fewer were from Quebec, 11%).

Visits to CRA Web Site

The large majority (85%) visited the Web site on multiple occasions since January 1st, 2010. Compared to previous years, in 2010 there were fewer first-time visitors of the site: 14% versus 20-21% in 2003 to 2005. Once on the Web site, a variety of search methods were used to look for material or information, with 71% of visitors at least moderately satisfied with the search methods. Turning to use, the CRA search feature was used more than any other search method. In total, 71% of visitors used this search method and more than two-thirds (68%) were satisfied with it. Following this, 61% used the links and just over half used the A to Z topics index (55%) or an external search engine, such as Google (53%). Since the last wave of this survey in 2005, use of all but one of the search methods – the CRA site map – has increased significantly.

Online survey respondents were slightly more apt to have accessed the CRA Web site for personal (57%) rather than business (51%) reasons. While the number of times that respondents visited the CRA site since January 1st varied considerably, 64% had been to the site 11 times or more in 2010, including 31% that visited the CRA site more than 50 times. Among those who visited the site for business reasons, a majority did so to find a form or publication (55%) or to obtain information about business tax matters (51%). As for those who used the site for personal reasons, the top two reasons for visiting the site were to obtain general information about income tax or benefits (42%) and specific information about benefits and credits through My Account (38%).

Overall Ratings of CRA Web Site

Overall satisfaction with the Web site remains high (76%) and has changed very little over time (78-79% in 2003-2005). After dropping four percentage points in 2005, success ratings are now back to a high of 78%, with nearly four in five visitors finding everything they sought or completing every task they tried on the site. Most of these visitors, although slightly fewer than in previous years, were satisfied with the time it took to complete tasks and find information on the CRA Web site (71% in 2010 vs. 72-74% in previous years).

Visitors' assessments of aspects of the CRA Web site continue to be positive, but they have declined slightly in all areas except one – confidence in sending personal information to the CRA using the Agency's Web site. In total, 72% were confident that it is safe to send personal information to the CRA using the site compared to 66% in 2003, 71% in 2004, and 68% in 2005. The most noteworthy difference over time was recorded for the visual appeal of the site, the aspect of the Web site that received the least positive assessment. Fewer than half (49%) agreed that the CRA's site is visually appealing (a drop of eight percentage points since the baseline was conducted in 2003). Visitors were most likely to rate the site's content positively – that the information is useful (87%), understandable (77%), and complete in terms of what was needed (75%). Following this, 62% said it was easy to locate what they were looking for.

The perceptions of online survey respondents are virtually identical to those of phone survey respondents when it comes to ratings of the CRA's Web site: 77% were satisfied overall, 80% were able to do everything they tried to do on the site, and most (83%) of those who completed at least some of what they tried to do at the Web site were satisfied with the time it took. Like phone respondents, online respondents offered mixed assessments of aspects of the CRA's Web site. In general, visitors were most likely to rate the site's content positively – that the information is useful (83%), understandable (75%), and complete in terms of what was needed (67%). In addition, 78% were confident that it is safe to send personal information to the CRA using the Web site, and 63% found it easy to locate what they were looking for. Notably, a much greater proportion of online survey respondents, 62%, agreed that the CRA's Web site is visually appealing (vs. 49% of phone respondents).

Use of Web Site Activities

In general, respondents were more apt to visit the CRA site to obtain information than to complete a transaction or use a service. Heading the list in terms of activities were getting general information about income tax or benefits (71%) and finding a form (68%). Following this, 58% visited the CRA Web site to get an answer to a specific income tax or benefit or deduction question, and 52% to submit their income tax return using NETFILE. All other information and services were tried by fewer than half of the surveyed Web site visitors.

Compared to previous years, there have been notable changes in the levels of reported use. In most cases, use has increased since 2005, and generally is considerably higher than it was when the baseline was conducted in 2003. The most significant increases over time were reported for: getting general tax and benefits information (71% in 2010 vs. 49-50% in previous years); getting an answer to a specific question (58% in 2010 vs. 32% in 2005 and 40% in 2004); and tracking their refund status (48% in 2010 vs. 22-30% in previous years).

My Account and Quick Access

Most My Account and Quick Access users claimed to be familiar with the types of things they can do using the services: 75% of My Account and 72% of Quick Access users. In addition, site visitors who have used these services since January 1st, 2010 were satisfied with them: 77% of My Account and 80% of Quick Access users. Satisfaction ratings, however, were more likely to be moderate than strong, with 33% and 32%, respectively, saying they were very satisfied with each service.

Satisfaction with Web Site Activities

Satisfaction with Web site activities was generally widespread, with at least six in ten respondents saying they were satisfied with the process involved in using each activity. Satisfaction was highest for checking an RRSP deduction limit (92%) and tracking the status of an income tax refund (90%). Following this, 85% expressed satisfaction with the process of getting information about their CCTB account, 81% with getting information about their GST/HST credit account, and 78% with entering a change of address online. Visitors were least likely to say they were satisfied with the process of looking for information about registered charities or making a donation (60%).

Visitors' satisfaction with the activities has changed over time. Most notably, there have been significant increases in satisfaction levels since 2005 for checking an RRSP deduction limit (92% in 2010 vs. 75% in 2005 and 83% in 2004), tracking a tax refund (90% vs. in 2010 vs. 73% in 2005 and 80% in 2004), and getting CCTB account information (85% vs. 71% in 2005 and 86% in 2004). For other activities, there has been a decline in satisfaction ratings, with the greatest decreases recorded for finding out about jobs at the CRA (67% vs. 78-83% in 2003-2005) and looking for information about registered charities (60% vs. 69-73% in 2003-2005).

Multiple Channel Use

Fewer than one-third (31%) of surveyed Web site visitors used multiple channels – phone and Web site – to deal with the same issue at approximately the same time. Respondents who used multiple channels were far more likely to visit the Web site first (82%) rather than calling the CRA first (16%). Most of the site visitors who used the phone and Web service channels were not likely to revisit the original channel. Among those who did revisit the original channel, the incidence of doing so was highest among those who visited the Web site first versus those who phoned the CRA first (44% vs. 25%). Compared to previous years, there has been very little change in the number of visitors who used both the Web site and the phone to contact the CRA. That said, a slightly larger proportion of users who visited the Web site first are going back to it after phoning the CRA (44% in 2010 vs. 40% in 2005).

Future Intent

There was widespread agreement that fillable electronic forms and email correspondence would be useful new services. In total, 87% of site visitors attributed value to fillable electronic forms, 83% to email correspondence from the CRA about new items that can be accessed through My Account, and 72% to email correspondence from the CRA related to their personal account. [Footnote 1] Respondents were less likely to attribute usefulness to services that involve new media or technology: 84% said that access to information on income tax and benefits through social media would not be useful and 77% felt this way about access to information on their cell phone or other mobile device.

Impact of Age

Age was the most significant differentiator amongst visitors to the CRA Web site. Younger visitors were generally more satisfied with the site and its features than older visitors, though they utilized different features. They were further more likely to have been able to do everything they tried to do on the site, be a registered My Account user, have used Quick Access, consider the proposed new services to be useful, and have visited the site for a greater number of purposes.

Conclusions and Implications

The research findings suggest that the CRA's Web site is performing quite well as a communications and service delivery channel. Satisfaction with the site overall continues to be relatively widespread. Most Web site visitors found everything they looked for or completed every task they tried, and many were satisfied with the amount of time it took them to do so, as well as with the search methods used. In addition, the site's content, as in past years, received generally positive assessments, with strong majorities agreeing that the information available on the site is useful, understandable, and comprehensive.

Not only did site visitors hold generally positive views of the overall CRA site, they also tended to provide favourable evaluations of the specific activities that they tried at the site. Most visitors expressed satisfaction with the process involved with each activity, felt that it was clear where to go on the site to obtain the information or use the various services they needed, and agreed that the information they found was understandable, useful and clear.

Positive ratings notwithstanding, there continues to be room for improvement. Satisfaction tends to be moderate, not strong, in terms of intensity – surveyed visitors were more likely to express modest levels of satisfaction overall and with respect to specific aspects of the site. As well, the extent to which visitors successfully completed the activities they tried at the Web site varied considerably. Activities with lower success ratings or satisfaction scores, such as finding job opportunities, looking for charities information, and getting answers to specific tax and benefit questions, provide a focal area for the CRA when working to improve the ease of finding information on the Web site. These visitors tended to highlight problems related to finding or accessing information on the site. Efforts to improve navigation for individual users, therefore, can be expected to have a positive impact on users' experience and perceptions of the Web site.

In closing, we offer a number of other related observations for the CRA's consideration:

• When there is multiple channel use for the same information at the same time, the Web site is confirmed as the main go-to destination to start the process. This was not the case to the same degree during the earlier years (2003-2004).
• Satisfaction with multiple channel use is continuing to decline from a high in 2003, and is now significantly lower than in the baseline survey (down 18 percentage points). This is a troubling trend. The top reasons offered to explain the lack of satisfaction focused both on the Web site and phone service: difficulty finding the needed information, the amount of time this took, and insufficient/unclear information, as well as criticisms about phone service (access difficulties, unhelpful representatives).
• There is increased use of almost all the potential methods available to navigate the site, a positive trend when combined with general satisfaction with the search methods. That said, the search method ratings leave room for improvement (68‑71% satisfied), and this is a key function/capability of any Web site.
• The visual appeal of the site is the least positive rating at 49%. The decline compared to previous years might be attributed to changed expectations among Internet users in terms of what constitutes an appealing site (since this is an area of continuous evolution).

Footnote

[Footnote 1]

My Account users were asked about receiving email that could be viewed through My Account and respondents who were not My Account users were asked about receiving email correspondence related to their personal account.

Date modified:

2011-06-16