Redeveloped CRA Web Site Accessibility and Navigability Testing

Prepared for the:
Canada Revenue Agency
March 28, 2014
Contract Number: 46558-133409/001 CY
Contract Award Date: 2013-02-26
Project Cost: $56,504.47 (HST included)
POR Number: 051-12

Prepared by:
Sage Research Corporation

Media Enquiries:
Media Relations
Canada Revenue Agency
4th Floor 555 MacKenzie Avenue
Ottawa ON K1A 0L5
media.relations@cra-arc.gc.ca

Executive Summary

The Canada Revenue Agency (CRA) has developed a new design for its Web site that incorporates features of the Treasury Board Secretariat's Standard on Web Accessibility.

The purpose of this research was to test the new design with users who are disabled to ensure the Web site meets their requirements and expectations. The research also produced information pertinent to the impact of the new Web design on CRA Web site visitors overall, because assessment of navigation of the site considered not only accessibility factors but other design factors as well.

The three target groups for this research were people with:

• Very low or no vision who use a screen reader
• Limited vision who use screen magnification
• A physical disability who use assistive technology or other adaptations to control their Internet browser

The specific information objectives included:

• The participants' ability to locate the information and site locations needed to complete common CRA-related tasks for different parts of the Web site (common in the sense that metrics indicate the tasks are those attempted frequently by visitors to those parts of the CRA Web site).
• Their ability to complete the tasks.
• Their opinions about the ease or difficulty in carrying out these tasks.
• Their suggestions of how navigation to complete the tasks could be facilitated.

Twenty-four online one-on-one interviews were conducted with people with visual or physical disabilities: people who use a screen reader - 6 participants, people who use screen magnification - 7 participants, people with a physical disability - 11 participants. Seventeen interviews were conducted in English and 7 in French. Participants were given assignments to find information or forms on the CRA Web site, and on average each participant completed four assignments (out of eight possible assignments). The interviews were conducted between September 5 and October 23, 2013.

It is important to note that this research was qualitative in nature, not quantitative. As such, the results provide an indication of participants' views and behaviour, but cannot be statistically generalized to the full population.

The project cost was $56,504.47 (HST included).

The research yielded useful feedback on navigation issues associated with each of the eight assignments used in the research, and this is detailed in the body of the report. The Executive Summary will focus only on accessibility-related results.

Overall, success levels in completing the assignments were relatively high across all three disability types: participants in each of the three disability type groups successfully completed a substantial majority of the assignments they attempted. The majority of unsuccessful attempts did not involve accessibility issues.

The general picture is that the CRA Web site performed quite well in terms of accessibility. This was also evident in participants' comments about the Web site, which were generally favourable. That said, there were some accessibility issues, and more for participants with a visual disability than for participants with a physical disability.

PEOPLE WITH A PHYSICAL DISABILITY

The types of technologies used by the participants with a physical disability included:

• Special keyboard arrangement - 2 participants (one-handed keyboard; special cover to prevent pressing more than one key at a time)
• Special hand-operated mouse arrangement - 5 (trackball, track pad, roller bar, mouse sensitivity set to maximum)
• Mouth control of the mouse pointer - 2
• Voice recognition software to control the browser and to enter search terms - 3

There were no obvious accessibility issues. This was true both observationally and in what participants said. Participants typically said their set-up is functionally equivalent to using a regular mouse and keyboard in terms of selecting links and moving around on a Web page, in that they can do anything that someone using a regular mouse can do.

This is not to say using the technology is not without its difficulties. Mouth control of the pointer can be slow or tiring; using the special hand-operated mouse arrangements can be painful if done for an extended period of time; using voice recognition software can be slow in terms of positioning the mouse cursor. However, participants did not feel that the design of the CRA Web site exacerbated any of these difficulties.

PEOPLE WHO USE SCREEN MAGNIFICATION

The levels of magnification used ranged from 2x magnification to 8x magnification. Magnification affects how much of a Web page is visible on the screen at any one time. For one participant, it also affected the layout and composition of the CRA web pages.

The CRA Home page and Landing pages use a 3-column format for displaying navigational elements and content (left and right areas, and a wider centre area). At 2x magnification in normal view, the 3 columns are visible, although content on the edges is partially cut off. There were no accessibility issues among participants using 2x magnification in normal view.

At 3x magnification and higher, the 3-column page structure is not immediately evident. This did not, however, cause significant difficulty doing the assignments - although it certainly is the case that navigation can require more time and effort because of the need to move the mouse around in order to learn what is on a Web page. Participants using higher levels of magnification have strategies for scanning and navigation of Web sites, and these strategies worked for the most part.

Overall, there were no general accessibility issues for people using screen magnification when working with the normal view of the CRA Web site. That said, there were some relatively minor issues with specific elements of the Web site, and a major issue with the "non-compatibility" view of the Web site. The following are the more notable accessibility issues.

• Location of the I want to list
  
  The Home page and the Landing pages all contain an I want to list. On the Home page it is near the top right, while on the Landing pages it is on the upper left.
  
  The location of the I want to was not a factor in being able to complete the assignments for any of the participants using screen magnification. The Home page I want to list was used once by one of the participants, otherwise participants used other navigation options at the Home page to try to complete the assignments. Behaviourally, the I want to list on the Landing pages was more likely to be seen and more likely to be used than was the I want to list on the Home page.
  
  Two of the participants (3x magnification, 8x magnification) commented that they preferred the left-side location on the Landing pages because they are more likely first to scan the left side of a Web page than the right side. One of these participants never realized there was an I want to list on the Home page.

• Wide rows with white space separating the entries
  
  In the Charities section of the Web site on some of the pages with detailed content ¹ , there are some tabular displays which use wide rows with quite a bit of white space separating elements in the rows. There are also no lines demarcating the rows.
  
  The problem for a participant using 8x magnification was that reading an entire row requires horizontal scrolling, and because of the white space it is easy to lose track of which row one is on. The participant suggested inserting a graphic element to define the rows – either a horizontal line separating the rows, or a "leader" (e.g., dots) connecting the elements of a row.

• 2012 T1 General Returns, Forms and Schedules - Effect of long link labels
  (http://www.cra-arc.gc.ca/menu/LBA-e.html)
  
  This Web page has a list of over 80 links, with each line of text spanning about two-thirds the width of the page. Most of the documents are province-specific, with the document name occupying the first part of the link name, and the province being at the end.
  
  The difficulty for a participant using 5x magnification was that on many of the lines, the provincial designation is not visible when looking at the beginning of the line, nor is the form designation visible when looking at the end of the line. This requires a lot of horizontal scrolling to read down the list. The participant suggested there should first be the option to select the province of interest, as this would greatly reduce the number of different links one would have to review.

• "Headline Box" on the Home page
  
  The "Headline Box" on the Home page displays text on the left and a picture on the right. It cycles through five headlines, with each headline being on screen for about seven seconds.
  
  Participants using magnification of 5x or higher complained that this is very hard to perceive and to read:
  
  The problem for a participant using 8x magnification was that reading an entire row requires horizontal scrolling, and because of the white space it is easy to lose track of which row one is on. The participant suggested inserting a graphic element to define the rows – either a horizontal line separating the rows, or a "leader" (e.g., dots) connecting the elements of a row.
  • It changes too quickly to be able to read easily.
  • When looking at the picture one cannot see the text, and vice versa.
  • Depending on the level of magnification and the length of the text lines, the entire width of a text line may not be visible, so the user has to scroll laterally to read the text - all within seven seconds or less.

• "Non-compatibility" view of the CRA Web site
  
  This issue occurred with a participant using 2x magnification. Instead of the normal 3-column view of the CRA Home and Landing pages, she saw a rearranged, single-column view. We have called this the "non-compatibility" view of the Web site because it can be produced by deactivating the "Compatibility view" setting in Internet Explorer. This means that Internet Explorer does not adjust for a Web site designed for a previous version of Internet Explorer.
  
  A very significant feature of the non-compatibility view is that the four tabs - i.e., Individuals and families, Businesses, Charities and giving, and Representatives -no longer show up as links near the top of the page. Instead, they are accessed through an icon on the top right consisting of a button with three horizontal lines. Only when mousing over this icon, does the word Menu appears as a little pop-up.
  
  The participant did not access the Menu icon, and said the icon meant nothing to her and that she would not have clicked on it had she not been told to do so by the interviewer. This poses a significant navigational challenge, because the four tabs represent a major organizing principle for the CRA Web site.

PEOPLE WHO USE A SCREEN READER

There were six participants who used a screen reader. Five used the program JAWS, and one used the program MAGIC. These screen readers provide a large variety of navigation commands. However, for all but one participant, basic navigation typically consisted of using some combination of moving through a Web page by heading and by down-arrow (this reads the next "line" of text, and was referred to by participants as "arrowing").

Overall, there were no major accessibility issues for participants using screen reading software. Similar to the other participants with a disability, as a group they successfully completed the majority of assignments given them, and more often than not the reason assignments were not successfully completed involved an issue unrelated to accessibility. That said, there were several aspects of the CRA Web site that affected accessibility. The most significant ones involved accessibility of the four tabs and of the I want to list on the Home page.

• Accessibility of the four tabs
  
  The four tabs are Individuals and families, Businesses, Charities and giving, and Representatives. These are a major organizational framework for the CRA Web site, and so these should stand out for users.
  
  For a screen reader, the four tabs are underneath the heading Site Menu, which is the third heading from the top of the page, just after Search.
  
  When JAWS reads clickable elements, it states the nature of the clickable object. Most often these are "links". In the case of the four tabs, these are "menus." For example, when JAWS reads Individuals and families, what it says is, "Individuals and families menu". Because the four tabs are "menus" and not links, they do not appear if one uses the JAWS command to bring up a list of the links on a Web page.
  
  There are four accessibility issues involving the four tabs: the heading label Site Menu, the behaviour of Skip to main content, the fact that the tabs are not links, and how MAGIC reads the four tabs.
  
  Heading Label "Site Menu"
  
  Most participants found this label to be unclear, and not to be a heading they would be likely to explore (i.e., have the screen reader read what is underneath the heading).
  
  That said, most spontaneously accessed the four tabs in doing their assignments. Therefore, the problem with the label Site Menu was not a "fatal" one for doing the assignments. However, what happened was that these participants accessed the tabs in spite of the heading label, rather than because they were drawn to it. This happened because of how participants used arrowing on the Home page, and arrowing is a way of hearing everything on a "line" of the page.
  
  Nonetheless, particularly for users who like to navigate by heading, a more descriptive label would help draw attention to the content associated with this heading.
  
  Behaviour of the "Skip to main content" link
  
  The Skip to main content link at the top of the page takes the user to the first heading after the four tabs. A couple of participants had navigation difficulties as a result of using this link because it skips over the four tabs.
  
  The four tabs are not links
  
  The four tabs are "menu" items, not "link" items. A consequence is that if a person uses the command to bring up a list of links, the four tabs do not appear in that list. This resulted in one participant being unsuccessful in completing an assignment, and two participants perceived this as a significant inconvenience for quickly moving to the tabs once one knows what they are.
  
  How MAGIC reads the four tabs
  
  One participant used MAGIC as a screen reader. As with JAWS, down-arrow reads the next "line." However, using the down-arrow at Site Menu results in different behaviour for JAWS versus MAGIC. JAWS reads the first tab, "Individuals and families menu" and stops. MAGIC reads all four tabs without stopping or pausing: "Individuals and families menu Businesses menu Charities and giving menu Representatives menu." Because of the high speech rate setting this participant used (similar to all the JAWS users), this entire phrase is spoken very quickly.
  
  The participant felt strongly that it would be better if MAGIC read the four tabs one at a time, stopping after each. Although he heard the four tabs spoken several times, he never picked up on the Representatives tab because of the way the tabs are read, and this adversely affected his navigation on an assignment involving representatives.

• Location of the I want to list
  
  On the Home page, the I want to list is near the top right, while on the Landing pages it is on the upper left. From a screen reader perspective, it is the eleventh heading on the Home page, while on the Landing pages it is the sixth heading.
  
  Of the six participants using a screen reader, four never heard the I want to list on the Home page. All at some point heard it on a Landing page and some used it there.
  
  The fact that some participants never heard the I want to list on the Home page was not a critical factor in doing the assignments because of the nature of the assignments and the CRA Web site. All the assignments have navigation paths that do not require going through the I want to list on the Home page.
  
  That said, participants preferred the location of the list on the Landing pages because it comes up much more quickly on the screen reader. At minimum, it was suggested the location needs to be consistent across pages on the Web site so that a screen reader user always knows where it is.

Date modified:

2014-04-28