Evaluation Report

Corporate Audit and Evaluation Branch
June 2009

Table of Contents

• Executive Summary
• Introduction
• Background
• Evaluation Methodology
• Evaluation Findings
  • MG assignments are occurring outside the HRMEPP
  • HRMEPP and TLM managers benefit from headquarters-regional assignments
  • Value of participating in HREMPP not always recognized
  • Cost of HRMEPP is higher than a comparable program
  • New human resource management strategies are more relevant for the CRA
• Conclusion

Executive Summary

This report presents the findings on the evaluation study of the Headquarters-Region Managers Exchange Pilot Program (HRMEPP). The HRMEPP was initiated to provide managers from headquarters and regions with the opportunity to gain a better understanding of one another's perspective. The first cycle of the HRMEPP was from October 2006 to February 2007 with 20 exchange assignments. The second cycle of the HRMEPP was initiated from January to June 2008 with 15 exchange assignments.

For this evaluation we surveyed HRMEPP participants, home and host managers, Management Group (MG) Champions and MGs on temporary lateral moves (TLM), with a headquarters-regional exchange component to allow for a comparison of the two groups. Through interviews and document review, we gathered program information and reviewed CRA human resource management strategies.

Our analysis revealed that MG assignments with a headquarters-region exchange component existed within the CRA, without a formal program. On March 31, 2007 there were 34,250 indeterminate employees in the CRA. Within that group 2,639 were classified in MG positions.[Footnote 1] Participation in HRMEPP during 2006 – 2007 represented 0.75% of the total MG population. During 2007 – 2008 this number decreased to 0.57%. Of the total TLM population of 4,742, 8% of the TLMs were at the MG or equivalent level. This percentage decreases to 1% when we take into consideration the headquarters-regional exchange component.

Participants in the HRMEPP and TLMs recognized the benefits of being on assignment, the value of diversifying their experiences, broadening their horizontal view and knowledge of the CRA however this was not always recognized on the employee performance management reports of HRMEPP participants. The main difference identified between the two groups was that a significantly greater portion of TLM participants retained their contacts over time compared to those in the HRMEPP. This suggests that TLMs are more closely linked to the business needs of their home organization and opportunities for continued networking were more likely to occur upon return from their assignment.

With respect to opportunities for networking and sharing of best practices, the MG Champions, participants, home and host managers and MGs on TLMs, identified over 60 committees, working groups and conferences that MGs can attend that have a headquarters-regional exchange component. This suggests that networking and sharing of best practices exist within the CRA and can occur without a formal program.

Expenditures for the HRMEPP in 2006-2007 were $340,062 or $17,635 per participant and in 2007-2008 expenditures were $302,542 or $19,836 per participant. The average length of the HRMEPP assignment was 72 working days in 2006-2007 and 134 working days in 2007-2008. Comparatively, the 2008–2009 Middle Management Development Program (MMDP) had 72 participants in 2008-2009 and expenditures were estimated at $950,400 or $13,200 per participant with an average assignment length of 252 days. The estimated per-day cost was $151 for the HRMEPP[Footnote 2] and $52 for the MMDP. This analysis suggests that the expenditures of accommodations and meal allowances associated with the HRMEPP accounts for the difference in cost between the two programs.

Since the creation of HRMEPP in 2006-2007, new CRA human resource management strategies have been designed to complement those already in place to better respond to CRA's need to have the right people, at the right time and in the right place. New human resource management strategies include the Talent Management Policy and the Management Development Readiness Initiative. These human resource strategies favour the sharing of knowledge, diversification of experience and exchange of skills between individuals in the CRA's different business lines. It is likely that potential candidates to the HRMEPP, as well as all CRA employees, will be accommodated by the CRA's new human resources management strategies that were introduced since 2006-2007.

While there was a perceived need for this HRMEPP to ensure managers are given the opportunity to gain a better understanding of headquarters and regional perspectives, our analysis revealed that the HRMEPP goals can be attained without a formal program. Therefore we recommend that the HRMEPP be discontinued and that the current and planned human resource management strategies be actively pursued.

Introduction

In January 2009, the Management Audit and Evaluation Committee approved the evaluation framework for the Headquarters‑Region Managers Exchange Pilot Program. The evaluation framework identified the following three issues for the evaluation:

1. Is the Headquarters-Region Manager Exchange Pilot Program relevant for the Agency?
2. What outcome has the Headquarters-Region Managers Exchange Pilot Program achieved?
3. Are there modifications or alternatives that should be considered by the Agency?

This evaluation report summarizes the findings related to these issues.

Background

The Headquarters-Region Managers Exchange Pilot Program (HRMEPP) was initiated to provide managers from both headquarters and the regions with opportunities to gain a better understanding of one another's perspective.

It was expected that participating managers would:

• broaden their outlook and horizontal awareness of the organization;
• diversify their experience; and
• network and share best practices with other CRA managers from different branches and regions.

The opportunity for participation was open to all managers and manager equivalents in both headquarters and the regions that were willing to spend between two and four months away from their home base. Management Group (MG) Champions in each Branch and Region were responsible to promote this HRMEPP, find host managers, assignments, and make the final selection of candidates.

The first cycle of the HREMPP was from October 2006 to February 2007 with 20 participants. The second cycle of the HRMEPP was initiated from January to June 2008 with 15 participants.

Evaluation Methodology

For this evaluation study, we used the following methodologies.

• Document Review – An examination of all relevant documentation for the HRMEPP was undertaken and included a review of goals, objectives, individual learning plans, performance appraisals and costing information.
• Cost comparison – HRMEPP costs were compared against the costs associated with the Middle Management Development Program (MMDP) as it is the only program within the CRA that has similar attributes.
• Survey – All HRMEPP participants, home and host managers, and MG Champions were surveyed to obtain their views and experiences on the HRMEPP. A survey was also issued to a group of MG's that were identified in the Corporate Administrative System as being on a temporary lateral move (TLM) with a headquarters and regional exchange component and a similar profile as the HRMEPP participants. This allowed for a comparison analysis to determine the similarities and differences between the participants in the HRMEPP and those on a TLM.
• Interview – Interviews were held with CRA human resource experts who were directly linked to career development programs such as Talent Management, MMDP and MG Focal Point.

Evaluation Findings

There is no definitive way to measure the direct impact of the HRMEPP however indirect measures were used to gather information from various sources to assist CRA senior management in decision making.

MG assignments are occurring outside the HRMEPP

On March 31, 2007 there were 34,250 indeterminate employees in the Canada Revenue Agency (CRA) of which 2,639 were in MG positions.[Footnote 3] Of the total MG population, 0.75% participated in the HRMEPP in 2006‑2007 and 0.57% in 2007‑2008. For comparison, an analysis of data from the Corporate Administrative System (CAS)[Footnote 4] for temporary lateral moves (TLM) revealed that in 2006-2007 there were 4,742 TLMs within the CRA. This represents approximately 14% of the total indeterminate workforce. Of the total TLMs in 2006-2007, 8% of the TLMs were at the MG or equivalent level. This percentage decreases to 1% when we take into consideration the headquarters-regional exchange component. This suggests that MG assignments with a headquarters-regional exchange component exist within the CRA and can be undertaken without a formal program.

In addition, according to CAS data on MG individual learning plans in 2008‑2009, 22% of MGs were interested in a TLM.[Footnote 5] The 2005 Employee Survey[Footnote 6] reported 38% of the 1,833 MGs had requested a development assignment. Of those that requested a development assignment, 32% of the requests were refused of which 71% were denied without any explanation provided. CAS and Employee Survey data did not indicate if assignments were given or if they were related to an exchange from headquarters to regions or vice-versa. Based on CAS and Employee Survey data, we can conclude that the MG interest for development assignments exceed the number of TLM opportunities available in the CRA.

HRMEPP and TLM managers benefit from headquarters-regional assignments

A total of six surveys were developed to gather opinions from participants, home and host managers and MG Champions. A survey was also developed for a group of MGs that were identified in CAS as being on a TLM with a headquarters and regional exchange component. The TLM control group had a similar profile as the participants in the HRMEPP with respect to years of service (average 20 years) and the number of years within a MG or equivalent position (average 8 years). This allowed for a group comparison analysis to determine the similarities and differences, in terms of survey responses, between participants in the HRMEPP and those on a TLM.

Overall survey results for the HRMEPP and the TLMs were similar in terms of satisfaction on their assignment. Both groups recognized the benefits of being on assignment, the value of diversifying their experiences, broadening their horizontal view and knowledge of the CRA. The main difference identified was that TLMs reported retaining their contacts over time compared to HRMEPP participants. There are two possible reasons for this. The length of a TLM tends to be longer and the assignments are more closely linked to the business needs of their home organization thus opportunities for continued networking would naturally occur on the return from their assignment.

With respect to opportunities for networking and sharing of best practices, the MG Champions, participants, home and host managers and MGs on TLMs, identified over 60 committees, working groups and conferences that MGs can attend that have a headquarters-regional component. This suggests that networking and sharing of best practices exist within the CRA and can occur without a formal program.

Value of participating in HREMPP not always recognized

Under the HRMEPP, participants, home and host managers were tasked with establishing expectations for participants. Based on a review of HRMEPP participants' expectation reports, annual employee performance management reports and documents provided by the MG Focal Point and the Human Resources Branch, we noted that for the first cycle of the program, only 3 of the 20 participants had clearly identified HRMEPP performance expectations, 6 had limited performance expectations and 11 had no expectations. For the second cycle, we noted improvements with 10 participants having detailed HRMEPP performance expectations while 1 had some limited expectations and 4 had no performance expectations.

For assessing employee performance results, the same trend was noted with establishing HRMEPP performance expectations. In the first cycle, 3 out of 20 participants received an employee performance management report relating to the HRMEPP's expectations which results in little program visibility in their personal career file. In the 2nd cycle improvements were noted as 11 out of the 15 participants in the HRMEPP included reference to their participation on the HRMEPP.

The overall satisfaction for host managers with the assignment process was positive (88%) however they reported having little contact with the home manager (2006–2007, 12%; 2007–2008; 0%). This could have attributed to the lack of HRMEPP expectation reports and employee performance management reports. In addition, 44% of the 2007–2008 host managers did not note or experience an improvement in their work while most participants reported they did experience improvement. For home managers, the low survey response rate (28% in 2006-2007 and 27% in 2007-2008) and the high percentage of ”not applicable” responses to questions did not enable us to make an assessment on the impact of the HRMEPP on them or their organization.

The HRMEPP participants' experiences were not always included in their annual employee performance management reports. This suggests that the value of participating in the HRMEPP was not always recognized by participants, home and host managers who are jointly responsible for this report.

Cost of HRMEPP is higher than a comparable program

To obtain a perspective on the cost investment of the HRMEPP we compared it against the Middle Management Development Program (MMDP) as it is the only program within the CRA that has similar attributes. The MMDP is a three‑year cycle program to develop high potential employees for more senior level positions. MMDP assignments are usually annual and often can include a headquarters-regional exchange component.

Expenditures for the HRMEPP in 2006-2007 were $340,062 or $17,635 per participant and in 2007-2008 expenditures were $302,542 or $19,836 per participant. The average length of the HRMEPP assignment was 72 working days in 2006-2007 and 134 working days in 2007-2008. Comparatively, the 2008–2009 MMDP had 72 participants in 2008-2009 and expenditures were estimated at $950,400 or $13,200 per participant however the assignments were on average 252 days. The estimated per-day cost, considering assignment length and number of participants was $151 for the HRMEPP[Footnote 7] and $52 for the MMDP. This analysis suggests that the expenditures of accommodations and meal allowances accounts for the difference in cost between the two programs. Table 1 presents the costs for both programs, excluding salary expenses.

This evidence suggests that expanding the HRMEPP either through an increase in assignment length or the number of participants would likely assume an increase in potential program costs.

Table 1: Cost ratio or budget per participant according to development program

Source: MG Focal Point and Management Development program
*Without salaries

New human resource management strategies are more relevant for the CRA

Since the creation of HRMEPP in 2006-2007, new CRA human resource management strategies have been designed to complement those currently in place in order to better respond to CRA's needs to have the right people at the right time at the right place. New human resource strategies include:

• The Talent Management Policy; and
• The Management Development Readiness Initiative (MDRI)

The Talent Management Policy, introduced on September 23, 2008, is founded on the direct participation of managers whose main role will be to establish human resource requirements based on organizational business needs. They will have to determine the gaps between human resource needs and competencies required to reach their operational and corporate objectives. A review and analysis of objectives and employee performance management reports for participants in the HRMEPP revealed some problems occurred in matching the operational needs of the host manager with the participant's skill set. The Talent Management Policy is very promising in this respect since it is based on the identification of organizational needs in terms of employee skills and competencies, for which the ultimate goal is “to have qualified employees in the right position at the right time”. The gap exercise should be able to identify situations in which an employee would benefit from a transfer to another position—for example, from headquarters to a region—in order to gain experience or training in a specialized business line. While employees generally express their needs in terms of assignments, it is nevertheless important that assignments be connected to the operational needs of the organization.

The MMDP, in addition to being maintained and strengthened by increasing the number of participants have created a feeder pool. In January 2009, the Management Development Programs Division of the Human Resources Branch introduced the Management Development Readiness Initiative (MDRI). This program will allow employees that did not show the level of readiness necessary to gain acceptance into the MMDP to undertake training and assignments (headquarters to region or vice-versa) that are geared towards preparing candidates for future entry into the MMDP. Profile data on the HRMEPP revealed that 55% of the participants had applied to the MMDP however only 9% were accepted. A strengthened management development program could meet the assignment needs of a greater number of managers without creating a new program with higher costs.

These human resource management strategies favour knowledge-sharing, diversifying experience and exchanging skills between the CRA's different business lines. It is likely that potential candidates to the HRMEPP, as well as all CRA employees, will be accommodated by the new CRA human resources management strategies that were introduced since 2006-2007.

Conclusion

While there was a perceived need for this program to ensure managers are given the opportunity to gain a better understanding of both headquarters and regional perspectives, the evaluation revealed that MG assignments and networking opportunities with a headquarters-regional component exchange already exist within the CRA outside the HRMEPP. In addition, the new human resource management strategies should support MG development, as well as all CRA employees, while also ensuring that the business needs of the CRA are met. This evidence suggests that the HRMEPP goals can be attained without a formal program.

Recommendation:
We recommend that the HQ-Region Managers Exchange Pilot Program be discontinued and that the current and planned human resource strategies be actively pursued.

Management Response:
The Assistant Commissioners for the Pacific Region and the Human Resources Branch concur with the findings and recommendation.

Footnotes

[Footnote 1]

This information is from the Canada Revenue Agency's 2008‑2009 Workforce Plan.

[Footnote 2]

The estimated cost per day and per participant was based on total program cost (not including salaries) divided by the number of average assignment working days and divided by the number of participants. The MMDP operates on an annual basis and the number of days used in the calculation was 252 working days. Figures are from 2007–2008 for the HQ Exchange Program and 2008–2009 for the MMDP.

[Footnote 3]

This information is from the Canada Revenue Agency's 2008‑2009 Workforce Plan.

[Footnote 4]

A statistical report was prepared by CRA HR experts using codes from the Corporate Administrative System (CAS) that identify only temporary lateral transfers in 2006 and their start dates.

[Footnote 5]

According to the 2008‑2009 Workforce Plan, there were 2,639 MGs within the Agency and, according to the HR statistical report provided, 593 MGs indicated in their ILPs that they would like an assignment.

[Footnote 6]

The 2005 results are substantially the same as those of 2002.

[Footnote 7]

The estimated cost per day and per participant was based on total program cost (not including salaries) divided by the number of average assignment working days and divided by the number of participants. The MMDP operates on an annual basis and the number of days used in the calculation was 252 working days. Figures are from 2007–2008 for the HQ Exchange Program and 2008–2009 for the MMDP.

[Footnote 8]

Cost based on 19 participants versus 20 as one participant had no expenditures