It’s time to move beyond funder-imposed metrics of success and take ownership of our story of positive impact

Deirdre Pickerell, Sareena Hopkins and Lynne Bezanson

Over the past 30 years, and counting, career development professionals (CDPs), their managers and employers, along with various funders, have hotly debated the data that demonstrate the need for services and how to best evaluate the success of various interventions.

The sector has been caught in a seemingly never-ending loop of funding priorities based largely on work-search interventions, which are then measured by number of clients in education and training or employed. In some jurisdictions, these are not only the metrics used to evaluate services, but also what triggers funding. With the shift to performance-based outcomes, service providers are increasingly incentivized to shape their programming and interventions to achieve these narrow outcomes.

Yet, career professionals know that the individuals they serve present with a wide range of needs beyond wanting referrals to training or jobs. CDPs know they make a difference in the lives of their clients in a myriad of other meaningful ways, including positive improvements to hope, confidence and mental health, and reductions in poverty and addiction. These changes have important positive implications not only for the individuals served, but also their families, communities and, broadly, the socio-economic health of the country.

As we consider career development reimagined, there may be no better time to explore what we measure and to shift our mindset. It’s time to move beyond funder-imposed metrics and truly take ownership of our data, our services, our evidence and our story of positive impact. With this in mind, a team of career development leaders recently came together to develop and study a logic model for Canada’s career development sector.

Building a vision of career development

A logic model is a linear picture of a program or intervention that comprises the resources needed – which are called inputs (e.g. staff, budget) – and the services being offered, called activities (e.g. counselling, training). These combine into various programs and services offered, which are called outputs. The inputs, activities and outputs need to result in some sort of expected change in clients who engage with services; these are the outcomes. The outcomes are generally broken down into three timelines for when the desired change might occur, such as short-term, mid-term, long-term.

Within the sector’s logic model, our ultimate outcome (i.e. what would happen if every other outcome was achieved) is to have Canadians manage learning and work to both build the lives they want and to contribute to a thriving and inclusive economy. This is a lofty goal, but it serves as an important north star, shaping the mindset we need to move forward.

What the research showed

One key interest in studying a logic model for Canada’s career development sector was to understand whether respondents could provide evidence for specific outcomes and, if yes, what kind of evidence they could provide. The research consisted of a targeted survey of CDPs across Canada. We had close to 400 responses, with representation from almost every region. The research team explored four distinct outcomes from the logic model; two were short-term outcomes and two were more mid-term:

The research consisted of a targeted survey of CDPs across Canada. We had close to 400 responses, with representation from almost every region. Our key interest was whether respondents could provide evidence for specific outcomes and, if yes, what kind of evidence they could provide.

“It’s time to move beyond funder-imposed metrics and truly take ownership of our data, our services, our evidence and our story of positive impact.”

The majority of respondents felt fairly confident they could provide evidence. However, specific examples of evidence were limited and included data such as year-end statistics, quotes and testimonials from clients, and information from clients about their wages and employment status. Much of the respondents evidence referenced was focused much more on securing future funding than on client progress toward longer-term career development outcomes. As one respondent put it, “I believe most organizations look at the needs of clients when providing services. What makes it hard is to try to match to our funder needs also.”

This is the self-fulfilling loop referenced earlier; what gets measured by funders is what is incentivized to get done, with funders’ priorities potentially superseding clients’ real needs and priorities. But what if we could shift the mindset of all stakeholders, including funders, to recognize that the employability needs of clients should drive services and that their progress toward goal achievement is critically important – and, in fact, the only way to achieve sustainable outcomes? To do this, the sector must have the ability to provide evidence (proof) that clients have made progress, even if they are not yet employed.

---

More from this issue of Careering:

The hybrid future: Shifting employment services to meet client needs

Cross-cutting concerns for public policy in career development

What students want from employers to create safe, inclusive workplaces

---

A new way of evaluating outcomes

Over the past decade, the Canadian Career Development Foundation (CCDF) has partnered with multiple jurisdictions and frontline CDPs across Canada on the “art of the possible” when it comes to employability assessment, service planning and delivery, data collection and the story of impact our sector can tell. The resulting tool, PRIME, engages clients in a robust assessment of employability strengths and needs, captures a richer range of progress and outcomes indicators, and uses this data to feed improved service and impact. PRIME is now being integrated into career/employment services, with clients actively shaping the interventions they need and CDPs providing strong evidence of the deeper and wider impacts of quality career services.

Here are just a few highlights from the most recent research:

• Statistically significant positive changes in clients across 48 progress indicators across the full spectrum of employability dimensions (e.g. pre-employability/readiness, career exploration/decision-making, skills enhancement, intra-personal factors such as hope, resilience and motivation);
• Statistically significant positive changes in clients across 16 indicators of mental health/wellness;
• Data on client community engagement and involvement in part-time work, volunteerism, entrepreneurship and self-care;
• And, of course, outcome data on employment and training.

CDPs involved in using PRIME as part of this research – and who were able to generate and use the resulting evidence – were transformed as well. A broader mindset shift affected CDPs’ ownership and empowerment, with positive impacts on their sense of professionalism, commitment to ethical practice and quality service, client engagement and – not surprisingly – evidence-based practice.

These kinds of developments that begin to shift our thinking about progress and outcomes are moving the career development profession in Canada forward. Tools like PRIME allow CDPs to take charge of our own data, ensuring we can measure meaningful and concrete progress in addition to the outcome data funders value. Only then will we be positioned to tell full and real stories of client transformation – both the progress they made, and the outcomes achieved.

To achieve this goal, CDPs must shift their mindset to owning their data, and the story it tells. What we measure matters, but decisions about what gets measured must come from the profession, not the funder. Our collective minds must shift from:

• a preoccupation with outcomes to a preoccupation with context, progress and outcomes
• services driven by funder priorities and program availability to services driven by client needs and evidence of progress
• a focus on client numbers to a focus on client change

The research briefly described here creates a foundation for providing tools and supports as our field shifts its mindset to truly embracing our capacity and identity as evidence-based professionals.

Dr. Deirdre Pickerell is an award-winning career development leader. She has spent many years exploring the data that demonstrates impact of career services and has led the recent research teams examining evidence-based practice.

As Executive Director of CCDF, Sareena Hopkins works to strengthen the impact of the career development sector by moving from ideas to action in areas of public policy, research and development, capacity building and advocacy.

Lynne Bezanson is Executive Director Emeritus of the Canadian Career Development Foundation (CCDF), which means that she has been contributing to the career development field for a very long time and simply cannot stop!