A decade ago, a group of policy wonks obsessed with gathering hard evidence for decision-making began meeting regularly over beers in Ottawa. They put together a campaign for the creation of an independent evaluation watchdog to make sure taxpayers\u2019 money is spent on federal programs that work.<\/p>\n
The idea hasn\u2019t quite gotten off the ground, but the group is more convinced than ever that Canada needs an evaluator general<\/a> reporting to Parliament, making sure data, evidence and evaluation drive policy and funding decisions.<\/p>\n \u201cWe need someone to do the deeper dives and wade into and understand at a level of some depth what policies and programs work for whom, in what conditions and why,\u201d said Steve Montague, a member of the advocacy group and an Ottawa-based management consultant.<\/p>\n \u201cAn evaluator-general could act as a check in an era when government doesn\u2019t have enough independent analysis.\u201d<\/p>\n Government evaluators do a systematic examination of a program\u2019s design, its implementation and ultimate results to understand why it worked or not. They examine the context or events that triggered the program, such as a terrorist attack, opioid crisis, economic downturn or, at a local level, an increasing number of cyclists killed on a particular route. They look at the relevance or need for the program. Was it effective? Are recipients of program assistance and\/or the broader community better off?<\/p>\n The Ottawa group proposing the creation of an evaluator general say this new agent of Parliament would be positioned between the auditor general and the parliamentary budget officer with the three of them providing independent \u201cadvice on the propriety of government spending, the credibility of government budgets and the likelihood that programs and policies will achieve desired objectives.\u201d<\/p>\n The auditor general investigates whether programs are run in compliance with accounting rules. The evaluator-general would investigate whether programs are achieving the expected results.<\/p>\n Management consultant Michael Obrecht, considered an architect of the proposal, said the evaluator general would collect and synthesize evaluations, gathering studies from around the world to help improve decision-making and the determine the likelihood that programs and policies will achieve desired objectives.<\/p>\n The proposal calls for an office with a $2-million start-up budget and a team of experts in program evaluation that could do its own evaluations as well as assess the reliability and validity of data it gathers globally.<\/p>\n Four decades of evaluations<\/strong><\/p>\n The government has had an evaluation function in departments for 40 years, but the scope of evaluations is typically narrow \u2013 often focused on a specific program rather than on the big policy issues that straddle various departments and other levels of government. Treasury Board has an online database<\/a> of more than 1,600 evaluations<\/a> conducted by various departments and agencies over the years.<\/p>\n The advocacy group, however, has long argued the evaluation function is not a priority for departments and has been underutilized for years. For parliamentarians, the reports are too narrow to help them wrestle with complex national and international issues that transcend a single department.<\/p>\n The evaluations have also been criticized as poor and questionable in their quality because departments are evaluating their own work and deputy ministers don\u2019t want to receive bad news that they\u2019ll have to share with the ministers.<\/p>\n \u201cIt would be a naive MP who took a departmental performance report or evaluation at face value,\u201d the evaluator general group concluded in one of its papers.<\/p>\n Frank Graves, president of Ekos Research Associates, is a strong supporter of the group\u2019s proposal and argues it\u2019s more needed today than a decade ago, when he wrote a paper calling for an evaluator general \u201cto champion and raise public consciousness about the importance of knowing what works and what doesn\u2019t.\u201d<\/p>\n \u201cThe capacity to assemble and interpret rigorous empirical evidence to test causal hypotheses about program effectiveness atrophied\u00a0badly from the mid-\u201990s on,\u201d Graves said.<\/p>\n \u201cDespite a [Liberal] commitment to\u00a0restoring evidence-based policy and decision-making there appears to be little progress to recovering that capacity in the federal public service.\u201d<\/p>\n Are evaluations being marginalized?<\/strong><\/p>\n New digital technologies that are changing the world at an unprecedented pace are ramping up the pressure on a risk-averse public service.<\/p>\n The Impact and Innovation Unit within the Privy Council Office has been examining new ways to improve the delivery of government programs and to ensure their efficacy. This new focus is partly in response to the unprecedented pace of technological change, which is putting pressure on normally risk-averse federal policy-makers to keep up with Canadians\u2019 expectations. Public servants can no longer create and map out a new policy or program over five to 10 years and assess how well they work after that.<\/p>\n The unit recently released a \u201cGuide to Impact Measurement,<\/a>\u00a0while its sister organization within PCO, the Results and Delivery Unit, was behind the controversial \u201cdeliverology<\/a>\u201d approach to achieving results on political promises and priorities.<\/p>\n The Impact and Innovation Unit is also overseeing a pilot project called the \u201cImpact Genome<\/a>,\u201d which is using meta-analysis of research studies and predictive analytics to help develop the government\u2019s Youth Employment Strategy.<\/p>\n But some in the evaluation field are worried that the government is marginalizing\u00a0the traditional theory-based evaluations for which Canada is still seen as a world leader<\/a>.<\/p>\n At the centre of these concerns is that a focus on results tends to prioritize short-term targets and on what can be measured simply and easily while missing out on the bigger picture. A similar concern was flagged in a recent report<\/a> by the Organization for Economic Co-operation and Development (OECD) on results-based management.<\/p>\n But Privy Council Office officials insists that high-quality evaluations are as important as ever, and are central to its guide to impact measurement. New measurement tools, however, enable policy-makers to gauge the likely success of a program before<\/em> it is rolled out and to make tweaks and adjustments to it once implemented.<\/p>\n International interest in evaluations<\/strong><\/p>\n Governments around the world are wrestling with how to determine what policies are achieving outcomes based on evidence rather than politics, ideology or gut feelings.<\/p>\n In fact, Obrecht said the advocacy group\u2019s campaign for an evaluator general got new life when other countries recently called for creation of similar posts.<\/p>\n In Australia, the Labour Party supports the creation of an evaluator general office<\/a>. The evaluator would be housed inside the Australian treasury to help assess what policy programs are working.<\/p>\n Last year, a parliamentary working group in France urged the creation<\/a> of an autonomous Parliamentary Evaluation Agency in a bid to boost Parliament\u2019s oversight and role in evaluating policies.<\/p>\n Still, it’s unclear how much appetite there is for another agent of Parliament in Canada. A recent report by Public Policy Forum<\/a> on the nine existing parliamentary watchdogs concluded \u201cfewer, stronger agents\u201d would better serve Parliament. The report recommended a high bar for creating new agents and to consider consolidating the work of agents with similar mandates.<\/p>\n The group advocating for an evaluator-general agrees on the need for the job but the structure of the office has generated much debate.<\/p>\n Some argue that a full agent of Parliament position isn\u2019t necessary, and the work could be done by a chief evaluation officer, similar to the chief science officer appointed by the Liberals. Others say the work could be handled by expanding the role of the auditor general or the comptroller general.<\/p>\n Obrecht argued an independent evaluator general is even more critical with so much more information and misinformation floating around.<\/p>\n \u201cDecision-making seems to be contracted into sound bites, and the capacity to collect extensive information and digest it seem to have been weakened by instant access to easy answers.\u201d<\/p>\n Photo:\u00a0Shutterstock by create jobs 51<\/span><\/p>\n Do you have something to say about the article you just read? Be part of the\u00a0<\/em>Policy Options\u00a0discussion, and send in your own submission.\u00a0Here is a\u00a0<\/em>link<\/em><\/a>\u00a0on how to do it.\u00a0<\/em>|\u00a0Souhaitez-vous r\u00e9agir \u00e0 cet article ?\u00a0<\/em>Joignez-vous aux d\u00e9bats d\u2019<\/em>Options politiques\u00a0et soumettez-nous votre texte en suivant ces\u00a0<\/em>directives<\/em><\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" A decade ago, a group of policy wonks obsessed with gathering hard evidence for decision-making began meeting regularly over beers in Ottawa. They put together a campaign for the creation of an independent evaluation watchdog to make sure taxpayers\u2019 money is spent on federal programs that work. The idea hasn\u2019t quite gotten off the ground, […]<\/p>\n","protected":false},"featured_media":275915,"template":"","meta":{"_acf_changed":false,"content-type":"","ep_exclude_from_search":false,"apple_news_api_created_at":"2025-10-08T02:34:43Z","apple_news_api_id":"e2d7a463-9641-4884-bc4e-54f7e293d28f","apple_news_api_modified_at":"2025-10-08T02:34:43Z","apple_news_api_revision":"AAAAAAAAAAD\/\/\/\/\/\/\/\/\/\/w==","apple_news_api_share_url":"https:\/\/apple.news\/A4tekY5ZBSIS8TlT34pPSjw","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":"","apple_news_is_paid":"","apple_news_is_preview":"","apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":[],"apple_news_suppress_video_url":false,"apple_news_use_image_component":false},"categories":[9387,9372],"tags":[],"article-status":[],"irpp-category":[],"section":[],"irpp-tag":[],"class_list":["post-267317","issues","type-issues","status-publish","has-post-thumbnail","hentry","category-elaboration-de-politiques","category-recent-stories-fr"],"acf":[],"apple_news_notices":[],"yoast_head":"\n
\n