On March 9, I spoke on a panel in a course taught by Susan Phillips, professor at Carleton University. The course, “Policy and Program Evaluation,” is part of Carleton’s Master of Philanthropy and Nonprofit Leadership. One principal theme of this program is that nonprofit organizations face strong expectations to demonstrate their effectiveness. It is very important that future nonprofit leaders be knowledgeable and competent in the areas of research and data. I was asked to speak from the vantage point of my role as director of research and data at the Calgary Homeless Foundation (CHF). With this in mind, here are 10 things future nonprofit leaders should know.
- In 2008, Calgary became the first city in Canada to launch a plan to end homelessness. Calgary’s plan was based on a model that has been used in more than 300 communities in the United States. Today, more than a dozen Canadian cities have a similar plan. Also, since 2008 homelessness in Calgary (as measured by point-in-time counts) has decreased on a per capita basis by 17 percent.
- Calgary’s Homelessness Management Information System (HMIS) may be the most sophisticated of its kind in Canada. When Calgary developed its plan, it decided to also develop this information management system, which could, among other things, help it track the progress of the plan. Indeed, last fall, I wrote that many of Calgary’s homeless-serving organizations enter client information into the HMIS database. Today, all Calgary nonprofit programs that receive funding from the CHF must use the HMIS (it’s stipulated in their contracts); and some nonfunded agencies voluntarily use the system for some of their programs.
- The development and implementation of Calgary’s HMIS has been guided by several community committees. For several years, an HMIS advisory committee met to test the “big brother” concern about the system. The committee consisted of both staff from agencies serving the homeless and clients from the sector. It addressed privacy concerns, and clients were part of the decision-making process (they were assured that the police would not have access to client records). A still existing HMIS user group was formed, that is attended by staff who use the HMIS. That group meets on an ad hoc basis to discuss more technical matters, such as updates to the database system and reporting cycles, as well as “how to” matters (it met more frequently in the early days of the system). Finally, now that the system has been up and running for some time, the CHF convenes smaller committees on an ad hoc basis to help guide specific initiatives.
- An important success of Calgary’s HMIS has been its assistance with program referrals. Many (but not all) homeless people in Calgary go through an intake process with the help of the Service Prioritization Decision Assessment Tool (SPDAT). The SPDAT gives the client an acuity score, which assists with their placement in CHF-funded housing programs (information gathered during the SPDAT process is entered in the HMIS). Based on the goals set out in Calgary’s Plan to End Homelessness, clients with higher SPDAT scores are often given higher priority for placement in CHF-funded housing. Committees meet on a regular basis to recommend clients to be placed in the limited amount of subsidized housing available.* The formal name for this entire process is called Coordinated Access and Assessment (CAA). (For more on Calgary’s CAA system, see this recent book chapter by Jerilyn Dressler.)
- Some nonprofit organizations have been happy to share their data with the CHF, others less so. In my experience, before a nonprofit will share data voluntarily with the CHF, they like to know what exactly the data will be used for and how they might benefit from sharing their data. Until they see how sharing data can benefit their organization and its clientele, they’re reluctant to share (unless they’re mandated to do so by their funder). Rather than thinking of receiving data as an entitlement, organizations like the CHF need to work hard with other nonprofits to build trust and demonstrate how data-sharing can be mutually beneficial.
- The CHF disburses annual funding to Calgary-based nonprofits in the homeless-serving sector: to monitor their outcomes and impact, it benchmarks them against key performance indicators (KPIs). The various programs have different objectives—for example, the KPIs developed for some programs emphasize the effectiveness of those programs in creating stable housing situations for their tenants. Thanks to the HMIS database, the CHF staff are able to monitor each funded agency’s progress on KPIs. The CHF then makes annual funding decisions based in part on each funded program’s performance.
- The HMIS provides invaluable support to the benchmarking system. Indeed, this has been one of the system’s major successes. It’s through the HMIS that program performance data is gathered from CHF-funded programs.
- One drawback of HMIS data is that most of its client data are based on self-reporting. However, it should be noted that self-reported information is gathered by an experienced case manager during an in-person interview. What’s more, many well-respected data sources in Canada are also based on self-reporting; these include the Labour Force Survey and the Census. In future, CHF researchers would like to cross-reference self-reported HMIS data with administrative data from the health and justice systems, in order to be able to compare information on the same individual. (Such a research exercise would obviously require client consent, as well as the cooperation of the authorities involved.)
- The main success of Calgary’s initial Plan to End Homelessness was that it helped galvanize public attention and stopped homelessness from rising. When the original plan was developed in 2008, Calgary had experienced a 650 percent increase in homelessness over just 10 years. I consider the 17 percent decrease in per capita homelessness since the original plan was unveiled to be a very impressive accomplishment. There is little doubt in my mind that there are people alive today thanks largely to that plan. In retrospect, eliminating homelessness by 2018 (a key goal of the original plan) was a very ambitious target.
- My principal advice to nonprofit leaders is to be humble with data. By that, I mean they shouldn’t try to over-interpret it. Nonprofit leaders need to be honest about the limitations of their data and the statistical analysis they undertake using that data. They should also be forthright about assumptions they make in long-term projections. When in doubt, they should seek guidance from more senior researchers. Though at times it may be tempting to exaggerate one’s knowledge and foresight, remember that chickens eventually come home to roost. And with that in mind, I’ll remind readers of what the late John Kenneth Galbraith said in a 1993 article in the Wall Street Journal about economic forecasters: “There are two kinds of forecasters: those who don’t know, and those who don’t know they don’t know.”
* Even with Calgary’s sophisticated use of data, the city still has far more homeless people in need of housing than it has subsidized housing units available. Thus, due to a lack of affordable housing, some people experiencing homelessness can wait years to be placed into housing; others die while on the waiting list. That’s a big reason why the CHF endorses this recent policy statement and continues to lobby all levels of government for more funding.
The inner workings of government
Keep track of who’s doing what to get federal policy made. In The Functionary.
The Functionary
Our newsletter about the public service.
Nominated for a Digital Publishing Award.
This blog post originally appeared on the website of the Calgary Homeless Foundation. The following individuals were very helpful in writing it: Britany Ardelli, Janice Chan, Francesco Falvo, Louise Gallagher, Darcy Halber, Chantal Hansen, Ron Kneebone, Ali Jadidzadeh, Jennifer Legate, Kevin McNichol, Natalie Noble, John Rowland and Kelsey Shea. Any errors are mine.
The inner workings of government
Keep track of who’s doing what to get federal policy made. In The Functionary.
The Functionary
Our newsletter about the public service.
Nominated for a Digital Publishing Award.
Do you have something to say about the article you just read? Be part of the Policy Options discussion, and send in your own submission. Here is a link on how to do it. | Souhaitez-vous réagir à cet article ? Joignez-vous aux débats d’Options politiques et soumettez-nous votre texte en suivant ces directives.
