Canada does not need to start from scratch to figure out how to govern artificial intelligence responsibly. Developing federal disability policy over many years has taught us lessons that are remarkably relevant to the AI challenge now confronting governments: how to anticipate harms, involve people most affected by them and make those responsible for design and purchase accountable before problems become entrenched.
The Accessible Canada Act moved federal policy toward identifying, removing and preventing barriers rather than placing the onus on individuals to ask that their needs be accommodated. Federally regulated organizations must develop accessibility plans, consult people with disabilities, establish channels for communication and report on progress.
This framework underscores that prevention, participation and ongoing feedback must be built into how complex AI systems operate.
Three federal bodies are particularly well positioned to act as artificial intelligence becomes embedded in hiring, benefits, immigration, government services and other consequential decisions. They are the Treasury Board, departments using AI, and Public Services and Procurement Canada (PSPC).
Treasury Board should require participatory testing
The first lesson from disability policy is that barriers are easier to prevent when the people likely to encounter them are involved before anything is implemented.
The Treasury Board should require departments using high-impact AI to demonstrate that it has been tested with people who may experience it differently, including people with disabilities. This should go beyond technical testing or performance measures. A program can perform well overall and still create serious barriers for particular groups.
Consider automated hiring that analyzes speech, facial movements, response times or interactions with technology. A candidate’s disability may be evident in any of these, but not affect their ability to do the job. A chatbot may function perfectly for most users while being difficult or impossible to navigate by someone using assistive technology.
Canada shouldn’t try to build everything it needs for the AI era
Ottawa’s AI strategy is built to inspire trust, not protect Canadians
The Carney government’s embrace of AI will put lives at risk
Accessibility policy has increasingly recognized that the experiences of people with disabilities help identify barriers that others may not see. AI governance should adopt the same principle.
The Treasury Board should establish a minimum standard for participatory testing before AI is built into government services. Departments should identify who could encounter barriers, include representatives from those groups in testing, document problems identified and explain how they were addressed.
Build permanent feedback and correction into AI systems
The second lesson from accessibility policy is that assessment cannot end when a plan is approved or AI is launched.
Canada’s accessibility rules require that organizations continually cycle through planning, consultation, feedback and progress reporting. The premise is sensible because barriers change, organizations transform and solutions that once seemed adequate may not be so in practice.
AI makes continuing oversight even more important. Data can change, uses can expand and problems become visible only after interactions by hundreds or thousands of people.
Every federal institution using AI should establish a way for continuing feedback and correction. Who receives reports of unexpected outcomes? How can a person challenge a barrier without having to understand the technology that caused it? Who is responsible for investigating? What evidence triggers corrective action? When must a system be retested or suspended?
The objective should not be another reporting exercise. It should be an accountability loop: test, measure, listen, correct and test again.
Departments should publish information about significant AI problems and corrective measures taken, subject to privacy and security constraints. The Treasury Board could establish a common framework so Canadians would not be confronted with radically different accountability measures in different departments.
This would also generate institutional knowledge about what goes wrong after AI is adopted and which fixes work best.
Use federal procurement power to set standards
The third lesson may be the easiest to implement.
The government does not have to wait for Parliament to resolve every question about artificial intelligence before influencing the technology. It has the power of being a customer — a very large one.
The PSPC, working with the Treasury Board, should establish baseline contractual requirements for high-impact AI purchased by federal institutions. Those requirements should include:
- Evidence of accessibility and discrimination testing
- Disclosure of known limitations and groups for whom performance may differ
- Access to information necessary for independent evaluation or audit
- Obligations to co-operate with monitoring once AI is operating
- Requirements to correct significant problems after AI is in use
Departments should also know, before signing a contract, what happens if a vendor cannot demonstrate that its product performs fairly for all people.
For decades, people with disabilities have too often been expected to identify barriers after a building, product, service or technology has already been designed. By then, removing the barrier may be difficult, expensive or impossible.
Federal AI procurement should instead shift responsibility toward designers, sellers and the public institutions that are buying.
Procurement requirements could also influence the broader Canadian AI market. Vendors that know the federal government would demand the above-mentioned requirements would have an incentive to build those capabilities into their products from the beginning.
That is not a barrier to innovation. It is one way the government can help create a market for more trustworthy innovation.
Canada can act before a new AI law
None of these steps requires Canada to wait for a comprehensive new AI law.
The Treasury Board can strengthen expectations for participatory testing. Federal departments can establish permanent feedback and correction mechanisms. PSPC can use its purchasing power to require stronger evidence and greater accountability from vendors.
Our disability policy has taught us to prevent problems where possible, involve the people most likely to experience them, create protocols to identify failures and make someone responsible for correcting them. These, along with using the government’s purchasing power, can help move responsibility closer to the designers and builders of AI programs.
Canada learned these lessons slowly, often after people with disabilities had already paid the price for barriers that should have been anticipated.
Artificial intelligence is developing much faster. We should not have to learn the same lessons again.
