3 July 2018

Evaluation of autonomous community action in Quebec

Summary: Community action in Quebec has developed its own approach to evaluation over the last few decades. The Manuel d’évaluation participative et négociée (Gaudreau and Lacelle, 1999a) is a good example of this practice. It is generally centered on the priorities and needs of the organization, in a perspective of learning and improvement rather than impact measurement.

We can learn from the experience of autonomous community action (ACA) in Quebec in the 1990s and especially from the Pratiques novatrices en milieu communautaire program. This program, which ran from 1995 to 1999, was conducted jointly by the Table des regroupements provinciaux d’organismes communautaires et bénévoles, the Coalition des Tables régionales d’organismes communautaires, the Centre de formation populaire and the Service aux collectivités de l’UQAM. The program gave rise to the Guide d’évaluation participative et de négociation (Midy, Vanier and Grant, 1998), and to a new version the following year, the Manuel d’évaluation participative et négociée (Gaudreau and Lacelle, 1999a).

This work, in conjunction with discussions in a ministerial committee on evaluation and negotiations with the Parti Québécois government of the day, led to a convention on how to evaluate community action. At the heart of this concept was the notion of participatory and negotiated evaluation, which Gaudreau and Lacelle (1999b) define as follows:

  • Participatory evaluation:
    • the organization, its own members as well as those of the community in which it is located are stakeholders in the evaluation.
  • Negotiated evaluation:
    • Internally, evaluation must be done through open discussion and choices must be validated at all levels of the organization;
  • Externally, the community organization is recognized as a partner by its funder, the Quebec government.

(Gaudreau and Lacelle, 1999b, pp. 6‒8, our translation)

These gains, although never formally questioned, are gradually eroded through an inadequate, if not absent, indexing of programs funding. This should explain why community action organizations perceive impact measurement as a threat. However, this is not to say that they are therefore refusing evaluations. Quite the contrary, the community sector is accustomed to evaluating itself, as evidenced by the work of Hébert et al. (2005) or the Centre de formation populaire, whose EvalPop initiative aims to promote and implement “an evaluation by and for the community sector.” But all these evaluation experiences are about much more than impact evaluation and address a host of other aspects. Then and now, what the sector is looking for is an adapted evaluation, adapted to the specificities of the interventions of the organization being studied as well as, and above all, to the needs and aspirations that motivate the evaluation process in the first place. 

The same applies to the philanthropic side. Innoweave, the program sponsored by the McConnell Foundation, aims to support “organizations to be more innovative in order to maximize their impact.” According to an interview conducted by TIESS with this organization, most organizations do not need to measure their impact first, but rather clarify it by identifying the change they actually want to achieve. Innoweave therefore refers them to the module Impact and Strategic Clarity. The Lucie and André Chagnon, J. Armand Bombardier and Centraide Foundations also favour a more formative approach focused on learning rather than encouraging organizations to measure their impact. Several groups such as Communagir, Dynamo, the coop Niska, the CFP and many others are active in the field of evaluation and accompaniment of collective initiatives.