Summary: The first decade following the 1996 Summit on the Economy and Employment was a period of intense reflection on the theme of evaluating the then booming social economy in Quebec. This reflection took place particularly at UQAM, as part of the Community-University Research Alliance (CURA), and at CSMO-ÉSAC.
In the early 2000s, in Quebec, the question of social profitability indicators (indicateurs de rentabilité sociale) in the social economy was the subject of several research projects. For example, a workstream (chantier d’application partenariale, CAP) on the subject of evaluation and systems of information was organized as part of a CURA on the social economy, while CSMO-ÉSAC was likewise looking at the issue (Neamtan, 2000, pp. 18-20).
These initiatives were driven by concerns that had been around for many years but which had gained in importance following the 1996 Summit Conference on the Economy and Employment and in a context of economic downturn that made it “essential to be able to identify impacts of the social economy beyond the creation of jobs” (Neamtan, 2000, p. 10, our translation). Moreover, the “challenge of evaluation,” as articulated then by the Chantier de l’économie sociale, was not much different from the one we are dealing with today:
This involves not only statistically measuring the scope of the initiatives (number of jobs, contribution to GNP, etc.) or the direct social impact (response to unmet social needs, effects on marginalized populations, etc.), but also assessing the more global impacts on the behavior of all actors in society and institutional behaviour (public administrations, local authorities, social movements, the private market) and, consequently, on the current development model. (Neamtan, 2000, p. 13, our translation)
The CURA on social economy has mobilized a number of researchers associated with UQAM, including Marie J. Bouchard (Bouchard, 2004; Bouchard et al., 2001; Bouchard et al., 2003; Fontan et al., 1997 ; Rondot and Bouchard, 2003). In the context of their collaboration, the focus was not impact measurement but rather evaluation using a results and impact approach. In their paper titled “L’évaluation en économie sociale: petit aide-mémoire” (2003), Rondot and Bouchard compile these various findings as follows:
Outcomes are the expected effects of an intervention. It is generally these expected results, depending on the objectives pursued, that will be the subject of the evaluation. Evaluating the results of an intervention is sometimes difficult because it is necessary to be able to isolate the effects specific to the intervention from changes that would have occurred independently of the intervention’s action. (Perret, 2001, p. 25)
It should also be noted that several types of outcomes should be identified by the evaluator: immediate outcomes that can be related to the process (e.g., volunteer and paid worker satisfaction), outcomes achieved at the end of the intervention, and outcomes achieved in the longer term. (Scriven, 1991, p. 250)
But effects other than those already foreseen or desired by the intervention can also occur. In addition to results, the concept of impact covers externalities not incorporated in the price or indirect effects which may be positive or negative (Berthelette, 1998, p. 6). For Laurent Fraisse, “the recognition of social economy actors increasingly involves demonstrating their social usefulness” (Fraisse, 2001, p. 2), which can be measured based on the positive externalities of their activities. The theory driven approach attempts to take these multiple potential impacts into account in the evaluation. We will return to this later.
In the social economy, results and impacts are measured in terms of social profitability or collective benefits. These terms aim to go beyond the evaluation of simple economic benefits (Patenaude, 2001, p. 3).
For instance, let us consider the typology of collective benefits of the social economy proposed by Jean Gadrey. For this author, these benefits are:
- the lowest direct collective cost of some services;
- the indirect contribution to the reduction of various economic, public and private costs;
- the indirect contribution to the increase in the activity rate and vocational training of some users;
- the contribution to the economic and social dynamism of the territories, to their attractiveness and collective quality of life;
- the contribution to the reduction of various inequalities considered excessive;
- the contribution to social capital, local democracy and relational local solidarities (Gadrey, 2002, pp. 3–8)
Impacts on economic activity (tax revenues, salaries, etc.) can also be identified, for example with the input-output model developed at the Institut de la statistique du Québec. For example, a study on the economic and social impact of the community sector in Montreal (Mathieu, van Schendel, Tremblay, Jetté, Dumais and Crémieux, 2001) concludes that, although difficult to compare with the private sector, the community sector generates a significant portion of its revenues. These induced effects can be considered in the evaluation. (Rondot and Bouchard, 2003, pp. 11‒13, our translation)
Some of the reflections carried out within the framework of the CURA on the social economy can be consulted in volume 36(1) of the journal Économie et solidarités, which deals with the social economy and development indicators.
As early as 1998, Lynda Binhas and the Comité sectoriel de main-d’œuvre de l’économie sociale et de l’action communautaire (CSMO-ÉSAC) took an interest in the issue. An approach adapted to the social economy and community action sector was developed in order to report accurately and rigorously on the social and economic impact of organizations in this sector. It is an approach that can be adapted to each sector of activity. Subsequently, two documents were published by the CSMO-ÉSAC with the aim of providing the community with tools for developing indicators and measuring social impact (Binhas, 2004, 2005, 2007). The CSMO-ÉSAC still offers conferences, workshops and training on social impact measurement in order to demystify the concept and support organizations wishing to embark on this process. An online training was also under development in 2020.