Exchange 2 Improve

WHERE

DESE and XoP transnational partnerships were implemented in the second round of the European Commission’s EQUAL Initiative. The two partnerships co-operated closely in producing Exchange2Improve, as well as under several other topics. DESE had three national Development Partnerships (DPs) from three countries and XoP had six national DPs from five countries. The DESE partners contributing to Exchange2Improve were Response from Finland and the C3 Partnership from the UK. The XoP partners contributing were PSPP from Austria, Dialogos and EC.CO.MI. from Italy and Rhewin from Germany.

Key concerns of DESE partners were sustainability, credibility and competitiveness of the social economy sector and social economy enterprises. A primary objective was to create a collection of good practice and methods for measuring, improving and promoting the impact and performance of social economy enterprises in the partner countries.

XoP partners improved competencies and skills in managing social enterprises through exchange of experience and parallel development in the topics of: financial supporting structure, agency to support new social economy enterprises, building up regional network between public, social and private partners, evaluation of social capital, improving performance of social enterprises, and new ways to promote employment and entrepreneurship for disadvantaged groups in rural areas.

WHY

Exchange2Improve, the DESE and XoP website of good practice, aims to provide a bank of empirical, credible knowledge of social economy. The website includes examples of good practice from SEEs from the partner countries, as well as general performance improvement tools proven in practice and contextualised in SEE reality. The target groups of the website are:

• SEE leaders and quality champions to enable them to review their practice, to implement performance improvement, and to influence decision-makers,
• SEE advisers, enterprise supporters, capacity building workers, employment and economic development centres, and NGO unions, to improve SEEs performance and gain recognition,
• Investors and stakeholders to recognise and demand quality, and
• The quality community to link closer with social economy enterprises.

To make sure that this audience would benefit from the examples of good practice, a transnational evaluation of the examples was considered necessary. The objective of the evaluation was to ensure the users’ interest in the material as well as quality and comprehensibility.

HOW

The material published on the website Exchange2Improve underwent an evaluation by a transnational team of four assessors. The evaluation consisted of two phases: individual assessment and joint assessment meeting.

Individual assessment
At first, the good practice examples were sent to the assessment team members for individual assessment in their home country. In addition to the examples, the assessors received a form for recording their individual evaluation results. Each assessor evaluated the good practice examples against a set of ten criteria on a scale of 1 to 5:

1. The basic idea is clear
2. The organisational background (context) is clear
3. The need/reason for the practice is clear
4. There is evidence that it is working well
5. There is a process for improving it
6. There are innovative elements
7. There are transferable elements
8. It is interesting for SEEs in our country
9. It is interesting for SEE funders, customers and investors in our country
10. It is interesting for SEE advisors and development organisations in our country

The assessors also checked that the text followed the agreed basic structure:

1. Where: Description of the organisation, its main products/services, and personnel, including number of personnel, or the context (network) in which tool has been developed or is being used.
2. Why: Reason for developing the tool, such as area for improvement, problem to be solved, or challenge.
3. How: Description of the tool/good practice in detail (with examples), including expected benefits or achieved results of using the tool/practice, and other considerations, such as lessons learned, development ideas for the future, transferability of the practice.

Joint assessment meeting

After the individual work, the assessment team had a joint meeting to reach a common understanding of the needs for improvement in each of the good practice examples evaluated. The meeting was guided by a facilitator and a secretary recorded the results using the assessment form and a data projector.

The evaluation of an example was started by discussing its title. The aim was to come up with a title that would describe the essential idea of the practice in general terms and that would be understandable to a wide audience (e.g. Customer satisfaction goals and their measurement). After this discussion, the assessment team members presented their grades for one evalution criterion at a time in turns. At the same time they could make remarks concerning the point under discussion. After everyone’s grades and remarks, the discussion was moved on to the next criterion.

The assessment team members had also identified terms that were unclear to them. Many of these were acronyms or terms specific to a partner country, such as Italian ‘type A’ and ‘type B’ social co-operatives or translations of country-specific concepts from the national language into English. Some of the remarks were concerned with English vocabulary difficult for the non-native readers to understand. Such terms were rephrased into plain English.

In addition, the assessment team members made suggestions for keywords for the website Search Index. These described the main topic of the good practice example, such as ‘leadership’, ‘communications’ or ‘employee participation’. The team members also made suggestions for tips or hints to be shown on the website.

As a result of the assessment, it was decided whether the example were to be included, included after revisions, or excluded from the website. For most of the good practice examples, some revisions and modifications were considered necessary.

After the assessment, the team identified the EFQM Excellence Model criteria into which the example would be categorised on the website: 1 Leadership, 2 Policy and strategy, 3 People, 4 Partnerships and Resources, 5 Processes, 6 Customer results, 7 People results, 8 Society results, and/or 9 Key performance results. Also, the assessors chose the category of the ‘Plan-Do-Check-Improve’ cycle into which the example would fit best. However, this framework was not used in the final website as a basis of classification of good practice.

To end the assessment, the assessors made a recommendation to the author. In this recommendation, the essential changes needed in the good practice example were summarised.

After the meeting, the examples were returned to the authors for revision. The idea was that the assessment team members would review the cases once more to check that the recommended changes had been made.

Experiences and lessons learnt

• All of the examples were improved thanks to the evaluation. Culture-specific contents could be reduced, the texts became easier to understand, and more uniform in structure. Aspects interesting for readers in other countries could be emphasised. The assessment brought up what was missing and additions could be made as needed.
• The joint assessment meeting was essential to ensure the quality of evaluation. Individual assessors viewed the good practice examples on the basis of their own experience, but in the joint meeting, the reasons behind each assessment could be brought up and discussed.
• During the process, the assessment team members found many good practice examples quite useful in their own work and learnt from partner countries’ practice. Learning took place in the individual work, but it deepened in the joint evaluation discussion.
• The authors responded to the need for improvements and learnt to describe the good practice in view of the transnational audience. This was also an important learning process.
• The in-depth co-operation produced insight into possibilities of transnational co-operation and how a common learning process can produce meaningful results. The process also deepened the personal relationships between the participants and helped them understand different ways of working, etc.
• The process demands a lot of time, effort and commitment from all participants. It is important that the parties follow the agreed guidelines and deadlines. Good briefing to the authors is necessary and helps the whole process.

Enclosure: Form for evaluation of good practice examples (PDF)

For more information:
Jaana Merenmies, jaana.merenmies(at)syfo.fi
Pekka Saukkola, pekka.saukkola(at)redcross.fi
Julie Smith, julie(at)socialenterpriseworks.org
Evelyne Schneider, evelyne.schneider(at)chanceb.at
Dirk Momper, dirk.momper(at)isb-berlin.de