Evaluating CSR Programs: Lessons from The Civic 50

This year marked the launch of The Civic 50, a national initiative led by the National Conference on Citizenship, Points of Light, and Bloomberg News, to comprehensively evaluate the community engagement practices of the nation's largest corporations.  The goal: to recognize those companies employing the most progressive civic-engagement practices, while illuminating the path for others to follow suit.

Measuring civic engagement is not an easy task.  With input from a broad range of corporate, nonprofit, academic, and government leaders, The Civic 50 ultimately created a multi-tier rubric for assessing corporate practices and policies across a range of dimensions. True Impact was privileged to provide technical expertise and the web-based infrastructure to help translate The Civic 50’s rubric into a national survey and evaluation process.

The following outline summarizes key principles of The Civic 50’s innovative rubric -- sound guidance for evaluating any program or initiative, CSR-related or otherwise.

3 Guiding Principles for Program Evaluation

1. Be outcome focused (begin with the end in mind)

What does success look like? Define that, and then collect data that helps you to evaluate performance compared to that end goal.  The Civic 50 working group ultimately identified seven characteristics (or dimensions) that definied successful corporate community engagement. These dimensions became the foundation upon which the data collection (survey) and evaluation (scoring structure) could be built.

Seven Dimensions of Engagement

    • Measurement / Strategy.  How strategic a company is in designing its programs, and how well it tracks its performance towards achieving those goals.

    • Leadership.  How involved the company’s leadership is in guiding and supporting its civic engagement programs.

    • Design.  How well the company’s civic engagement activities are integrated into the company’s operations.

    • Community Partnerships. The company’s level of stakeholder engagement, including feedback from employees and community partners.

    • Employee Civic Growth. How civic engagement programs recognize volunteer efforts, build employee skills, and nurture talent and leadership.

    • Cause Alignment. How the company addresses issues, whether reactively based on outside pressures, proactively based on company assets, or in a way that aligns the company’s mission with community capacity in order to solve social problems.

    • Transparency and Awareness. How transparent the company is in their civic engagement, including public relations, cause marketing, and disclosure of environmental and labor practices.

The following table summarizes the performance levels required to achieve each medal status (bronze, silver, and gold) across each of the seven dimensions:
civic 50 rubric resized 600

2. Use metrics that reinforce your objectives (mixed methods are OK!)

Often "success" is defined by multiple characteristics, with varying criteria and levels of importance.  A single rating approach may not be sufficient to accommodate these differences.  The Civic 50 provides a good example of mixed-method evaluation, including qualitative rating (for text responses), quantitative scoring (for multiple choice responses), and scoring on a curve (for open-ended quantitative responses).  Further, responses to particular dimensions were weighted more than others, to reflect variations in importance.

Such flexibility with measurement methods is essential if your subjects are multidimensional and nuanced.  Additional details:

Qualitative Scoring

The Civic 50 surveys included long-form qualitative responses detailing the structure, impact, and institutionalization of corporate civic engagement programs. Using a scoring rubric of the seven dimensions of civic engagement, multiple evaluators reviewed each submission's qualitative responses, assigning points to each question.  Final scores for each question were calculated based on an average of all the evaluators' ratings.

Quantitative Scoring

Companies earned points based on their responses to multiple-choice questions capturing the frequency or scope of their community engagement activities. To control for corporation size, points were awarded based on percentages reported and not net values (e.g., the percentage of employees engaged, not the total number of employees engaged).

Curve Scoring

As part of the quantitative scoring process, some questions were evaluated on a curve. For questions with a continuous distribution like volunteer hours or pro bono services per employee, companies were scored on a curve (again, accounting for size).  Consequently, a company in the 25th percentile performed better than 75% of the other ranked companies, even if the performance of all of the companies clustered around very high or very low values.

3. Practical implementation

Finally, it doesn't matter how logically sound the design of your data-collection and evaluation structure is if it's not practical to implement.  That means ensuring that the process is easily understood and implemented by both those who are submitting information to be evaluated and those who are conducting the evaluation.  Examples include:

  • Economy. Extraneous survey questions increase the burden on respondents and evaluators. The Civic 50 was careful to streamline questions and identify those similar to the CECP’s Corporate Giving Standard (CGS), to reduce work for applicants.

  • Specificity. In order to delineate between levels of engagement, specific indicators for bronze, silver, and gold responses were incorporated into the scoring regime. These indicators acted as “bright lines” to ensure evaluators rated each company proposal with a common understanding what types of answers earned what level of points.

  • Simplicity. Even with a large amount of data to collect and review, the entire process -- from the initial Participant Packet, to the the web-based survey tool, to the evaluator interface -- employed clear and consistent categorizations and user-friendly point-and-click tools to help applicants, judges, and managers to easily navigate the process.

To learn more about The Civic 50, visit www.civic50.org, or contact Civic50@NCoC.net.

To learn more about True Impact's proposal management and evaluation tools, click here

 

Click me

Related Posts