The Matrix
New grant assessment scoring matrices explained
“Clarity is kindness” is a mantra repeated often in our office. It applies to many situations including guiding us to a new method for grant assessments. In the past, Calgary Arts Development juries were provided program criteria for the applications they were reviewing and told to score each application accordingly. However, there were limited parameters as to what excellent, good or weak meant so each assessor used their own definitions.
Learning from some best practices and really looking at clarifying definitions, we are introducing a program criteria matrix in 2020 to help our juries, artists, and arts organizations have common scoring criteria.
To quote Morpheus from The Matrix, “Unfortunately, no one can be told what The Matrix is, you have to see it for yourself.”
But, nevertheless, we will try to explain it. The scoring matrix is based on three primary principles:
- Reduce bias as much as possible.
- Provide standard and clear definitions.
- Increase scoring range to ensure a more balanced assessment.
Our hope is that with more clear guidelines and scoring criteria, arts organizations and artists can see how they need to approach the application questions and assessors can more consistently score those applications.
What it will look like in action?
We will use the 2020 Operating Grant Program funding increase as an example. That grant program will be based on four criteria: artistic vibrancy, community connection, organizational capacity, and strategy & evaluation. Each of the program criteria areas will be scored within five categories from weak to exceptional. To make this easier, each scoring category has specific criteria that assessors can choose from to reduce arbitrary criteria.
To break it down even more, let’s take artistic vibrancy as an example. The definition for artistic vibrancy is:
- The organization understands and demonstrates how they develop, preserve, or innovate within their artistic and cultural disciplines and communities.
- The organization understands and demonstrates how they contribute to the vibrancy and vitality of Calgary.
- The organization provides relevant and high quality artistic or cultural programming aligned with their mandate and organizational context.
- The organization understands and demonstrates how their programming and services create opportunities for artists, participants, or partners to develop and present artistic work.
And the scoring ranges for artistic vibrancy are as follows:
| Weak | Fair | Good | Strong | Exceptional |
| 1 – 18 | 19 – 25 | 26 – 29 | 30 – 32 | 33 – 35 |
And the artistic vibrancy criteria for each scoring category are as follows:
Exceptional: Responses to application questions are clear, relevant, and directly address criteria. The applicant demonstrates a deep understanding of their role in Calgary. The application provides a clear and distinct vision of their context, mandate, programming, and communities. The application creates overwhelming trust and confidence that the organization will be adaptive, resilient, and impactful in the future.
Strong: Responses to application questions are clear, relevant, and directly address criteria. The applicant demonstrates a strong understanding of their role in Calgary. The application provides a clear vision of their context, mandate, programming, and communities. The application creates significant trust and confidence that the organization will be adaptive, resilient, and impactful in the future.
Good: Responses to application questions are sufficient and address criteria. The applicant demonstrates a general understanding of their role in Calgary. The application provides some clarity on their context, mandate, programming, and communities. The application creates trust that the organization will be adaptive, resilient, and impactful in the future.
Fair: Responses to application questions are limited and may not directly address criteria. The applicant demonstrates a vague understanding of their role in Calgary. The application provides limited clarity on their context, mandate, programming, and communities. The application provides limited evidence to create trust and confidence that they will be adaptive, resilient, and impactful in the future.
Weak: Responses to application are insufficient and do not address criteria. The applicant does not demonstrate or articulate their role in Calgary. The application does not provide clarity on their context, mandate, programming, and communities. The application does not provide evidence to create trust and confidence that they will be adaptive, resilient, and impactful in the future.
With these definitions, the assessors now have clearer parameters to assign scores. And as a result, the hope is that this scoring matrix provides more transparent assessment for everyone. As Trinity says, “The Matrix has you.”
Beyond the new scoring matrix, we are also making a couple other changes to Operating Grant juries:
- We are increasing them in size to ensure a spectrum of voices and perspectives.
- For Operating Grant Program increases, the juries will not be deciding grant amounts but instead the grant amounts will be allocated by formula across the cohort based on assessment scores and the total funding envelope.
Along with these changes, we will continue to do the anti-bias training and provide compensation for jury work as part of our commitment to equity, diversity, inclusion, and access. If you (or someone you know) are interested in joining a 2020 jury either for arts organizations or individual artists and collectives, please use the Assessor Nomination Form.
Sara Bateman is the Director, Community Investment & Impact at Calgary Arts Development.
With 12+ years of experience in corporate community investment, evaluation, and social innovation, she leads the team providing meaningful grants and capacity resources to Calgary’s arts organizations and artists.
Her team also demonstrates the impact of the arts to Calgarians and our city. As a graduate of the Social Innovation Graduate program at the University of Waterloo, Sara brings adaptive strategy and system-thinking lens to her work.
