by Hildy Gottlieb
Copyright ReSolve, Inc.
2007© |
|
The phone rings. Someone
wants to know, “What do you charge to do a strategic plan?” or
“What do you charge to do a workshop?”
Instead of responding with a price, we instead have
questions of our own.
What is driving you to
do this plan / this workshop?
What outcomes are you
hoping this work will accomplish?
What do you want it to
look like when we’re done?
What will have
changed?
Almost without fail, the person does not know. No one
has asked them directly about their intended outcomes before.
When we think of “outcomes,” we think of
grant applications asking how we will measure them. We think of Funders
requesting that we distinguish between “real outcomes” and
“outputs.” We think of the time spent in classes and discussion,
trying to figure out how to do what we have been asked to do - measure those
outcomes.
After all the emphasis placed on outcomes in all these
years, most organizations begrudgingly fill in what they hope will pass as an
acceptable answer on their grant application, without ever really understanding
how that measurement will provide any practical information to anyone.
The
purpose behind measuring program outcomes was supposed to have been improvement
in the conditions in our communities. But somehow, with all this measuring, our
communities have not changed. How can that
be?
|
Measuring the Outcomes of the
Measurement Movement (Unscientifically - No Rubric
Required) |
It was about 10 years ago that
Funders started asking a pertinent question: Why do we keep funding programs
that aren’t changing the conditions in our communities?
All these years later, the most visible result of that
question has unfortunately not
been huge change in our communities. It has instead simply been a huge change
in the sector’s emphasis on measurement. What went wrong?
In our own observations about both the sector and
humanity overall, we have often noted that we get what we aim for. And in our
opinion, that has been the main downfall of the measurement movement. While it
has not accomplished what it hoped to accomplish, the movement has, in fact,
accomplished precisely what it has aimed for.
And that is because what the movement aimed us at was
measuring.
The motivations of those who encouraged the sector to
begin measuring outcomes was indeed that we create more significant change in
our communities. These folks rightly noted that we achieve best what we measure
best.
The problem, however, is that before we measure, we
need to aim. Then we can measure whether or not we approached what we aimed at.
And so, without teaching the sector how to aim for
creating significant change - significant outcomes - before encouraging them to
measure their results, all we could measure was what we were already doing.
The effect of the movement, therefore, has been that
the Community Benefit Sector is now measuring like crazy. While we are arguing
over whether we are measuring the right thing or the wrong thing (short-term
quantitative change vs. long term qualitative change), we are still doing our
best to do what we were asked to do. We are measuring.
|
The Potential Behind the
Jargon
|
If you ask most organizations
about measuring their results, they are likely to feed back some of the jargon
the sector has begun to embrace - words like outcomes and outputs and rubrics and logic models.
It is the rare organization that understands the
incredible potential and possibilities behind all the jargon. It is the rare
organization that sees that it isn’t about the act of measuring, but about
creating something worth measuring!
So what will it take for that to happen? It won’t
take more emphasis on measurement, as doing more of something that already
isn’t working is not a likely route to success.
What it will take is that we all step outside our
comfort zones, into the land of what is possible. And when I say
“all,” I mean not just those who provide service in our communities,
but those who fund those services, and those who teach those Service Providers
- Funders and Nonprofit Resource Centers.
For this sector to achieve real, visible, significant
outcomes, we are all going to
have to aim at creating such results. That means we will all have to change the way we see the work
we do.
Once we have taken aim at what we really want to
accomplish, then and only then should we begin to talk about how we might
measure that.
|
From “What We Do” to
“Why We Do It” |
Community organizations are
used to providing programs. That is, in large part, what we as a sector do. We
see a need, and we create a program to meet that need. Whether it is feeding
hungry people, creating a musical performance, keeping a riparian area clean -
this sector is all about “what we do.”
| Outcomes are about these questions:
How will life be better?
For
whom?
And for how long? |
|
The power of accomplishing significant outcomes is not
about what we do. It is about
why we do what we do.
Outcomes are about creating a better future. Outcomes
are about the conditions you want to create both for individuals in your
community, and for your community as a whole.
So now, jargon free, see if you can answer the
following questions.
• What future do you
want to create for your community? What positive conditions do you want to see
in your community?
• What future would
you like to create for the individuals you serve? What positive conditions
would you like to create in their lives?
• What would success
look like if you had a crystal ball and could look deep into the future - for
the individuals you serve, and for the community as a whole?
Those
are the outcomes your programs will ideally be aiming for.
Outcomes are the difference
your program will make in someone’s life, or in the life of your community
- the future you want your program to create in the lives of the people who
walk through your door, and the lives of everyone in your community.
Outcomes are about these questions:
How will life be better?
For whom? And for how long?
|
These questions will also help you craft
your vision statement!
(CLICK
HERE) |
|
Steps for Creating
Outcomes
JARGON FREE VERSION:
Steps for Reaching for Your Organization’s Highest
Potential |
Part of the problem with
reaching for outcomes is knowing where to start. Because, to date, we have
considered outcomes as something to discuss when seeking a grant, we typically
think we should begin the quest for more significant outcomes with our existing
programs - simply identify and measure those outcomes. As noted above, however,
and as you will see in the steps below, it’s not as simple as that.
For that reason, and because you already have
something at stake with your current programs, you will find it easier to
consider this approach with new programs, rather than re-looking at your
existing programs.
By focusing on new programs, you will be starting with
no assumptions except the most important one - that you have the potential for
creating a significant, long term difference. Once you get used to thinking in
this way, we encourage you to revisit your existing programs, using the same
thought-process.
The following steps will begin aiming your new effort
at creating significant improvement - first
in the lives of the people who walk through your doors, and
then, in your community as a whole.
1) Creating
the Future
As you set out to create your program, ask the
questions noted above:
• What future would
you like to create for the individuals you serve?
• What positive
conditions would you like to create in their lives?
• What would you like
their future to be like?
(From there, if you are ready, begin asking those same
questions about your community as a whole.)
Note that these are not problem-solving questions.
That is because creating significant outcomes is not about simply eliminating
one or more particular negative conditions from a person’s life; we want
to create a great future for him/her. Eliminating a problem tends to focus
narrowly. The more positive you can keep the context of your program, the more
positive a force for change you will become.
2) Factors
Beyond Your Mission
As you consider creating that positive future, an
interesting discussion usually ensues. It goes something like this:
If we are to create that
future, we can’t just look at our mission, but other related work as well.
But that isn’t our job! Our job is X - there are other organizations who
do Y and Z! This is just too big - we can’t possibly accomplish all
this!
There are other variations on this theme, but you get
the gist - that creating a positive future often means moving beyond the narrow
confines of your mission, into the related missions of other organizations.
Perhaps creating the future will move your thinking from substance abuse to
parenting to education to homelessness and beyond. Or perhaps creating the
future will move you from music to education to counseling to the
environment.
We have been told so often to stick closely to
“what we do,” that many organizations intentionally avoid expanding
their thinking; they intentionally avoid addressing all of what it would take
to actually accomplish their mission!
So for this step, we encourage you (no - we order
you!) to let your mind wander beyond your immediate mission, out towards all
the other factors that will influence the likelihood of your being able to
create the future you have envisioned.
Write down all those other variables and issues. What
factors would need to be considered, outside the immediate scope of your
mission, for that future to be possible? (The example below may give you some
ideas.) You will see in Step 3 that this is not a futile exercise, but a
critical one, and a practical one at that.
3) Who Else
Can Help Us Accomplish This Future?
As you begin looking at all the factors that will
contribute to your success, now is the time to consider all the other
organizations and individuals in your community, who also have a stake in
creating that future.
Your list will include the organizations and
individuals who are doing some of the related work you noted in Step 2 above,
as well as those who may not already be doing that work, but who are passionate
about creating such change. As you compile this list, don’t forget to
include those individuals who walk through your doors every day - your clients,
patrons, program participants.
To help jump-start your thinking for this step, you
may want to use the Life List Generator on page 25 of FriendRaising (if you do
not already own FriendRaising, you can get just the
Life List Generator here).
4) Working
Together
Bring together all those individuals, and before
starting to craft the program, clarify together what the desired results will
look like. The following questions may help:
• How can we clearly
define the difference we want to make in people’s lives?
• Whose lives do we
want to impact?
• For how long?
5) Indicators of Success
From the answers to the questions in Step 4, discuss
how you might identify whether or not the program is accomplishing the
difference you want to make. With the group you have gathered, ask the
following types of questions:
• How might we measure
benchmarks along the way, to see if we are getting close to changing the
conditions we want to change?
• What indicators
might we look for along the way? What might we measure to see if we are making
a difference?
• How long would we
need to measure those indicators?
Again, we urge you to keep those indicators positive,
rather than focusing on the negative (for example, the difference between
noting the graduation rate vs. the drop-out rate). Generating positive
indicators will help focus the context of your program on the creation of
positive outcomes for both individuals and your community.
By building measurement directly into the root of your
program, you will be able to monitor whether the program is accomplishing what
you wanted to accomplish, or whether you may need to make adjustments. It is
therefore important to note at what intervals and how you will measure. And
then be sure to do that monitoring!
6) Craft
Your Program
Now that you are clear in your understanding of why
you will be doing what you are doing, and you know what results you will want
to be able to measure, you will be ready to craft your program - ready to focus
on the thing you already know how to do so well.
You can see where the approach
of “aiming at outcomes” is a very different approach from providing a
service in response to a narrowly-defined need. This approach - aiming at
creating a whole positive condition for that individual - that is what Funders
and others have been trying to get at with the measurement movement. They just
started from the wrong place; they started with teaching us how to measure,
rather than first teaching us how
to aim.
Unless we
first aim our programs at creating significant change, we can measure all day
long without ever being able measure such change. And the reason is simple:
Without building our programs to accomplish significant improvement in the
quality of life in our communities, there will be no significant change to
measure.
| Without building our programs to
accomplish significant improvement in the quality of life in our communities,
there will be no significant change to measure |
|
Therefore, as you seek to create real results with
your new programs, you will find that there are all sorts of benefits from this
approach, and truly, making a grant application more appealing to Funders is
the lowest on that list of benefits.
First, you will be aiming at accomplishing something
significant - and to repeat just one more time, we are more likely to
accomplish something if we aim at it. While these are indeed long-term results
- results that might not fully unfold over the next 2 years or 5 years, but
perhaps over 10 years or maybe even 20 or 30 years - you will finally be taking
steps towards accomplishing those big picture goals we all have for our
communities. These are the goals that will make a great difference to the
future of the places we love. They will not exclude today’s issues, but
provide a larger context for those issues.
Second, you will want to measure, because if the goal is
those outcomes, you will want to know if you need to make adjustments along the
way - you will want to be sure you can accomplish those incredible goals. And
beyond just your own organization, you will likely have partners in this
effort, and they will ALL want to know together if it is working or if it needs
to be adjusted.
Which leads to one of the biggest perks of all: This
approach to building programs breaks us out of the habit of thinking that we
have to do it all ourselves, and then feeling overwhelmed because we can only
do so much.
Instead, focusing on outcomes makes us want to reach
beyond our walls, to link arms with those who also care about what we care
about. (While this article is not about collaboration, you can see where these
are natural collaborations, built from the inside out, rather than the common
refrain, “The Funder wants to see a collaboration - who can we partner
with?”)
And finally, when we create programs in this way,
building indicators and measurement right into the planning, the question of
“How to measure?” begins to make more sense than it ever did
before.
Because it is always helpful
to see how theory becomes practice, the following are provided as practical
examples of what is possible. Below you will find 3 examples, for three of the
most active participants in the Community-Benefit Sector:
 SEE Below to view examples for
Provider Organizations
CLICK
here to skip to examples for Funders
CLICK here to skip to examples
for Nonprofit Resource Centers
|
 Provider
Organizations
|
 |
Practical Examples
for Creating and Measuring Real Outcomes:
|
Art
for Kids
The mission at
Art for Kids is to provide
“at-risk” kids with productive and creative outlets for their
energies, to reinforce their potential, rather than their risk.*
Step 1: The
Future We Want to Create
Focusing for now solely on the young people who
walk through our doors, these might be our responses:
• We want young
people to choose positive outlets - sports, art, academics - for their energy
and curiosity
• We want young
people to graduate high school
• We want young
people to be excited about their future
• We want to
create a lifelong passion for the arts
• And etc.
Step 2: Factors Beyond Our Mission, that will Affect Those
Outcomes
External factors will be many when it comes to
teaching art to “at-risk” kids. These might include any number of
factors in our labeling a child “at risk” in the first place.
• Poverty (and
subsets of hunger, homelessness, etc.)
• Child abuse /
neglect
• Substance
abuse (either by the parents / guardians, or of the young person
him/herself)
• Lack of an
attentive parent / caring adult
• And etc.
Step 3: Who
Else Can Help Us Accomplish This Future?
From the list in Step 2, it is easy to come up
with names of other organizations and individuals in town who can help. What is
harder to remember to do is to involve young people themselves, as well as
their parents. So don’t forget to engage kids who have succeeded, kids who
are having a hard time, kids who have not succeeded, kids who have gone through
your programs - and their parents.
Step 4: Working Together
If you want to see energy soar in a room, bring
all those people together, and have a skilled facilitator ask the questions we
noted above, about building the future. We recommend a skilled facilitator
because this cannot become a gripe session, and cannot focus on the problems
you want to solve, but instead must focus on the positive future you wish to
create. Problem-solving will NARROW your focus, and outcomes are not about
narrow - they are about the whole human being. If you can keep your eye on that
prize - a positive future for that individual, rather than simply the
elimination of a problem - you will have created the context for not only a
great program, but a great life!
Step 5: Indicators of Success
What positive indicators might we measure to see
if we are making a difference? Some of the following might be indicators (and
you will likely think of others):
• High school
graduation rates of participants
• The rate of
participants who go on to college, or art school, or to become professional
artists
• Rate of kids
who participate in art shows around town / other creative outlets
• Rate of kids
who return to volunteer to teach others in the program
• Rate of kids
who volunteer in other programs (organizations, churches)
• Rate of
participants who come from referrals from other participants
Again, it is important to note that these
indicators are all positive indicators. We don’t want to simply eliminate
one or more particular negative conditions from this person’s life - we
want to create a great future for him/her.
You will also note that these indicators are not
only measurable at the end of the effort, however far into the future that is.
They are also indicators you can use to measure the trends along the way, to
know if the project is doing what you anticipated it would do while en route to
that successful future. (And if not, you will have time to re-assess, and make
adjustments.)
Step 6: Craft the Program
You can already see that the program that will
come out of this effort will be incredible.
• It will have
no problem getting funded, and it will address the many many issues that face
young people today.
• It will aim at
creating a positive impact on both those young people and the community
overall.
• It will be
open to addressing other issues that arise that may not have been considered
from the beginning.
• And it will
easily be able to answer the question any Funder may have, because the program
will have been crafted with those questions at its core. |
|
Funders
|
|
Practical Examples
for Creating and Measuring Real Outcomes:
|
As we noted above, for our communities to
achieve significant outcomes, it will require not just Provider organizations,
but EVERYONE to take a step outside his/her comfort zone - stepping outside
“the way we’ve always done it.” And that means it will take some
changes in the way Funders perceive their own work as well.
One of our favorite activities is convening the
Funders in a community, engaging them to consider the following question:
What will it take to
create an incredible future for the community you serve?
During these invigorating conversations, the
same issues surface repeatedly. And it is both interesting and disheartening to
see how many of those conversations begin with those Funders blaming Provider
organizations for not accomplishing enough.
As we work to shift the conversation from blame
to possibility, here are some questions we encourage the Funders we work with
to consider:
1) Consider the
outcomes you want to see from your own funding, and then plan your grant
efforts around those outcomes.
What change do you want to see for the dollars
you are investing? Can you re-create your grant program to aim at that
change?
Example: A health-care Funder who
realized, after years of funding clinics and other “health-related”
programs, that what they were really funding was “sick-care.” They
re-worked their grant process to fund only those programs that were aiming at
creating more lasting community health.
2) Consider who
you can work with, and what other resources you can leverage, to create those
outcomes.
Just as you might tell a Provider, it
doesn’t necessarily take more money to create more significant outcomes;
it takes identifying the array of resources you have to work with, that have
been heretofore ignored. How might you convene the organizations in your region
in an ongoing learning community to change entrenched conditions? Are there
speakers you can bring in to address issues of particular interest in your
community - speakers who might do an advocacy workshop about those issues? Are
there initiatives you could embark upon if you collaborated with other Funders
who also want the same outcomes?
Example: A progressive Funder who cared
passionately about equity issues, created a grant process related to
“diversity,” and then convened grantees in an ongoing learning
community, to encourage them to share the wisdom they all gained, thus
extending that learning even farther. Another proactive Funder, hoping to
create significant change among the hospitals in their community, offered
funding and other benefits to assist in a hospital-related project, but only
if all the hospitals in the community worked together on that initiative.
If even one failed to engage with the effort, the funding would disappear. Talk
about peer pressure!
3) Consider
researching (or funding research for) evaluation and measurement approaches
that can begin to evaluate the long term / transformational change you want to
see.
How can you use the steps noted above for
Providers, to begin determining the indicators you would want to see? What
other tools are available for measuring those results?
There are a number of existing evaluation tools,
and more being developed all the time, that are beginning to address the
evaluation of long term, transformational change.
If you do not know the answer when a grantee
asks, “How can we measure change?” and you cannot (for whatever
reason) gain that knowledge, then consider dropping the question entirely, as
it is frustrating both your grantees and yourselves, and providing little
benefit to the community.
But if you do not want to stop asking the
question, and you indeed want to see better measurement of more effective
programs, then learn all you can about some of the remarkable work being done
in this area. You will find it overwhelmingly appreciated by your grantees, and
you will be taking a huge step towards enhancing their ability to create
significant change in their communities.
Example: A leadership program wanted to
measure whether the program was having the transformational effect it wished to
have on its participants. By working with a major foundation and a group of
forward-thinking evaluators, a methodology was developed that not only took
into consideration the spectrum of change that might occur (short term, ongoing
learning, true transformation), but the extent of the impact of that
transformation (Was it just a change for that individual, or did that change
ripple to others - his/her employer, family, etc.?)
Funders have, in many cases, become accustomed
to thinking that their only job is to provide money - that their hands are tied
when it comes to creating more significant outcomes, and that all they can do
is try to force the hands of those who do that on-the-ground work. The more
Funders see that their greatest potential lies not in their dollars, but in the
combination of dollars and wisdom and other resources they have, all leveraged
together, the more Funders will be able to more proactively create the change
they want to see in their communities.
We have been encouraged by the number of Funders
who have contacted us, asking how they can focus their own work more on
longer-term outcomes, and we look forward to reporting more about those efforts
as time goes on. |
|
Nonprofit
Resource Centers |
|
Practical Examples
for Creating and Measuring Real Outcomes:
|
There are few communities in North America who
do not have some form of Nonprofit Resource Center - an organization dedicated
to enhancing the resources / improving the management of the organizations in
their community / region. Typically, Nonprofit Resource Centers put on
workshops, offer consulting, and provide other services to strengthen the
organizations in their region.
If we were asked, “Of all aspects of the
sector, where do you find the most frustration about outcomes?” we would
likely say we find it in the leaders of Nonprofit Resource Centers. Time and
again we hear stories of workshop after workshop about measuring outcomes, to
virtually no effect. When it comes to creating significant outcomes, truly
there is no one at wits end like the community’s teachers - the Nonprofit
Resource Centers.
As we work to share the possibilities we see for
creating more significant outcomes in the communities we care about, here are
some questions we encourage the leaders of Nonprofit Resource Centers to
consider:
1) Consider your
own outcomes first, then re-create your programs to focus on those outcomes.
As we work with Resource Center leaders around
the country, those leaders are quick to tell us what success would look like
for them.
• They want the
organizations in their communities to be more sustainable, less worried about
money.
• They want them
to begin working more closely together, collaborating in a more genuine way.
• They want
boards to be energized.
• And in the
bigger picture, they want to create a culture of philanthropy and service in
their community.
When we ask if they are accomplishing these
things, however, we hear of their frustrations. Their programs have not been
crafted to attain those outcomes; they have instead been crafted to respond to
the needs voiced by the organizations in their community. And as we have noted
above, there is a big difference in what can be accomplished when one is aiming
for great outcomes, rather than responding to perceived need.
Our recommendation to Resource Center leaders is
therefore that they consider using similar steps to those noted above,
determining first what they want success to look like, and then working
backwards to determine how their programs can create that success.
The day-to-day pulls us down so quickly, that it
is as important for the leaders of Resource Centers as it is for anyone else to
keep your eye on that bigger prize - the change you want to create - and have a
plan for how you might accomplish and measure that. You may be surprised that
creating the plan alone is energizing!
2) Consider
rethinking the workshop model of service delivery.
As you consider the outcomes you wish to
achieve, don’t be surprised if you end up scrapping the workshop model. We
have been encouraged by the number of Resource Center leaders who realize that
one-day workshops have limited effectiveness, and who are seeking a different
way. We are further encouraged by those who are anxious to talk with us about
approaches that cost the same or even less, while creating more long-term
impact for those who participate.
In our experience, there are 2 reasons most
Resource Centers continue to do one-day workshops:
• They believe
that is all they can afford, or
• They simply do
not know other approaches.
Workshops are not the only way to provide
education to busy professionals, and they are not the most affordable. And
obviously, they are not the most effective, or we wouldn’t see such
frustration on the part of Resource Center leaders! As we continue to work with
Resource Center leaders to develop and implement more effective models for
delivering their educational programs, we will continue to write about them, to
share their learning with others in the field.
3) Consider
alternatives to the Membership Model.
By definition, the membership business model is
an exclusive one, providing service primarily for members, rather than for the
whole community. As you consider the outcomes you want to achieve, and the best
approaches for achieving those outcomes, we hope you will consider re-thinking
the Membership Model.
We do know that Resource Centers are hard to
fund. But truth be told, every organization we have ever worked with feels it
is the hardest type of organization to fund - whether their work is that of a
Nonprofit Resource Center, a recovery organization, or an arts group. Finding
sustainable models does not have to conflict with providing the very highest
level of outcomes to the widest variety of organizations.
Again, we hope the steps above help with your
thinking in this process (especially Step 3 above - noting all the resources
you can bring to bear). You may also want to consider the articles linked here
as a way of re-thinking the funding of your Resource Center. http://www.help4nonprofits.com/H4NP.htm#Fundraising
4) Consider
changing your mission statement.
While it seems a simple thing, words are
powerful. Instead of stating only what you do, consider stating why you do it.
“Our mission is to create an incredible future for our community, by
strengthening the organizations that work to make that change
possible.”
We appreciate that the job of Executive Director
of a Nonprofit Resource Center is often one of the most tiring and frustrating
jobs there is. We are encouraged by the number of Resource Centers that have
expressed interest in moving to the next level of creating more significant
outcomes themselves. And because Nonprofit Resource Centers are all about
learning to do things well, we look forward to these paragraphs being the first
to eventually be deleted from this article, due to their no longer being
applicable! |
Reaching for outcomes does not
have to be a jargon-laden path sought only by those who need to fill in the
line on a grant application. Reaching for outcomes is reaching for our highest
potential - our greatest possibilities for creating amazing places to live, not
just for those we serve today, but for their children, and their
children’s children.
We are creating the future, right now. We hope you
will aim high.
* Our work
with Rick Miller and Kids at Hope has shown us that “at risk” is a
horrible way to describe anyone, and especially a child. Kids at Hope has
turned the definition on its ear, and has made significant outcomes possible
for many many children. Hope and positive approaches are powerful tools, and we
encourage you to check out Kids at Hope (www.KidsAtHope.org)
|
Home