ORS Impact News

The ORS Impact team has earned a global reputation for leadership, insight and innovation. We work collaboratively with our clients, bringing our distinctive expertise in planning, measurement and evaluation to their most vexing challenges.

Thinking Outside the Box: More Ideas for How to Make Evaluations More Useful for Foundation Strategy and Practice

Over the past year we’ve been working as part of an action team with the Funder & Evaluator Affinity Network (FEAN), thinking about how to strengthen the use of evaluation findings in philanthropic strategy. In a forthcoming brief, “Good Intentions Aren’t Enough: Making Evaluations More Useful for Foundation Strategy and Practice,” we suggest eight areas where focused action by external evaluators and foundation evaluation staff can help evaluation projects garner more use, uptake and relevance for foundation strategy and practice.

These areas, like “Ask questions that will produce information that can inform decisions” and “Build ownership for evaluation findings” came from the experiences of many external evaluators and foundation evaluation staff who had chosen to participate in this FEAN action team: what they had seen, experienced, or heard about that we could share back to the field in order to accelerate progress in this area.

The process we undertook to develop the piece importantly grounded these ideas in the experience and insights of evaluators and users of evaluation.  However, it also inherently places our thinking in the constraints of what we know, the current system, and our current context. To spur some breakthrough thinking, we also did an exercise with a set of oblique strategy promptsi  

The oblique strategy cards were originally created by Brian Eno, the music producer, to help musicians have creative breakthroughs.  They include non-directive prompts, like, “Go the long way around.”  Or, “What would your best friend do?” ii  

We used a set of these prompts with a group of evaluators during our action team meeting ahead of AEA in 2019.  While they didn’t lend themselves to neat focus areas and proven potential actions, we do think they could elucidate totally new and fresh ways of thinking about how to address this issue.  We wanted to share the prompts and what they elicited in hopes that they might help spur some new, creative responses from those of us in the field who might find this generative approach energizing and inspiring! 

We’d encourage you to also sit with these prompts and think of your own ways of solving the issue of how evaluations could see more use in foundation strategy and practice.  



Oblique Strategy Prompt


Go slowly all the way around the outside

·   Look at the history of the strategy; how it was set or how a portfolio was developed; and think about history of evaluation and think about where they might intersect

·   Have evaluation partners interview strategy consultants before they are selected; or have strategy consultants critique evaluation.

·   Intentionally create paced moments of reflection that set us up/frontload the knowledge we need to make fast decisions.

·   People who are external to strategy process: how can we make it clear how strategy was decided or improved upon, or how evaluation will inform strategy

Do the words need changing?

·   Both evaluation and strategy are big, ambiguous and loaded terms: how do you align evidence and decision-making/reflection and action/ move away from those to something more clearly actionable

·   Think about the Waze app: very simple—this is our destination, this is where we are, this is the path that seems sensible now—but there’s constant updating based on what’s happening 

Question the heroic approach

·   POs often see external evaluators as bunch of ivory-tower sitters who crunch magical numbers and turn out reports; how can we have them seen as real people who can help you in your learning etc.

Do something boring

·   In the front end, could unpack what you think the strategy is a bit more, e.g., sentence diagramming but for a strategy—break something down in excruciating detail; theory of change often thinks about rosy picture and focuses more on outcomes than actors and assumptions; go really deep on understanding strategy and where you can focus evaluation

·   If you diagram strategy: could you identify what could be addressed with secondary data? Get a bunch of questions out the way with that, and see that as a valuable part of the evaluation process

·   If CEOs knew evaluation that’d be great; and if evaluators knew strategy that’d be great!

Your mistake was a hidden intention

·   Hidden agendas: being transparent among different stakeholders about purpose, such as the ‘what’ of the evaluation, how do we want to partner, minefields and obstacles.

Try faking it

·   Long-term evaluations as both a co-funder and doing some of the evaluation: what do you do when you can’t do an evaluation as planned? What do you tell the funder about this (or all eight funders)? How can you be honest and yet still have an evaluation be a good investment—because there are likely still things that are really important to talk about? How do you manage it if the honesty isn’t reciprocal?

Retrace your steps

·   Commissioning evaluation, designing it, doing it etc. creates a record and that’s a record we can review and be critical about: evaluating our evaluations

·   Not a lot of reference to empirical research about what makes evaluation useful- opportunity there to bring in more of an evidence-based perspective even to this conversation

What would you closest friend do?

·   Museum Camp!

·   Different themes clashing; ideate wild ideas that initially you’d be skeptical about; orientation to risk and openness, and trying new things

·   Find a way to create community around this question; make it a shared experience; and make it fun (to have a good process, with the goal in mind, rather than just for the sake of it)


·   Data visualization and making learning visible: is there a trade-off between being pretty and being useful?

·   When do you decorate? When you launch, when you’re celebrating- are there other opportunities?   

Steal solutions

·   Is there anything we could steal from other sectors? Are there organizations who are doing this well and at scale? Processes, tools, trainings that we can steal?


·   Struggle to enact best practices due to risk, willingness to be candid, willingness to take a hard look at strategy

·   Would take courage just to distill and enact some 101 measures—if you could change some of that, would change relationships with grantees, with consultants etc.

Work at a different speed

·   International development: long and short-term strategies (e.g. development vs. emergency)—and the evaluation frame would also be very different; and the feedback loops of when you’re actually getting the information would be too.

·   Course adjustment: looking at the big picture and then backing into our outcomes; they’re very linear right now—how do you connect both the columns to each other, but then also to strategy in different stages?

Remove middle and extend edges

·   If you didn’t worry about reports of findings and thought about initiation and post-evaluation—what if the first product was a scope of work? What if final product was basically an implementation plan, concretizing the actions that come out of evaluation?

·   If you think about who is usually at the center of the conversation, it’s usually foundation staff; could we answer questions constituents have, would it provide different answers and inform the work differently?

[i] Thanks to Fay Twersky for the original introduction to oblique strategy cards.

[ii] For more information and sample prompts, see:  https://en.wikipedia.org/wiki/Oblique_Strategies

Write A Comment

Post A Comment