Thinking Outside the Box: More Ideas for How to Make Evaluations More Useful for Foundation Strategy and Practice

Oblique Strategy Prompt

Response(s)

Go slowly all the way around the outside

·   Look at the history of the strategy; how it was set or how a portfolio was developed; and think about history of evaluation and think about where they might intersect

·   Have evaluation partners interview strategy consultants before they are selected; or have strategy consultants critique evaluation.

·   Intentionally create paced moments of reflection that set us up/frontload the knowledge we need to make fast decisions.

·   People who are external to strategy process: how can we make it clear how strategy was decided or improved upon, or how evaluation will inform strategy

Do the words need changing?

·   Both evaluation and strategy are big, ambiguous and loaded terms: how do you align evidence and decision-making/reflection and action/ move away from those to something more clearly actionable

·   Think about the Waze app: very simple—this is our destination, this is where we are, this is the path that seems sensible now—but there’s constant updating based on what’s happening 

Question the heroic approach

·   POs often see external evaluators as bunch of ivory-tower sitters who crunch magical numbers and turn out reports; how can we have them seen as real people who can help you in your learning etc.

Do something boring

·   In the front end, could unpack what you think the strategy is a bit more, e.g., sentence diagramming but for a strategy—break something down in excruciating detail; theory of change often thinks about rosy picture and focuses more on outcomes than actors and assumptions; go really deep on understanding strategy and where you can focus evaluation

·   If you diagram strategy: could you identify what could be addressed with secondary data? Get a bunch of questions out the way with that, and see that as a valuable part of the evaluation process

·   If CEOs knew evaluation that’d be great; and if evaluators knew strategy that’d be great!

Your mistake was a hidden intention

·   Hidden agendas: being transparent among different stakeholders about purpose, such as the ‘what’ of the evaluation, how do we want to partner, minefields and obstacles.

Try faking it

·   Long-term evaluations as both a co-funder and doing some of the evaluation: what do you do when you can’t do an evaluation as planned? What do you tell the funder about this (or all eight funders)? How can you be honest and yet still have an evaluation be a good investment—because there are likely still things that are really important to talk about? How do you manage it if the honesty isn’t reciprocal?

Retrace your steps

·   Commissioning evaluation, designing it, doing it etc. creates a record and that’s a record we can review and be critical about: evaluating our evaluations

·   Not a lot of reference to empirical research about what makes evaluation useful- opportunity there to bring in more of an evidence-based perspective even to this conversation

What would you closest friend do?

·   Museum Camp!

·   Different themes clashing; ideate wild ideas that initially you’d be skeptical about; orientation to risk and openness, and trying new things

·   Find a way to create community around this question; make it a shared experience; and make it fun (to have a good process, with the goal in mind, rather than just for the sake of it)

Decorate

·   Data visualization and making learning visible: is there a trade-off between being pretty and being useful?

·   When do you decorate? When you launch, when you’re celebrating- are there other opportunities?   

Steal solutions

·   Is there anything we could steal from other sectors? Are there organizations who are doing this well and at scale? Processes, tools, trainings that we can steal?

Courage!

·   Struggle to enact best practices due to risk, willingness to be candid, willingness to take a hard look at strategy

·   Would take courage just to distill and enact some 101 measures—if you could change some of that, would change relationships with grantees, with consultants etc.

Work at a different speed

·   International development: long and short-term strategies (e.g. development vs. emergency)—and the evaluation frame would also be very different; and the feedback loops of when you’re actually getting the information would be too.

·   Course adjustment: looking at the big picture and then backing into our outcomes; they’re very linear right now—how do you connect both the columns to each other, but then also to strategy in different stages?

Remove middle and extend edges

·   If you didn’t worry about reports of findings and thought about initiation and post-evaluation—what if the first product was a scope of work? What if final product was basically an implementation plan, concretizing the actions that come out of evaluation?

·   If you think about who is usually at the center of the conversation, it’s usually foundation staff; could we answer questions constituents have, would it provide different answers and inform the work differently?