Patrick Germain

Patrick Germain

This is the second of a three part series on how internal evaluators can build their organization’s evaluation capacity and is based on a talk at Eval 13 by the same name.  Last post, I wrote about starting from scratch when you first begin evaluation capacity building efforts.

Jim Collins, in his seminal work of business management Good to Great, talks about the ‘flywheel effect’.  If you aren’t familiar with it, take a few minutes to read this or better yet, buy his book.  Sometimes in the early days of building evaluation capacity, it can feel like you are trying to push a building up the block, and it isn’t until a year or two in that you look back and realize you have actually gotten somewhere!  But how do we create self-sustaining momentum around evaluation capacity?  I break it down into two buckets: engaging people, and engaging systems.  This post will be about engaging people, the next one will be about engaging systems.

Good evaluators have empathy and are able to understand the motivations that each stakeholder brings to the table.  Here is what I have found to be successful in engaging people:

  • Get their attention with jolts of reality:  One program director once asserted that the outcomes in a program were worse than before because the clients were getting younger.  I ran the numbers and discovered that, in fact, the clients were getting older!  When I revealed this in a meeting, it made her, her boss, and the CEO snap to attention.  Sometimes a carefully managed jolt of reality gets people to realize their assumptions might be misplaced and listen a little more closely to what you have to say.
  • Build motivation, both extrinsic and intrinsic: First, convince them that evaluation is not only necessary and responsible, it is heroic!  There is courage in being honest about the effect of the program, there is dedication in providing the best services they can, there is love in the act of doing math (something many people find distasteful) to help people they care about. Secondly, get their boss to tell them they have to do it.  That should help some, too.
  • Get short term and impactful wins: Do a quick analysis on a question that is vexing the organization.  While that may not be a full-fledged summative evaluation, the staff probably doesn’t know that.  And pitching in and helping someone figure out one of their problems is a nice way to make a friend.
  • Get evaluation talked about:  I made the CEO put evaluation on every agenda I could.  In my regular meetings with all the managers and directors, I put evaluation on the agenda.  I made the senior managers talk about evaluation with their direct reports.  I stood next to the water cooler and talked to everyone I could about evaluation.  Yes, I developed a slightly awkward reputation, but I also got the organization talking about evaluation.
  • Use Utilization Focused evaluation techniques: I won’t go into all of this now, but if you aren’t familiar with utilization focused evaluation, start doing some research.  But it’s a lot about engaging stakeholders appropriately and meaningfully and focusing constantly on how they will be using the evaluation to make decisions.

This won’t get everyone to come around to your point of view, but it will probably get enough people to get the ball rolling.  After that, it’s all about persistence and building systems to support them, which I will talk about in the next post.

What strategies have you found to work well in engaging non-evaluators in evaluative activities?  What hasn’t worked?

Patrick Germain is the Director of Strategy and Evaluation at Project Renewal, a large homeless services organization in New York City and is the President of the New York Consortium of Evaluators, the local AEA affiliate.