# Thoughts on the end of Hewlett’s Nonprofit Marketplace Initiative

Note: we sent a pre-publication draft of this post to multiple people who had been involved in the Hewlett program discussed here. A response from the Hewlett Foundation is available in the comments of this post; a response from Jacob Harold is available on the GuideStar blog.

Last April, the Chronicle of Philanthropy covered the decision by the William and Flora Hewlett Foundation to end its Nonprofit Marketplace Initiative, which in 2008 was the source of GiveWell’s first grant from a foundation, and has continued to be a source of substantial support for GiveWell’s operations in the years since. The Hewlett Foundation has been unusually transparent about the thinking behind its decision, and we have unusual context on the program as one of its grantees, so we find it worthwhile to reflect on this episode – how we perceived the Nonprofit Marketplace Initiative, its strengths and weaknesses, and the decision to end it.

The Nonprofit Marketplace Initiative aimed to improve the giving of individual donors. Hewlett states, “This Initiative’s goal was that by 2015, ten percent of individual philanthropic donations in the US (or $20 billion), would be influenced by meaningful, high-quality information about nonprofit organizations’ performance.” Grantees included GiveWell, GuideStar, Charity Navigator, Philanthropedia and Great Nonprofits. In short: • We believe that Hewlett’s philanthropy program was a strong use of philanthropic funds. The program is reported to have spent a total of$12 million over 8 years, and we think its impact on GiveWell alone will likely ultimately be responsible for enough influence on donations to easily justify that expenditure.
• We believe that ending this program may have been the right decision. With that said, we disagree with the specific reasoning Hewlett has given, for the same reason that we disagreed with its strategic plan while the program was running. We believe that Hewlett’s goal of influencing 10% of donors was unrealistic and unnecessary, at least over the time frame in question. We believe the disagreement may reflect a broader difference in how we see the yardstick by which a philanthropic program ought to be evaluated.
• We are very positive on how Hewlett ended the program. Great care was taken to end it in a way that gave grantees ample advance notice and aimed to avoid disruptive transitions. We also applaud Hewlett’s decision to publish its reasoning in ending the program and invite a public discussion, and we broadly feel that Hewlett is delivering on its stated intent to become a highly transparent grantmaker.

Our experience with the program

In 2008, Bill Meehan introduced us to Jacob Harold, who was then the Program Officer for Hewlett’s Nonprofit Marketplace Initiative program. Jacob met with us several times, getting to know us and the project. Late in 2008, we were invited to submit a proposal and were awarded a one-year, $100,000 grant. This grant was crucial for us. At the time, we had little to no name recognition, a major mistake on our record, and uncertainty about whether we’d be able to raise enough to continue operating. We were in the midst of a change of direction, after disappointing results from our first attempt at high-intensity outreach. We had determined that we needed to take a longer view and focus on research quality for the time being – and it was thanks to the support of Hewlett, among others, that we felt it was possible to do so. We benefited both from Hewlett’s financial support (which helped answer crucial questions about whether we’d be able to fund our plans at the time) and from Hewlett’s brand (being able to say we were a Hewlett grantee substantially improved our credibility and appeal in the eyes of many, something Hewlett was cognizant of). Over the years, we continued to meet periodically with Jacob and to periodically submit grant proposals. For the most part, Hewlett continued to fund us at the level of$100,000 per year (there was one year where the support temporarily dropped to $60,000). As our audience and budget grew, this support became a smaller part of our revenue and became less crucial to us, but it remained quite valuable. Hewlett’s support reduced the amount of time we had to spend fundraising and worrying about sustainability, and increased the amount of time spent on core activities. In addition to supporting us financially, Hewlett sought to integrate our work into its own vision for the “nonprofit marketplace.” Jacob encouraged us to attend convenings with other groups working on helping individual donors give effectively, such as Charity Navigator, GuideStar, Philanthropedia and Great Nonprofits (and we generally did so). He also discussed his vision for how impact would be achieved, and particularly emphasized the importance of working with portals and aggregators (such as GuideStar, where he now serves as CEO) that could pull together information from many different kinds of resources. He encouraged us to build an API in order to make aggregation easier, and saw aggregation as a more promising path than building our own website, brand and audience. We disagreed with him on some of these points. We felt that his vision was overly specific, overly focused on reaching the “average” donor, and was under-emphasizing the promise of different organizations targeting different audiences in different ways. When the Hewlett-funded Money for Good study came out, we publicly disagreed with the common interpretation, and argued that the most promising path for nonprofit evaluation groups is to target passionate niche audiences rather than focusing on the unrealistic (as both we and Money for Good saw it) goal of influencing 10%+ of all U.S. giving However, we never found Jacob or anyone else at Hewlett to be pushing its vision on us hard enough to cause problems. We certainly weighed Jacob’s encouragement when attending convenings and working on a partnership with GuideStar, but we were comfortable with the cost-benefit tradeoffs involved in these activities and didn’t undertake them solely to please a funder. We particularly valued some of the opportunities to get to know other organizations in our space. We didn’t build an API, and Hewlett didn’t pressure us to do so (its support continued). All in all, our general feeling was that Hewlett was accomplishing substantial good via its relatively reliable, unrestricted funding even as its strategy was something we disagreed with. Hewlett’s reasoning for ending the program, and our take on it In a response to the Chronicle of Philanthropy, Larry Kramer (Hewlett’s current President) wrote: We launched NMI in 2006 with the objective of influencing 10% of individual donors to be more evidence-based in their giving, a goal we sought to achieve by making high-quality information available about nonprofit performance. Based on independent research and evaluation, we concluded we were not going to meet that goal. And because we are committed to being transparent about our work – both successes and failures – we openly shared our reasons for ending the initiative in a video and blog post on our web site. Hewlett also states that staff transitions provided a good opportunity to reflect systematically on the initiative: between late 2012 and early 2013, Larry Kramer replaced Paul Brest as President, Fay Twersky became the first Director of the newly formed Effective Philanthropy Group, and Lindsay Louie replaced Jacob Harold in a slightly different program officer role. We believe that ending this program may have been the right decision. With that said, we disagree with the specific reasoning Hewlett has given, for the same reason that we disagreed with its strategic plan while the program was running. We believe that the goal of influencing 10% of donors was unrealistic and unnecessary, at least over the time frame in question. We believe that this is a case in which a commitment to specific quantitative targets, and a specific strategy for getting there, was premature and did not make the program better. Despite this, we believe that Hewlett succeeded in choosing an important problem to work on and in finding and funding promising groups working on the problem, and that it played a real role in the development of at least one organization (ours) that is poised to influence far more dollars than Hewlett spent on the program. For this reason, we think it would be reasonable to consider the program a success, though not necessarily something that should have been continued. In short, we feel this program was an instance of good and successful philanthropy, and that it may indeed have been time to end it, but we disagree with the way the program framed and evaluated itself and the way Hewlett justified the end of the program. How Hewlett ended the program Hewlett took great care to end the program in a way that would not be overly disruptive for grantees. We were notified well in advance of the public announcement about the program’s end; we were able to ask questions and receive helpful answers; and our two-year grant was renewed as an “exit grant.” We were told that other grantees had been treated similarly. By clearly communicating its intent to end the program and committing “exit funding,” Hewlett ensured that we would have ample time to adjust for the loss of this revenue. We also applaud Hewlett’s decision to publish its reasoning in ending the program and invite a public discussion. A note on Hewlett’s transparency Shortly after taking over as President of the Hewlett Foundation, Larry Kramer expressed his desire to further improve Hewlett’s transparency, and we think there has indeed been substantial progress. The public discussion of the end of the Nonprofit Marketplace Initiative represents some of this progress. In addition: • Hewlett’s relatively new blog is frequently updated and has given us a window into the day-to-day work and thoughts of its staff. • Hewlett recently held conference calls with open Q&A for grantees. As a result, we believe Hewlett has become one of the easiest foundations to learn about and get a feel for from the outside. We think this is quite a positive development, and may write more in the future about what we’ve learned from examining Hewlett’s output. Key takeaways Hewlett’s vision of good philanthropy, at least in this case, seems to have involved setting extraordinarily ambitious and specific goals, laying out a plan to get there, and closing the program if the goals aren’t reached. By this measure, the Nonprofit Marketplace Initiative apparently failed (though Hewlett followed its principles by closing a program falling short of its goals). Our vision for good philanthropy is that it finds problems worth working on (in terms of importance, tractability and uncrowdedness) and supports strong organizations to work on them, while ensuring that any “active” funding (restrictions, advice, requests of grantees) creates more value than it detracts. We think that specific quantitative goals are sometimes called for, but are more appropriate in domains where the background data is stronger and the course is easier to chart (as with our top charities). By our measure, we think the Nonprofit Marketplace Initiative was at least reasonably successful. Recognizing this difference in the way we think about good philanthropy will help us to better understand Hewlett’s decisions going forward, and will give us a disagreement to reflect on as we move forward with our vision. We’re glad to have examined Hewlett’s thinking on this matter, and see the chance to do so as a benefit of Hewlett’s improved commitment to transparency. A note on the role of Hewlett’s funding in our budget: Because this post discusses Hewlett’s work in an evaluative manner, we think it’s worth being clear about the support we receive so that people may take into account how this may influence our content. Hewlett has provided generous support to GiveWell since 2008. We hope that it will continue doing so even after the end of our current grant, depending on how our work and Hewlett’s evolve (our work on GiveWell Labs seems to us to be relevant to Hewlett’s work on encouraging transparency among major funders). We are currently projecting expenses of and revenues of over$1.5 million per year, and Hewlett’s support has historically been around \$100,000 per year.

• Lindsay Louie on August 5, 2014 at 12:46 pm said:

Holden, Elie & team – Thank you so much for your thoughtful post. We share GiveWell’s commitment to openness, critical reflection and dialog. In particular, we believe that sharing and listening openly are keys to gaining insights and improving our work.

The question you raised about by what yardstick a philanthropic program should be evaluated is a timely one for us. At the Hewlett Foundation, we have just recently been grappling with the question of when to set precise, quantitative goals and when a directional goal may be most appropriate. Our motto in the Effective Philanthropy Group is “purpose first” and that certainly applies to this question. Sometimes, a bold target like influencing 10% of donations can serve to inspire and galvanize support AND serve as a yardstick by which to measure progress. Other times, the goal can be so audacious as to be unrealistic and unhelpful in measuring progress or to inform future strategic direction.

In the case of NMI, in hindsight, it may well have been unrealistic to set the target of influencing 10% of individual giving with the initiative. At the same time, that goal was a helpful guidepost to us as we analyzed whether to continue the strategy. We did not have any indication that we were even directionally on track toward that goal, and thus we made the decision to end the initiative.

We have been pleased to support GiveWell since 2008 and through a key stage in your organizational development. We are also glad to know that our grant funds will influence multiples more dollars to be given more effectively through GiveWell. Thank you for walking your walk on openness and modeling for others what that can look like. We welcome ongoing dialog with you and all of our grantees.

– Lindsay Louie, Program Officer, Philanthropy Grantmaking
The William and Flora Hewlett Foundation

• Brian Walsh on August 6, 2014 at 11:25 am said:

Kudos to Holden, Jacob, and Lindsay who, through their thoughtful reflections on the Hewlett Foundation’s Nonprofit Marketplace Initiative, are walking the talk on openness and shared learning.

As a disclosure, my organization, Liquidnet, informally partnered with the Hewlett Foundation on different aspects of the NMI work and have entered into a formal partnership with them (and others) on their new efforts to increase the two-way openness in foundations. Liquidnet helped launch Markets for Good, which began as a forum for collaboration between online giving platforms (like GlobalGiving and Network for Good) and information providers (like GuideStar and GiveWell) and has become a broader conversation (in partnership with the Bill & Melinda Gates Foundation) about how the social sector can better generate, share, and use data and information to improve decision making, so that information might drive social impact.

I actually agree with each of them on separate points:
• I agree with Holden’s point that the NMI was a smart use of philanthropic resources and the way the strategy was wound down was open, fair, and thoughtful.
• I agree with Lindsay’s point that it was probably unrealistic for the NMI to set the specific goal of influencing 10% of individual donations in the US by 2015. Perhaps the NMI should have been explicit in changing its own benchmark(s) for success as the strategy evolved and the Hewlett Foundation learned more during the course of its work.
• I agree with Jacob’s point that the consumers of information on nonprofit performance are not *only* individual donors; beneficiaries, nonprofit managers, and others (e.g. researchers, policy makers, journalists, etc.) need access to high quality data and information in order to inform their own decision making.

I remain a big fan of GiveWell’s work and appreciate the influence they are having on a committed and growing community of donors, who value their thoughtful, thorough, and transparent analysis.

At the same time, I am very hopeful about Jacob’s efforts at GuideStar and the efforts of other organizations which play a critical role in the social sector’s “information infrastructure”. We may well be on our way to producing a system that reaches scale in *both* the quantity and the quality of information – and ultimately insight – provided to a broad audience. Time will tell, but through APIs and partnerships with “mainstream” platforms like Fidelity’s, Amazon, and other “private sector” distribution channels, we are indeed working towards getting the right information (high quality data on social issues, proven interventions, the capacity & performance of organizations, resource flows, outcomes, etc.) into the right hands (beneficiaries/constituents, donor of all sizes, nonprofit managers, policy makers, investors, etc.) to better inform decision making in the social sector.

Now let’s all agree to continue the work – and the open dialogue! – about how we might “maximize impact from society’s scarce resources” (in Jacob’s eloquent words).

• Holden on August 12, 2014 at 1:09 am said:

Thanks, all. We appreciate the thoughtful comments and discussion!