Philanthropedia Blog

Responding to GiveWell’s Analysis of our Microfinance Report – Part 2/2

January 4th, 2010 by Deyan Leave a reply »

In the first part of this blog post, I gave some perspective around our progress so far and plans for the future as a first step towards responding to GiveWell’s excellent analysis on our microfinance report. In this second part, I will attempt to address the specific concerns that Holden raised. If you are short on time, you can scroll to the bottom of each paragraph, where I have summarized our plans for improvement in the future (where it made sense).

Issue 1: the definition of “expert” is unclear

I believe that Holden is right on the spot criticizing us that we have not done a good job of defining an expert publicly. The very fact that he had to search in multiple places to find something so fundamental already speaks volumes about the urgent need to bring more transparency and reorganize the way we present information. We intend to do that as soon as possible.

The more substantial issue that GiveWell is bringing up is basically how we came up with these particular experts. Let me provide some details on our process here (a better explanation will most certainly become a central part of our webpage).

When we decide to expand into a new social cause, our first step is to prepare by researching and mapping out the space. In doing so, we identify – to the best of our abilities – all relevant experts in a given field. We list all foundations, nonprofits, and academic institutions and then find the experts’ contact information in these organizations. In addition, in some social causes we include journalists, policy makers, researchers, and other types of experts. The end result is a list of hundreds of experts who we then invite to participate in our surveys.

Of course, not every invited expert participates today. And of those who participate, we don’t necessarily include all responses from all experts because we do additional sorting of experts during the data collection process. What is more, not all experts are “equal” in terms of their expertise or experience. And some – most notably foundation professionals and academics – bring disproportionate value to our research. However, we strongly believe in our current inclusive model, which also invites nonprofit executives and senior staff to participate for several reasons. First, this group is obviously much bigger than researchers or funders. Not including them would be a missed opportunity to capture knowledge from those “working in the trenches.” Second, getting nonprofit executives involved opens the door to fix the broken feedback loops by using expert reviews as a starting point. For example, many nonprofits have expressed interest in being able to respond to the reviews (a functionality that we will be developing soon) – which would get us one step closer to increased transparency. The reason why nonprofit executives form such a large part of the final expert network is that there are simply more of them, compared to academics or foundation professionals. And to address the issue of the “others” category, we plan to develop more specific categories.

So, overall, I completely agree with GiveWell that we need to disclose substantially more information to build more trust among donors. We have internally identified the same issue and are working to redesign our webpage to address the concern.

Issue 2: who are the experts?

I have already mentioned the challenge of answering the question of who our experts are because of our still incomplete research in part 1 of this blog post. Our goal is to continuously get more experts “on the record” and include their bios. Based on our experience so far, experts are excited to participate in the research and I am confident that we will end up featuring more than the 31 people we have today in microfinance.

On the more specific issue of featuring microfinance skeptics, I want to make two points. First, both microfinance skeptics and proponents have recently been announcing results from different research projects that point in different directions with respect to the effects of microfinance. Regardless of who is right (which is often somewhat subjective), one obvious limitation of any crowdsourced approach (such as ours) is that it takes time to change the perceptions and opinions of a group of people. Second, our research actually features a number of microfinance skeptics at the urging of Dean Karlan, who has kindly provided us with critical feedback, served as a sounding board, and helped us connect with more researchers in the space.

Nevertheless, GiveWell is making a terrific point here: our ability to build trust and engage donors depends on being able to attract and feature credible experts. It is our intent to make substantial progress by being selective about our experts, asking them for more participation, and continuing to build large and representative networks of experts. In addition, we intend to provide better social cause summaries, which highlight the core issues as well as most important debates.

Issue 3: the recommended charities

GiveWell makes an interesting observation that the “list reads like a who’s who in large U.S. microfinance charities.” This statement is quite surprising to me, because our own experience when testing the expert mutual fund with donors is quite different. Perhaps to insiders, the list reads like the who’s who, but to the average donor that is certainly not the case. In fact, most people’s knowledge is limited to Kiva.

In some industries, it is perfectly fine for the “average” individual to not know the “who’s who.” But in most social causes in philanthropy that is not the case, because individuals provide the vast majority of funding – and they do it on the basis of very little good information. So for our national social causes, the lists might read like the who’s who to experts, but that is certainly not the case for our target user: the individual donor. And I would personally argue that perhaps an even bigger opportunity is our work on a local level – highlighting top nonprofits in different communities, which inspires people to join forces, support good organizations, and work towards solving the important societal problems of the day.

GiveWell also criticizes the expert-recommended charities and I personally certainly respect their opinion as microfinance skeptics. However, the Philanthropedia vision of accelerating the pace of social change has a different approach. We know that no organization out there is perfect – that is why we identify both strengths and areas for improvement. Our goal is to engage experts, donors, and nonprofits to collectively find and fund the best organizations that are solving some of the world’s toughest social problems. I believe that this can only happen through an inclusive process of continuous improvement, focused on bringing more transparency and demanding more impact from these nonprofits.

Issue 4: expert quotes / reviews quality

GiveWell also uses the Kiva expert quotes to analyze the usefulness of our expert reviews. The main concern that Holden identifies is whether experts are indeed doing a good job of helping donors increase their impact. This is a concern that I personally share and that is at the core of our push to increase the quality of our research in 2010 that I wrote about in Part 1. Some of the upcoming changes that we will be making in our methodology and survey design should help us get higher quality reviews. Further, as we gain more traction and educate more experts about our mission, this problem will gradually be reduced.

However, I should note that we continuously evaluate all aspects of Philanthropedia and look for ways to improve. We have performed research on our expert reviews in Bay Area homelessness and climate change (both with foundation professionals and donors) and the feedback we got was that the comments capture the essence of what the different nonprofit’s strengths and weaknesses are. For example, one of Kiva’s big accomplishments is that it has been able to capture the attention and imagination of the individual. This is a fascinating and rather unique achievement that our experts have legitimately pointed out. So we feel that the starting point is not as low as GiveWell might portray.

In summary, I believe that our expert reviews can be further improved by better surveying and educating experts on the importance of sharing their opinion. This issue is front and center for us as we look to expand into more social causes in 2010.

Issue 5: reinforcing bad incentives

Holden also hypothesizes that our methodology encourages experts not to disclose their names, and encourages charities to do whatever to get them recommended by Philanthropedia’s set of experts. This is the main issue identified by GiveWell that I disagree with. In my opinion, Philanthropedia has the opposite effect from the one outlined in the GiveWell post. First, we are increasing the transparency of experts. Even if only 31 experts agreed to publish their bios and stand behind the results, that is exactly 31 more than before. And our plans are to further push the bar, both in terms of what experts disclose and in terms of how many experts participate.

Second, Philanthropedia is getting areas for improvement of different charities “on the record” and thus inspiring them to participate in the discussion, increase their transparency, and further improve. The idea that charities will be able to manipulate the results is certainly plausible, but there are a number of reasons why I think it is highly unlikely:

  • Experts have high morals and ethics – foundation professionals, nonprofit executives, and academics operate in an environment characterized by conflicts of interest. For example, researchers might need access and data to assess nonprofits’ impact, or foundation professionals might receive multiple grant proposals from competing nonprofits, while collaborating with them on other projects. In general, in our dealings with experts, we have found them to have very high personal and professional integrity.
  • The Philanthropedia methodology relies on a high number of experts, which makes gaming the system challenging.
  • We currently have a number of internal checks that makes manipulating the results difficult. Moreover, we will be adding other policies such as random checks, more due diligence, better conflict of interest management, etc. to further reduce such risks.
  • Of course, I fully expect that if someone is dedicated enough, they will be able to game the system. That’s why we plan on very aggressively punishing any offenders, who violate the our trust, as well as the trust of the community.

In summary, I personally disagree that we are promoting the wrong incentives. I believe that the public discussions, engagement of new donors, and reactions that nonprofits have speak to that as well.

Conclusion

In conclusion, let me again thank GiveWell for their time and analysis. The rest of the Philanthropedia team and I very much agree with the vast majority of the issues that were identified, as well as the suggestions for improvement. We absolutely plan on raising the bar in terms of transparency and quality in order to be able to gain the trust of both donors and experts. Stay tuned for our reorganized webpage that addresses some of the concerns that GiveWell brought up.

GiveWell, in addition to many others, have kindly said that they see a lot of potential in our model. We at Philanthropedia certainly aspire to live up to that expectation, and with more analyses and constructive feedback, we have a real shot of getting there. So thanks again for your leadership Holden and Elie and for helping us get to the next level.

68 comments

  1. Therefore, renting is one of the best solutions for their housing
    needs. 3G data card will allow to stay connected with the whole
    world while moving from one place to another. In a consumer service industry where
    each member’s professional integrity is collectively criticized by mainstream media
    pundits at the start each new moving season, Interstate is one of the good guys.

    Feel free to visit my blog post – best food storage companies

  2. This type of lingerie shapes, tones, smoothes and sometimes even lifts
    the problem areas in the body. These exercises were recommended for them to tone
    the vaginal muscles in order to reshaping them and to strengthen the weakened bladder.

    They are application software in which generates the
    sheets where sprites are organized.

    my blog – gold’s gym men’s clothing cheap

  3. My grandmother always advised me to use onion juice on my hair,
    but I thought it was a crazy idea. They use a multi-therapeutic approach, trying to
    win the war on hair loss by using every possible angle.
    Regrow Hair – Hair Regrowth – It is easy to Regrow Hair
    and even make it grow faster with Procerin if
    your body has not stopped Growing New Hair and follicles altogether.

    Have a look at my blog igrow seiko watches for men

  4. Read on to find the vocabulary you will need and links to
    sites with house plans. Will any walls, especially load
    bearing walls, have to be knocked down. Here are the step-by-step
    instructions for refinishing old appliances in stainless steel paint, a home improvement project for women with old appliances
    but a modern sense of style.

    Feel free to surf to my blog post – plumbing moving companies pompano beach

  5. I am regular visitor, how are you everybody? This piece of writing posted at this
    web site is really fastidious.

    My webpage :: i need a diet To Gain weight

  6. You really make it seem so easy with your presentation but I find this matter to be really
    something that I think I would never understand. It seems too complex
    and extremely broad for me. I am looking forward for your next post, I will try to get the
    hang of it!

    my weblog – Natural Garcinia Trim Huawei Ascend Plus Reviews

  7. patch.com says:

    Does everybody else experience an error once this website page loads utilizing the current Chromium?
    This web site for no reason used to do this in the past.
    Anyways, cool post!

  8. I know this if off topic but I’m looking into starting my own blog and was wondering what
    all is needed to get set up? I’m assuming having a blog like
    yours would cost a pretty penny? I’m not very web
    smart so I’m not 100% sure. Any suggestions or advice would be
    greatly appreciated. Kudos

    Here is my webpage … garcinia acai slim free trial

  9. I was more than happy to find this page. I need to to
    thank you for your time just for this wonderful read!! I definitely
    loved every part of it and I have you saved to fav to check out new things on your website.

    Here is my weblog :: garcinia mathews ultra max reviews

  10. As all of us know that time is money so they take money and save your precious time and the hassles of shifting.
    Ask questions to get an accurate assessment of the move and cost.
    Their outstanding customer relations generate a very happy moving experience.

    My web blog olympia moving and storage company

  11. The ball at the bottom of the vacuum houses all of the engine components and the ball
    makes it easy to turn and maneuver it with ease.
    These types of couplings are a great asset and can really help, get the job done.
    A professional plumber must complete the essential training and
    education to be able to complete the wide-range of plumbing jobs and ask also
    for the years of experience that they have in the field.

    My blog :: water heater pompano beach fl

  12. Another useful outlet is to join a forum where you can ask
    question and gather information on certain RV from those that already own the
    particular motorhome. Many times I have tried to avoid the most popular swims for this reason because this
    is where the fish will often feed most carefully and be
    most hard to catch. Use your guam tar on the net trap and go at
    least two squares away from it.

    Feel free to surf to my site :: chest freezer bait tanks

  13. Comfort comes in the form of the beautifully upholstered reclinable
    deck seats and bow seats, and a drive. With the high
    temperature sensitivity of some perishable goods, it’s easy to understand why so many companies rely on vacuum insulated panels such a Thermo – Cor to ensure the
    quality of their products. Players doing solo or non-healing players benefit most from this profession as they can replenish themselves with food.

    Feel free to visit my web page :: marine kitchenaid refrigerator drawers

  14. The more one is exposed to the sun the more one can reduce the SPF factor of
    the lotion at the risk of the skin becoming dried out, leathery and full of wrinkles.
    Surgical treatment for Gynecomastia is considered
    the fastest and most effective way to reshape the male breast.

    IMPORTANT: Make sure you are purchasing UV protective clothing
    that is put together and manufactured with strict AAD guidelines, you don.

    Here is my blog … best salon fake tan

  15. Nice blog here! Also your website loads up very fast! What
    web host are you using? Can I get your affiliate link to your host?

    I wish my web site loaded up as fast as yours lol

    Here is my blog: garcinia max burn diet pills

  16. loss goals says:

    I read this post fully regarding the difference of hottest and earlier technologies, it’s awesome article.

    Review my web page … loss goals

Leave a Reply


Philanthropedia is a registered 501(c)3 organization. All of your donations are 100% tax-deductible.