GiveWell Board member Tim Ogden asks, is Kiva reporting phantom payments?

It’s hard to say for sure, but it seems that the answers is either (a) yes, they are or (b) no, but borrowers listed on Kiva’s website are not representative of the average borrowers visiting that MFI.

The table below shows default rates for Kiva’s largest field partners (in terms of total loans provided through Kiva) and compares the default rates reported on Kiva to default rates reported for those same institutions on MixMarket, a site devoted to sharing data on microfinance.

Data from the two sources aren’t perfectly compatible but the broad story is clear: Kiva’s partners have higher repayment rates on their Kiva-listed loans than on their portfolio overall, as listed on MixMarket. This could come about in a couple ways:

• The borrowers that partners list on Kiva could be substantially more likely to repay than “other” borrowers. If Kiva loans are effectively fungible between partners (as the recent discussions imply), this would mean that donors are likely “effectively” funding clients who are less likely to repay than those in the profiles they’re scanning.
• Kiva’s partners could be “covering for” their clients to keep their Kiva-listed repayment rates high.

(Kiva data comes from a superb site, Kiva Data, a site stating that it is unaffiliated with Kiva as well as the partners’ pages on the Kiva website itself. MixMarket data comes from the MixMarket website. For definitions of items in the table, see the notes section below it.

It’s worth noting that this analysis is only possible because Kiva makes its data available and due to the efforts of those at Kiva Data and MixMarket who have created extremely useful information resources. Similar analysis would be impossible for nearly every other charity I can think of.)

Notes on the data:

• The data in the table above comes from Kiva’s 10 largest partners (in terms of number of loans made to date) who provided full MixMarket data for 2008. 2 of Kiva’s 10 largest partners doesn’t have this data accessible at the moment. AMK, Kiva’s 7th largest partner, reported no outstanding loans for 2008 on MixMarket and ADMIC, Kiva’s 9th largest partner, has not reported any data for 2008 yet.
• p refers to the probability that, if the MFI’s Kiva borrowers were equally likely to default as its borrowers in general, the MFI would happen to have a 0% default rate for its Kiva loans. (Note that this calculation assumes that the time periods and the two definitions of “default rate” are comparable.) Low values mean it is extremely unlikely that this phenomenon would arise by chance, implying instead that the Kiva borrowers are systematically better than the normal borrowers, or that defaults are simply not being reported to/by Kiva.
• The data on MixMarket is self-reported. Nevertheless, Mix is a respected source, and we think it’s reasonable to rely on it.
• The date-ranges used for the Kiva data and Mix data above don’t sync up perfectly. Mix reports data for each calendar year. Kiva reports all-time data for each of its partners, some of whom joined in the middle of a calendar year. We’ve roughly adjusted for this by ignoring a Kiva partner’s first partial year and weighting the Mix data for the remaining years by the number of loans outstanding each year.
• MixMarket doesn’t use the term “default rate.” Instead, we’ve used “Write-off ratio %,” defined as Write Offs / Loan Portfolio, gross, average. We’d prefer to also look at Portfolio at Risk figures. However, Kiva only reports delinquencies for loans currently outstanding and therefore can’t be matched well to Mix data.
• MixMarket reports data as a % of the portfolio’s dollar value. Kiva reports data as a % of loans made and value loaned. Given that Kiva reports 0% default rates and Mix reports >0% default rates, there’s clearly some inconsistency. Nevertheless, without information regarding the size of the loans defaulted on at Kiva, we can’t guess at the magnitude of the inconsistency.
• All data used in this table is available in an Excel file. If you’re interested see Kiva major partners data (XLS). This file also contains links directly to Kiva’s partners page both on Kiva and MixMarket.