• My 2011 interview with SIAI representatives
• My followup conversation with Jaan Tallinn

In the first interview, SIAI implied (in my view) that it did not have room for more funding. Representatives have since implied that it does in fact have room for more funding, but haven’t indicated an interest in spelling this out for me (more).

What I’m saying is that in this case, I find the arguments for the existential risk reduction charities I’ve seen to be very uncompelling and that I can’t comfortably place something like P(SIAI will be successful) above any particular threshold. (Not even 10^-(3^^^^3).) The only reason I would place it above that threshold is out of a feeling that my brain pretty much isn’t capable of assigning that low a probability to any vaguely defined hypothesis.

Given this state of mind, it seems that it would be a mistake for me to invest, for the same reason that it’s a mistake to give in to a Pascal’s Mugging. If you have a different state of mind and feel confident that, say, P(SIAI will be successful)>0.01%, different conclusions follow.

At any rate, the concern about murky probabilities might be misplaced. Sometimes, I won’t stand behind a point-valued probability estimate, but I’ll be confident that the probability is at least a certain amount. Example: I don’t know how likely it is that the copper market will do really well in 30 years, but I’d say the odds are at least 1 in a thousand. If this is what I think, if I’m willing to bet at 1:1000 odds about some event of “known” probability, I should also be willing to bet at 1:1000 odds about the claim that the copper market will be doing really well in 30 years. Similarly, even if you can’t tell what the odds are for various existential risks, I think you can say that many of them have odds above certain low thresholds (1 in a million, 1 in a thousand, etc.). And if, given that those are the odds, we can reduce the probabilities, even by a small amount, I don’t think the mugging analogy will work out. [Brian Weatherson, a Rutgers philosopher, defends a view of this kind about probability. You can see some details of it on the first few pages of the paper “Keynes, Uncertainty, and Interest Rates”, available here: http://brian.weatherson.org/papers.shtml, under “Major Published Works”.]

I think it is important to remember here that ignoring the existential risks under discussion is also a choice; if the probabilities are higher than you think, it could be a big mistake.

On the Pascal’s Mugging point, I think it is important to remember that much of the low chances of making a difference associated with existential risk turn on the fact that we’re dealing with a collective action problem. Like voting, fighting in a just and important war, or not producing a lot of C02, the odds are that your personal action will not be decisive. However, when your action is part of a large collective and the action of the collective does well on average, it is often reasonable to cooperate. (Parfit makes related points in the section called “Five Mistakes in Moral Mathematics” in Reasons and Persons).

As an example, take voting. In some swing states, the odds that your vote is decisive in a presidential election is in the ballpark of 1 in 10 million (http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.2010.00272.x/abstract). If there were an important election going on, it would be unreasonable to argue that voting was irrational on the grounds that it is like a case of Pascal’s Mugging. Perhaps you have better things to do with your time, but it wouldn’t be a mugging. (Perhaps you wouldn’t have better things to do: if the decision would significantly affect the lives of 10 million people (which it easily could in an important election), a few minutes to vote might not be a bad idea.)

In my view, decreasing the chance of existential risk by 1% would be more important than having a presidential election turn out well (provided the election itself didn’t also significantly reduce existential risk; counting only people who already exist, this would save 70 million expected lives). By parallel reasoning, if some effort we could make would have a 1 in 10 million chance of decreasing existential risk by 1%, paying for the opportunity wouldn’t count as getting mugged. Here, though, one would only be decreasing the chance of existential risk by 1 in a billion. This suggests, but does not show, that the probabilities can be quite small before a low probability high stakes bet becomes a mugging.

The argument does not show that folks who worry about existential risk are not being mugged because the analogy is not perfect; in the case of voting, many other people are likely to cooperate (though your personal decision may have little to do with that). In the case of existential risk, few people are likely to cooperate (though here there seems to be more potential for influence). If this difference is deemed to be relevant, than existential risks folks are possibly being mugged. If it is not, then they are not being mugged (though other things may have higher expected return).

Let’s say the number of potential cooperators is not relevant in this way (provided the chance of success remains constant). If we could show that it were possible to decrease the probability of existential risk by more than 1 in a billion, that would show that we aren’t really dealing with a mugging. I can’t show this, but I do find it plausible. A lifetime of effective donations really could, I believe, decrease the odds of existential risk by 1 in a billion. If you buy all of that, we aren’t dealing with a mugging.

I think I am somewhere in between Paul and Robert on future/speculative vs. present/tangible impact. I definitely value future generations and think that projects with a substantial expected impact on survival (which would include, for example, reducing the odds of a major planet-wide catastrophe by 1%) are worth a lot of investment. But I also think there are limits to how well our rationality applies to thinking about such things. In particular, I am bothered by how non-contingent most of the arguments I hear for focusing on existential risk sound (that is, it sounds as though they’d sound equally good regardless of almost any empirical facts about our world and times as well as the specific proposals and personnel on the table). I think that projects that expect to see some short- or medium-term tangible impact are inherently more accountable, more prone to reality checks and meaningful “bets” on the underlying beliefs, and more likely to succeed.

Most of the supporters of existential risk charities I’ve engaged with do not seem, to me, to be giving enough weight to these considerations. On the other hand, most non-supporters seem overly dismissive to me.

Interesting to know that you had already read those articles. I personally do not find your views compelling but wish you well.

More specifically, my feeling and my rational analysis as well make me suspicious of preoccupations with future suffering when clear and present excessive sufferings are so neglected, and that’s why I suggest that research dealing with friendly intelligence or existential risks might become involved right now into the resolution of our present problems…

But my impression is that as a practical matter, one has to exercise special vigilance to make sure that an organization is making a positive difference even with respect to a well considered expected value calculation.

Some of my thinking on this point comes across in my post titled “Existential Risk and Public Relations.” But there’s more to say about this matter. Explaining my position will take some more posts.