Counterfactual credit assignment

Introduction

Effective altruists tend to assign credit based on counterfactuals: If I do X, how much better will the world be than if I didn't do X? This is the intuition behind the idea that the work you do in your job is at least somewhat replaceable, as well as the reason to seek out do-gooding activities that aren't likely to be done without you.

Perils of adding credit

We can get into tricky issues when trying to add up counterfactual credit, though. Let me give an example. Alice and Bob find themselves in a building that contains buttons. Each person is allowed to press only one button, at which point she/he is transported elsewhere and has no further access to the buttons. Thus, Alice and Bob want to maximize the effectiveness of their button pressing. There's a green button that, when pressed once, prevents 2 chickens from enduring life on a factory farm. There's also a red button that, when pressed twice in a row, prevents 3 chickens from enduring life on a factory farm. In order to make the red button effective, both Alice and Bob have to use their button press on it.

Alice goes first. Suppose she thinks it's very likely (say 99% likely) that Bob will press the red button. That means that if she presses the red button, she'll save 3 chickens, while if she presses the green button, she'll only save 2. There's more counterfactual credit for pressing the red button, so it seems she should do that. Then, Bob sees that Alice has pressed the red button. Now he faces the same comparison: If he presses red, he saves 3 chickens, while if he presses green, he saves only 2. He should thus press red. In this process, each person computed a counterfactual value of 3 for the red button vs. 2 for the green button. Added together, this implies a value of 3+3=6 vs. 2+2=4.

Unfortunately, in terms of the actual number of saved chickens, the comparison is 3 vs. 4. Both Alice and Bob should have pressed green to save 2+2=4 chickens. This shows that individual credit assignments can't just be added together naively.

Of course, the situation here depended on what Alice thought Bob would do. If Alice thought it was extremely likely Bob would press green, her counterfactual credit would have been 2 for green vs. 0 for red. Or, if she thought Bob would switch to red if and only if she pressed red, then the comparison was 2 for herself vs. 3-2=1 for Bob's switching to red and giving up his green.

Joint decision analysis

The decision analysis becomes more clear using a payoff matrix as in game theory, except in this case both Alice and Bob, being altruists, share the same payoff, which is total chickens helped:

	Bob press red	Bob press green
Alice press red	3	2
Alice press green	2	4

Alice and Bob should coordinate to each press green. Of course, if Alice has pressed red, at that point Bob should as well.

In this example, reasoning based on individual counterfactual credit still works. Imagine that Alice was going to press red but was open to suggestions from Bob. If he convinces her to press green and then presses green himself, the value will be 4 instead of 3 if he hadn't done that, so he gets more counterfactual credit if he persuades Alice to press green and then does the same himself than if he goes along with her choice of red.

Acknowledgements

This post was inspired by comments in "The Haste Consideration," which is a concrete case where counterfactual credit assignments can get tricky.

Mr. Rogers on unconditional love

Summary: Unconditional love is an attitude we adopt and a feeling we cultivate because of its salutary effects on people.

Fred Rogers ended many episodes of Mister Rogers' Neighborhood with the reminder that "people can like you exactly as you are," an expression he learned from his grandfather.

Episode 1606 of the program featured Lady Aberlin singing to Daniel the following song:

I'm glad
You're the way you are
I'm glad
You're you
I'm glad
You can do the things that you can do
I like
How you look
I like the way
That you feel
I feel that you
Have a right to be quite pleased with you
I'm glad
You're the way you are
I think
You're fine
I'm glad
You're the way you are
The pleasure's mine
It's good
That you look the way you should
Wouldn't change you if I could
'Cause I'm happy you are you.

Do these statements mean people shouldn't bother improving themselves? If others like them as they are, is there no incentive to get better at things?

Well, it's possible that conditional love could force people to try harder in order to seek approval, but at what cost and for what benefit? I think the cost is big: If you're not certain that anyone loves you, life can seem very scary, hopeless, and pointless. And I think there are plenty of other factors motivating people to improve in areas that matter without trying to use love as another carrot and stick. When people are in a rough emotional situation, they may not even have the motivation or support to undertake self-improvement, and might either wallow in despair or seek approval in unproductive ways -- including, as the song hints, through trying to look more attractive on the outside.

There's a time and place for incentives, but love by and for another person is one domain where trying to introduce incentives does more harm than good because of the nature of human psychology. Consider how popular the theme is in Christianity that God loves you no matter what: This is a powerful idea that can transform people's lives.

I feel unconditional love for a person even at the same time that I might prefer him/her to be different. If the person is open to advice on changing, I'll suggest things, but at the same time, I feel that even if the person doesn't change, it's okay -- s/he is still a special individual whose feelings matter just the same. In my mind, unconditional love is closely tied with hedonistic utilitarianism: When I realize that an organism feels happiness and suffering, at that point I realize that the organism matters and deserves care and kindness. In this sense, you could say the only "condition" of my love is sentience.

From "Then Your Heart is Full of Love" by Josie Carey Franz and Fred Rogers (1984):

When your heart can sing another's gladness,
Then your heart is full of love.
When your heart can cry another's sadness,
Then your heart is full of love.
[...]
When your heart has room for everybody,
Then your heart is full of love.

I'll close with another Fred Rogers song, possibly my favorite. It hints at this idea that the other person's feelings are the reason for our love of him or her.

It's you I like,
It's not the things you wear,
It's not the way you do your hair--
But it's you I like.
The way you are right now,
The way down deep inside you--
Not the things that hide you,
[...]
I hope that you'll remember
Even when you're feeling blue
That it's you I like,
It's you yourself,
It's you, it's you I like.

Agile projects

Give feedback early; give feedback often. Especially the early part.

When it comes to writing a paper or planning a campaign or picking a cause to focus on, a little bit of feedback at the beginning is worth hundreds of micro-edits or small optimizations later on. The topic that you write about can matter more than everything else in your whole article. If you complete a research paper about something unimportant, it doesn't much matter how well-written and well-researched the piece is (unless your goal is to establish prestige as a writer or build an audience that you can then direct toward your more important essays). If you pick an inefficient activism campaign, it doesn't much matter how well you carry it out (except for getting practice, personal experience, etc.).

Most of the time, feedback won't have the dramatic effect of reorienting the entire direction of a paper or campaign, but it may have smaller impacts, such as whether the author considers a given argument or whether the campaign undertakes measurement of its impact. A stitch in time saves nine, and it's easier (both physically and cognitively) to improve something at the beginning than near the end.

So why is it that people sometimes hesitate to share drafts, ideas, plans, etc. until they're almost completed? Maybe one reason is that slow feedback is sometimes more customary, and people fear that if they share totally incomplete drafts or brainstorms, people would judge them for not being thorough and polished and not having considered such-and-such objection. If this is the case, we should work to change the culture of feedback among the people we know, to make it clear that preliminary drafts are potentially better than polished products in terms of the benefit of giving feedback per unit time.

Another reason can be that when people comment on a rough draft, the author may already know that he needs to fix most of what the reviewer points out. But this can be largely allayed if the reviewer understands the stage where the project is. You don't (usually) give sentence-level edits on a paper outline. Also, the author could sketch out the areas that he knows are incomplete so that the reviewer won't comment on those.

The title of this post comes from agile software development, which is one area where the principles I described have been well recognized.

Pain vs. suffering and animals vs. humans

People sometimes ask me whether I make a distinction between "pain" and "suffering." The answer is "yes, I do," although one reason this might not be clear is that I have the following quotation from George Orwell at the top of my page called "On the Seriousness of Suffering":

Nothing in the world was so bad as physical pain.

Katja Grace wrote a blog post based on this quote, and in the comments, I made the following clarification:

First, I don’t entirely agree with Orwell’s choice of words, but I included the quote as he wrote it for the sake of readability. In particular, as many have pointed out, what matters is not “pain” directly but “suffering,” i.e., the response that “this feels really awful and I want it to stop.” The commenters raised several examples where pain itself isn’t aversive: Pain asymbolia, masochism, people given morphine, etc., not to mention self-cutting and other things people do in order to release endorphins/opioids to make themselves feel better.
I would also omit Orwell’s word “physical,” because because mental pain can be just as bad.

Pain asymbolia is the most clear proof that pain and suffering are distinct, because unlike masochism where one can imagine that pleasure chemicals are merely outweighing pain signals, in pain asymbolia, the quale of pain itself is not aversive.

This suggests the broader question, What gives valence to qualia? I think the details of how this happens are largely unknown, but presumably there are brain processes which "paint" a suffering gloss onto experiences in the same way as certain brain processes paint a hedonic gloss onto pleasure. It's these painting operations that I count as suffering and that I want to reduce.

A related theme is the classic distinction between nociception and conscious pain. As Jane A. Smith explains in "A Question of Pain in Invertebrates":

Invertebrates, it seems, exhibit nociceptive responses analogous to those shown by vertebrates. They can detect and respond to noxious stimuli, and in some cases, these responses can be modified by opioid substances. However, in humans, at least, there is a distinction to be made between the "registering" of a noxious stimulus and the "experience" of pain. In humans, pain "may be seen as the response of the whole awake conscious organism to noxious stimuli, seated.., at the highest levels in the central nervous system, involving emotional and other psychological components" (Iggo, 1984). Experiments on decorticate mammals have shown that complex, though stereotyped, motor responses to noxious stimuli may occur in the absence of consciousness and, therefore, of pain (Iggo, 1984). Thus, it is possible that invertebrates' responses to noxious stimuli (and modifications of these responses) could be simple reflexes, occurring without the animals being aware of experiencing something unpleasant, that is, without "suffering" something akin to what humans call pain.

And from Antonio Damasio, The Feeling of What Happens, as excerpted here:

Would one or all of those neural patter[n]s of injured tissue be the same thing as knowing one had pain? And the answer is, not really. Knowing that you have pain requires something else that occurs after the neural patterns that correspond to the substrate of pain – the nociceptive signals – are displayed in the appropriate areas of the brain stem, thalamus, and cerebral cortex and generate an image of pain, a feeling of pain.

So when I ask whether insects might be able to suffer, I don't mean just whether they can react against physical injury and learn to avoid it in the future. I'm asking whether they can perceive this injury as something that is happening to them and that they want to have stopped. I agree that the jury is very much still out on this question. If it seems as though I believe otherwise, it's because I'm trying to track the expected value rather than the most likely point estimate.

Now, given that suffering is different from pain and that suffering can involve strong non-physical emotional components, does this mean animals matter less than we might think because they don't suffer in high-level mental ways?

First, it's unclear whether the claim is true that animals have substantially less sophisticated mentation, at least for "higher" animals like mammals. Animals show many of the psychopathologies that humans do and are used as models for depression when testing drugs. Elephants have death rituals. Crows appear to go sledding for fun. Marc Bekoff, Jonathan Balcombe, and other ethologists have written numerous books documenting the complex emotional lives of mammals, birds, fish, and even octopuses.

But, suppose it is true that non-human animals don't have a similar degree of psychological depth to their experiences. It's not obvious that this means they suffer less intensely. Maybe the brain applies normalization to its experiences, so that it can appropriately encode relative priorities of various drives without using excessive amounts of energy/storage. For example, say a mouse's suffering is between 0 and -10, while a human's would be between 0 and -50 due to emotional depth. However, maybe the human brain doesn't care about perfect granularity among all of the values between 0 and -50; it only needs a sufficient granularity to make the right tradeoffs, so it downplays the importance of physical pain. In other words, a physical pain that would have been -10 for the mouse might be -2 for the human, because the human has so much else to worry about. This is pure speculation, and I wouldn't rest my argument on this point, but it seems possible. This discussion also gets into philosophical issues about how we want to care about and measure emotional intensity, which lie beyond the scope of the current post.

Finally, what if animals do suffer less, even after taking account of the brain's normalization processes? Well, I guess I would ask, How much less do they suffer? I don't think it's orders of magnitude less, and if not, then the basic calculations showing that, at the margin, animal welfare takes priority over human welfare would remain. Suppose you were a chicken being scalded and drowned alive in a boiling defeathering tank. How much less bad would this experience be if you didn't have broader thoughts about the end of your life, the injustice of your situation, how much you'll miss your friends, etc.? I suspect that the raw physical pain would overwhelm these subsidiary thoughts during the moment, and even if not, I don't think the higher-level thoughts would be 10 times stronger than the raw pain.

Moreover, there are many times when humans may in fact suffer less because of their understanding of the situation. Humans enduring a bout of food poisoning can know that the agony will end after a day or two and can know that their friends and family will help them in the mean time. Animals going through the same experience may have no idea what's happening to them, whether it will end, or what will become of their lives.

The points discussed above are fascinating to ponder, and it's valuable to hear from other people which of their own experiences they've found most unpleasant. That said, we modern humans live extremely comfortable lives compared with factory-farmed or wild animals, so it isn't surprising that most of our worst memories may be of purely emotional injury. In any event, regardless of where we settle on the question of the relative magnitudes of animal and human pain, physical and psychological pain, I don't think it's likely to tip the balance of our calculations about where our dollars and hours will do the most good.

Why I prefer public conversations

Many of my richest insights into philosophy and reducing suffering have come from written conversations with friends. Some of these have been on public fora like Felicifia, while others have been in private messages and emails.

In general, I always encourage people to conduct non-sensitive discussions in a public forum, and I'm quite passionate about this recommendation. :) There are a few reasons I like public conversations so much:

1. Linkability. There are many, many occasions when I want to refer someone to a previous thread on a very similar topic, so as to avoid re-explaining how the wheel works. This is much easier with public discussions, and I also don't have to ask the participants in the private conversation for permission to forward along their writings.
2. Searchability. When discussions are indexed by search engines, they're easy to find. I can search my Gmails and Facebook conversations as well, but this is more clunky.
3. Discoverability. Conversations indexed by search engines can be found by other people with similar interests. More than half of respondents to the question, "How did you find Felicifia?" said that they stumbled upon the forum through Google, rather than through friends or inlinks. Needless to say, this multiplies the impact of whatever you write, and it helps to keep your communities from being insular, since new people continue to join and discover the insights you have to share.
4. Unexpected feedback. The world is a big place, and there are lots of really smart people with great ideas and useful experience. If you restrict your conversations to just people you already know, you're closing off the possibility of feedback from people you don't know. In some cases, comments from people you don't know may be the most useful of all, because you're least likely to have heard their ideas before. 
5. Preservation for posterity. When the content is on a public website, it'll be available as long as the site remains up and running. If Internet Archive has a chance to crawl the content, it'll be available longer; this amounts to free file backups for you.
6. Sharing with AGI. This point I include mostly for fun, but it's 5% serious. If someone builds an AGI that cares about what people think and wants to learn about ethics, helping others, altruism, etc., then it will read through the entire Internet as background material. If your conversations are online, you can play a small role in shaping the opinions of the AGI. More mundanely, in the short term, your content will be factored into aggregate statistics about what people on the web are up to -- e.g., trending articles and topics as displayed by search engines, bookmarking sites, popularity graphs, or whatever.

Of these reasons, I think #3 and #4 are most important, followed by #1. Public conversations really are a public good, and their positive externalities deserve to be kept in mind.

Bekoff and Dawkins on animal consciousness

Here is another post that I originally made on a discussion thread in the Facebook group Reducing wild-animal suffering. It's in reply to two articles: (1) "Dawkins' Dangerous Idea: We Really Don't Know If Animals Are Conscious" by Marc Bekoff, and (2) "Convincing the Unconvinced That Animal Welfare Matters" by Marian Stamp Dawkins.

----

Thanks for sharing the pieces by Bekoff and Dawkins. I admire both of those authors, and I can definitely see where both sides are coming from.

I think the most important distinction that needs to be made is between “certainty” in science and “certainty” in ethics. Dawkins is right that science should remain skeptical of animal consciousness and that we should seek out proof independent of existing assumptions. Scientists don't like to stamp a statement as "true" until they're really sure about it from lots of angles.

But while Dawkins is correct that we don't know "for certain" whether animals are conscious, this statement is misleading to many laypeople who assume that she must mean the odds are around 50%. I don't know what she thinks the odds actually are, but I would give above a 90% chance of chicken consciousness and above, say, 93% for pig consciousness. With odds like that, it's best to say that the case is proved or else the public will misunderstand. Many people are not motivated by less than absolute certainty, and I think Bekoff is right that emphasizing scientific doubt is going to hurt animals on average. (Just look at what talking about uncertainty does for the global-warming debate.)

Now, Dawkins is totally correct that we don't understand exactly why animals are conscious. Indeed, we don't even know why people are conscious. What exactly does being conscious allow you to do that you can't do if you're not conscious? As blindsight shows, you can walk and avoid objects without being conscious of them. And as Libet's famous free-will studies showed, you can decide to move your hand half a second before you become conscious of your choice. If we ourselves didn't experience our consciousness through our own minds, then we would definitely have scientific doubts about whether people are conscious, too.

There are lots of excellent studies demonstrating sophisticated, self-reflective behavior in animals that Bekoff and others take to imply consciousness, and indeed these are excellent pieces of evidence. However, they are not conclusive proof of consciousness because we can't even prove that humans are conscious using such tests at the present time. (In the future, once we really understand how consciousness works in the brain, we should be able to assess consciousness just by looking at the brain itself. But that is a long way off.)

So I think arguably the strongest reason we should believe animals are conscious is that they're close to us on the evolutionary tree, and their brain structures are remarkably similar. In "New evidence of animal consciousness" (2004), Donald R. Griffin and Gayle B. Speck note that "the search for neural correlates of consciousness has not found any consciousness-producing structure or process that is limited to human brains" (p. 1). And in "Building a neuroscience of pleasure and well-being" (2012), Kent C. Berridge and Morten L. Kringelbach comment:

Progress has been facilitated by the recognition that hedonic brain mechanisms are largely shared between humans and other mammals, allowing application of conclusions from animal studies to a better understanding of human pleasures. […] 

Some might be surprised by high similarity across species, or by substantial subcortical contributions, at least if one thinks of pleasure as uniquely human and as emerging only at the top of the brain. The neural similarity indicates an early phylogenetic appearance of neural circuits for pleasure and a conservation of those circuits, including deep brain circuits, in the elaboration of later species, including humans.

There must be dozens of other papers that could be quoted in a similar fashion. Based on this, a probability for mammal and bird consciousness as low as 50% is completely unreasonable, IMHO.

Now, what about the effort that Dawkins proposes: Making people care about animals for human-welfare reasons? If we could press a button to do this, I'd be in favor of it. But when we're parceling out our scarce resources for helping animals, I think this undertaking should go pretty low on the priority list. It's great if we can help animals in the short term in this way, but if we're going to prevent future humans from multiplying wild-animal suffering into the galaxy or simulating vast numbers of suffering sub-human minds to make a profit, we had better make sure our descendants actually care about animals. The situations that cause harm to animals in the future may well benefit humans at that point -- we have no idea.

Finally, I did like this statement from Dawkins, as quoted in Bekoff's article: " ... it is much, much better for animals if we remain skeptical and agnostic [about consciousness] ... Militantly agnostic if necessary, because this keeps alive the possibility that a large number of species have some sort of conscious experiences ... For all we know, many animals, not just the clever ones and not just the overtly emotional ones, also have conscious experiences." (p. 177) This is how I feel about insects. They easily may not be conscious (I'd give a ~55% probability that they are not), but we should actively consider the implications if they are conscious because of their great numbers. It's totally appropriate to talk about probabilities and expected values in the right context, but my complaint to Dawkins is that among the general public, the language of uncertainty makes people confused and less motivated.

Grounding animal ethics

Following is a post that I originally made on a discussion thread in the Facebook group Reducing wild-animal suffering. It's sufficiently stand-alone as to be a blog entry as well. This piece is in reply to "The Paradox of Paternalism: A Dilemma for Naturalism" by Paul Hansen, 10 June 2012. Thanks for sparking this discussion, Paul.

----

Interesting essay. I'm so accustomed to naturalist consequentialist philosophers these days that to read Paul's piece feels like landing on an alien planet. :) There's such a huge gap between how we understand ethics.

There are two ways to read what Paul is saying. First the less charitable account. I think the project of trying to ground ethics -- whether in naturalism or in religion or in anything else -- is confused. We care about what we care about. A lot of people, including myself, care about the suffering of other organisms, and we want to reduce that suffering. This isn't the same as the "ethics of care" that Paul tries to refute. I'm not saying that people should treat animals well in proportion to how much they care about animals. Rather, I'm saying that *I* care about animals, so I'm going to do what I darn well can to make sure everyone else respects animals to the degree I think they should. :) It's as simple as that -- there's nothing more to say about the ethical underpinnings of animal welfare.

Now, the more charitable interpretation of Paul's essay is to read it not as talking about anything metaphysical -- not as talking about absolute ethics built into the fabric of the universe -- but rather, just as talking about what kinds of intuition pumps will persuade others to agree with you. Indeed, the reason why people started talking in terms of absolute morality may have been because it's more persuasive to say "God commands you to do X" or "Absolute morality commands you to do X" than it is to say "I would really like it if you did X and really dislike it if you didn't. Pretty please? :D"

Each of the approaches to grounding ethics that Paul cites may appeal to different people. Some feel moved by moral rights, some by utilitarianism, some by care ethics, and some by religious or spiritual messages. It's a matter of choosing the marketing pitch that's most compelling to the target audience. Certain people do feel uneasy with non-religious groundings because they sympathize with the intuitions that Paul lays out, e.g., that nature is red in tooth and claw, so why should humans act any nicer if they're just animals? Needless to say, that instance of the naturalistic fallacy is not an argument; it's just a sentiment that some people find compelling. (I myself find it to be nonsense.) For those who can't easily be persuaded in other ways, probably Paul is right that appealing to religion could help.

That said, I'm wary of doing this too much. While caring about animals is really important, so is not having a deluded outlook on the world. Religious views especially can fall prey to perverse assumptions that make animals get hurt, like the belief that nature is "how it was created to be" and therefore we shouldn't "play god" by intervening. Some of the most passionate defenses of the view that nature teems with joy come from religious believers trying to prove that God isn't evil. I doubt this is what Paul had in mind when he said that religion "has been remiss or complicit in practice, with respect to animal abuse," but it applies here as well as to direct human-caused suffering.