How Our Brains Grasp Faces

From the earliest months of life, we rely on faces to help us navigate the world. They tell us who’s safe, who’s familiar, and whether they’re paying attention to us. But do our responses to faces develop gradually as our brains mature? Or are we born prewired to lock onto the human face. In this episode of Under the Cortex, cognitive scientists Rebecca Saxe of MIT and Heather Kosakowski of Harvard University join host Scott Sleek to discuss their groundbreaking findings, published in Current Directions in Psychological Science, about the development of the brain’s face-processing network.

If you’re interested in learning more about this research, visit psychologicalscience.org. 

Send us your thoughts and questions at  [email protected].

Under the Cortex is going on hiatus and won’t be recording new episodes for the foreseeable future. Explore our archive of 150 episodes filled with scientific insights, stories from editors and researchers, and conversations worth revisiting again and again!

Unedited Transcript

[00:00:09.220] – APS’ Scott Sleek 

From the earliest months of life, we rely on faces to help us navigate the world. They tell us who’s safe, who’s familiar, and whether they’re paying attention to us. But do our responses to faces develop gradually as our brains develop? Or are we born pre-wired to lock onto the human face? This is Under the Cortex. I’m Scott Sleek with the Association for Psychological Science. With me today are cognitive and brain scientists, Rebecca Saxe of MIT, and Heather Kosakowski of Harvard University. In recent findings published in the APS Journal, Current Directions in Psychological Science, they challenged some long-held theories about the development of the brain’s face processing network. Rebecca, Heather, welcome to Under the Cortex. 

[00:01:02.760] – Rebecca R Saxe 

Thank you. Thanks so much for having us. 

[00:01:04.680] – Heather L. Kosakowski 

Happy to be here. 

[00:01:06.500] – APS’ Scott Sleek 

Can you start by giving us a quick summary of the key question your study set out to answer? 

[00:01:13.220] – Heather L. Kosakowski 

Well, your intro was actually quite great. This paper that we wrote together actually builds off the work of many scientists, not just our work. Collectively, we’re interested in how the human brain supports our ability to perceive and think about others and what the developmental origins of these abilities are. One place we can start with is with faces because they’re so fundamental to how we interact and understand other people, especially for people that have vision. 

[00:01:45.660] – APS’ Scott Sleek 

What are the main face selective regions in the adult brain, and what roles do they play in perception and social cognition? 

[00:01:54.980] – Heather L. Kosakowski 

Well, that is such a great question. There is actually a lot of debate in the field about what constitutes a face selective region and how many face selective regions are present in the human brain. The fusiform face area, or FFA, is the most famous face selective region, and it was discovered by Nancy Kenwisher. In human brains, neurons in the FFA respond really strongly anytime a person sees a face, but not as strongly when people are looking at non-face objects like tomatoes or clucks. When neurons in the FFA artificially stimulated, people say that they see a face, even if they’re just looking at a black box. So not only is the FFA face selective, but it has a causal role in face perception. Many, many experiments suggest that FFA responses may reflect the invariant features of faces that establish a person’s identity across viewpoints and styles. But like I mentioned already, there are other regions that respond really strongly faces and nonface objects, too. But these other regions also have strong responses to other types of stimuli that are not visual. In our paper, we talked about the superior temporal sulcus or STS, and medial prefrontal cortex or MPSC as two examples. 

[00:03:18.780] – Heather L. Kosakowski 

The STS goes along the lateral side of your brain, and a region in the STS has strong responses to faces compared to nonface objects, but also has strong responses to voices. Responses in the STS appear to capture a person’s momentary thoughts and feelings as expressed by their face, voice, or body movements. Let me give you an example. Imagine you over here, two people talking at the water-cooler, but you can’t make out what they’re saying. Or maybe you see two people playing hopscotch together at the park across the street. In either case, your STS would be engaged because you’re perceiving a social interaction. Like the STS, the MPS MPSC responds strongly to faces compared to nonface objects. In adults, the pattern of response in MPSC contains information about abstract social meaning and generalizes across stimulators types. It could be face images or face movies or animations without faces or just reading sentences, and your MPSC would be engaged whenever you’re thinking about meaningful self-relevant social relationships. If you think about a a parent, a coworker, a lover, or even your enemy, the MPFC will be engaged, especially when you’re evaluating how you feel about them. We picked these three regions, FFA, SDS, and MPFC, because we know they all respond strongly to faces in adults, but they have very different functions. 

[00:04:50.340] – Heather L. Kosakowski 

It gave us a good opportunity to ask about the function of these regions and how function develops in the infant’s brain. 

[00:04:58.280] – APS’ Scott Sleek 

A lot of abbreviations for me to I’ll keep track of. 

[00:05:00.960] – Heather L. Kosakowski 

Well, the three most important ones are FFA, which is the fusiform face area, STS, which is the superior temporal socus, and MPFC, which is the medial prefrontal socus cortex. The FFA is at the bottom of your brain, the SDS is on the side of your brain, and the MPFC is at the front of your brain where your two frontal lobes press together. Just three to remember. Okay. 

[00:05:27.040] – APS’ Scott Sleek 

Rebecca, prior to this work, what were the prevailing theories about how these regions develop in infancy? 

[00:05:35.780] – Rebecca R Saxe 

Well, thanks. That’s a great question. The big debate about where human cortical function comes from, it’s always about what the role of bias is that you’re born with? How much do our brains start some way, and then what’s the role of learning from experience? How much does it matter what we see in the world around us? And the answer to any question about development is always going to be both. But so there’s questions about how much do you depend on these two sources and also over what time. So a big question for the last 20 years has been those cortical regions that Heather described, how long does it take them to start having adult-like function? You could imagine all kinds of answers. Maybe it takes all the way into adolescence when we start having adult-like social relationships. And some of the first fMRI studies suggested it might take at least until age five or eight before kids’ brains had all of these cortical regions. The question that we were interested in was, what about babies? Do babies’ brains already have these cortical functions? Does it have one but not the others? Can we see change during infancy? 

[00:06:45.840] – APS’ Scott Sleek 

While your findings suggest that these regions emerge early and in parallel, what methods did you use to uncover this? 

[00:06:52.600] – Heather L. Kosakowski 

A lot of scientists used a lot of different methods, and we tried to bring them all together in this paper. Rebecca Jessica has, I’ll let her speak for herself, but she’s used both FNEARs and fMRI. I’ve used primarily fMRI, which is Functional Magnetic Resonance Imaging. That takes a picture of your brain, or in this case, a baby’s brain, every couple of seconds. Then you can compare responses in the brain for when babies are looking at faces, for example, compared to objects or toys. Fnears is similar. It’s Functional Near Infrared Spectrophies. Microscopy, but it uses lights instead of a magnet. You can’t get a picture of the whole brain. You can only get blurry measurements of large swaths of cortex that are close to the surface. We tried to bring together all of these findings across the field to say, if you look at everybody’s work, what has everybody found in terms of FFA, SDS, and MPSI, and how they might respond to different types of visual stimuli or auditory stimuli that to the functions we know that these regions have in adults? When we did that, we found some surprising but also very convergent results suggesting that FFA, SDS, and MPSI are all present and have meaningful responses to faces very early in development, as early as we were able to find anybody that took any measurements, which is about two and a half to five months. 

[00:08:27.700] – Heather L. Kosakowski 

Do you want to add more to that, Rebecca? 

[00:08:30.240] – Rebecca R Saxe 

Yeah. So I guess if the question is about what are the methods, if you want to study a baby’s brain, there’s a lot of limits and constraints, right? So we have some knowledge of baby brain development from 150 years of neuroscience. Most of that comes from postmortem brains. So studying the anatomy of a brain after death. That’s been done for 150 years. And there’s really important discoveries about brain development that come from studying brains after death. But one of the things you really can’t ask is about function. What does that anatomy let you do and not do? So here we’re trying to focus on methods to let you study function directly in an awake and functioning brain. And then, so that means measuring activity in a baby’s brain while they’re awake and looking at the world. The problems there are that all of the measurement tools we have for neuroscience are very sensitive to movement. To make these measurements a baby has to hold completely still. The measurements are noisy. So we get better images the longer you can do the experiment. So in an adult, a standard experiment might be two or three hours in which the adult has to hold perfectly still and do exactly what we ask them to do over and over again. 

[00:09:50.460] – Rebecca R Saxe 

If you’ve met a baby, you know that’s not a baby’s thing. So the key trick of baby neuroimaging is trying to extract meaning from the littlest glimpses that we can get with these neuroimaging tools. From just a few minutes of fMRI or just a few minutes of these NEARs measurements, what can we see in the blood flow changes in a baby’s brain, and what sense can we make of it? 

[00:10:19.060] – APS’ Scott Sleek 

That’s why you combine both FNEARs and fMRI in studying the brain. 

[00:10:27.680] – Rebecca R Saxe 

Yeah. Fnears has been used in babies for longer because you just put a cap on a baby’s head. They can be sitting on their mom’s lap. It’s more comfortable and familiar. There’s about a decade’s worth of experiments using FNEers to measure babies’ cortical activity. And those studies are super important, probably because there’s many of them, and they’ve asked many different questions and questions nobody’s asked in MRI. In particular, an MRI machine, if you’ve ever been in one, is very loud. Anything where what we’re trying to study is responses to voices is much harder to do in MRI. So there’s lots of studies in NEARs. There’s a decade’s worth of work. It’s more comfortable for the babies, and it’s better for sound. On the other hand, as Heather said, because NEARs is measuring at the skull, there’s a lot of the brain that just can’t see. One of the most important parts of the brain that you can’t see with NEARs is FFA. Ffa is the first most famous facelift collective brain region. Any study that’s about the origins of face responses in the brain has to include FFA, but NEARs can’t see FFA. That was part of the impetus to want to start trying fMRI, even though it’s this loud, dark tube. 

[00:11:46.300] – Rebecca R Saxe 

It’s scary. It shakes. You can’t sit on your mom’s lap. You have to lie on your back. Lots of things babies don’t like. But it lets us measure FFA, and it lets us measure the whole brain at once. One of the key questions we were studying here is, what’s the relationship between these different brain regions? In particular, a key question in the field is, do they develop one after the other or all at once? It’s really hard to ask that question if you can’t measure them simultaneously in the same baby. The main reason to want fMRI in the toolkit is because it lets you measure all the brain regions simultaneously. 

[00:12:25.360] – APS’ Scott Sleek 

Okay. Was there anything surprising or unexpected in how early these face responses appear in the infant brain? 

[00:12:34.140] – Heather L. Kosakowski 

Absolutely, yeah. When I started this project, I was a brand new graduate student. The first study that ever looked at all of these regions simultaneously had just been published by Rebecca and her then-grad student, Van Dien. It was very exciting, and it suggested that there were not face selective responses anywhere in the brain, in the infant brain. But there were some other studies suggesting that face selectivity emerges in childhood, but that there are patterns of response that could distinguish faces from other kinds of objects. Even though there weren’t selective responses, there were patterns of responses that could be meaningfully measured. We actually set out to try to measure those patterns of responses so we could do a longitudinal study to try to understand how face selectivity develops. But as a naive graduate student, I didn’t fully appreciate I appreciate how hard imaging infants was. It took me a really long time to collect any data. Then once I did collect the data to learn to analyze the data. But once I learned to analyze the data and we started seeing the results, we actually started seeing face selectivity in the brain, which was completely unexpected. 

[00:13:50.570] – Heather L. Kosakowski 

We didn’t expect to see it in FFA or STS or MPS or any region at all. That was surprising. But then when When we did more analysis and we just constrained it to only the babies that were two and a half to five months old, we found that all three of these regions were face selective in even the youngest babies. Yes, every Everything about our initial fMRI study or the fMRI study I did with Rebecca, all of the results were completely surprising. We did expect to see patterns of response, just not the full face selectivity that we ultimately found. 

[00:14:30.320] – Rebecca R Saxe 

One of the things that Heather’s understanding, so you didn’t say in your introduction that Heather was my graduate student, though she is now a postdoc and about to start her own lab. This has been a long journey for both of us. But when we first started, There were almost no fMRI studies, right? Almost no information about the origins of these. Almost no fMRI studies in babies, of course. There were many, many fMRI studies, but not in awake babies. The study that Heather is appealing to, the one study we had already done, the data in that study came mainly from four babies because it was brand new, and we didn’t know how to get a baby in an MRI machine or how to get enough data. The main way we not enough data was by having families who were willing to try this unknown new thing over and over again, who were very tolerant to all of the failures. And so that very first paper, most of the data come from my first child, my second child, my sister’s child, and my postdoc’s child. It’s not a sustainable way to run a research program, but it also explains why some of the answers were very provisional. 

[00:15:44.300] – Rebecca R Saxe 

We were seeing just the very first glimpse, it was already big news that we could see patterns in the brains of babies at all. But as Heather said, from that first glance, we thought we could see patterns, but we couldn’t see selective regions. Again, I don’t want Heather to undersell herself. From the first attempt where any baby was a miracle, Heather successfully collected data in 80 babies, and she only stopped because there was a global pandemic. I think It just never would have stopped. But also, she also designed and developed a whole new coil with much higher signal to noise ratio. Just everything about stage 2 of the science was better than stage one, as you hope science will be. The first thing we found is that we were wrong. The first time when we studied these babies and we thought, no selectivity, there’s going to be change after infancy. Then Heather went back and looked more carefully, and that wasn’t true. The first selectivity was already there in babies and already there as early as we could look. 

[00:16:50.600] – APS’ Scott Sleek 

You challenged the idea of a slow posterior to anterior developmental sequence. What are the broader implications of that? 

[00:17:00.360] – Heather L. Kosakowski 

Yeah. Like Rebecca mentioned earlier, there’s been a lot of anatomical study of infant brains, mostly in postmortem infant brains. All of those studies have shown that there’s a slow posterior to anterior developmental sequence. So your neurons and your myelin and all of that emerges first in regions that support sensation, like vision and somatosensation and hearing. For a long time, the dogma has been that you can’t have meaningful functional responses in a region until it’s fully anatomically developed. So that would imply that the MPFC is doing nothing and completely silent in infancy, which for a long time people thought was probably the case. By challenging this posterior to anterior developmental sequence, where arguing that infants are capable of processing more information about their environment than just sensory inputs. They’re not seeing and hearing a mess of stuff that doesn’t make any sense. There’s actually structure to the things that they see and structure to the things that they hear. That structure is influencing the way their brain develops. One thing that Rebecca and I are both really interested in is the social nature of humanity. So humans are inherently social and born into an increasingly complex social environment. 

[00:18:38.080] – Heather L. Kosakowski 

We argue that infants need neural structures or systems that will enable to process this more complex conceptual information while also processing sensory inputs. 

[00:18:49.660] – APS’ Scott Sleek 

I’m trying to understand, infants cortical responses to faces, are they already socially tuned then or are they simply perceptual at this early stage? 

[00:18:59.500] – Rebecca R Saxe 

Yes. It’s such a good question, in part because we have to make an argument. It’s not like the answer is already known. In fact, part of what’s fun about this type of article, The Current Directions, is that Heather and I are writing an argument at the boundaries of our understanding, suggesting, Here are the data. Here’s what we think they mean. But it’s not like the way you write history of science 150 years ago. We figured all this out, and it’s totally established. The pleasure of this article is we’re writing an argument when other arguments are possible, when scientists don’t already all know the answer or all agree. What we’re saying is from these, and just to go back to the story I was telling you a minute ago, I’ve been wrong before. It’s totally possible I’m wrong now. That’s the joy of this. The argument is, look, here are the little bits of baby data we’ve been able to collect so far. A little bit of f from Raya data, mostly from Ben Dean and then Heather’s heroism in my lab, a bunch of f NEARs data with their strengths and weaknesses, put them together, what do you see? 

[00:20:04.620] – Rebecca R Saxe 

In that context, what we’ve argued is in the fMRI data, we see face selectivity. It’s only a tiny response compared to an adult. Baby brain responses are incredibly small and weak, but they are already face selective as soon as we can measure them. That’s the fMRI data and all the brain regions at once. The NEARs data tell us somewhere in medial prefrontal cortex and STS are responses to more social meaning. Responses that are to, for example, infant-directed voices as opposed to adult-directed voices, familiar people as opposed to unfamiliar people, people looking at you as opposed to looking away. One way of putting all of those pieces together, and the one we argue for in the article is that just like in adults, babies have an architecture ready to not just see faces as interesting, important, and frequent shapes in their visual environment, but also to experience them as socially meaningful, to connect the face to the voice, which we know from behavior babies can do, and to connect the face and the voice to meaning or value, to something that the infant cares about. Because actually for a human baby, the only survival relevant thing is getting cared for. 

[00:21:21.420] – Rebecca R Saxe 

That’s how human babies survive. We’ve argued it makes sense, a priori and in light of all the data that we showed in this article, to think of a baby’s brain as in some way prepared to connect the social things in its environment, the faces and voices and sounds and touch, to a motivation to learn about caregivers and to connect with them, to form relationships with them. Of course, we’re not sure that’s true. That’s why you write a current directions article. 

[00:21:54.520] – APS’ Scott Sleek 

Okay. Well, you talked about the limits of infant neuroimaging, but I wonder if you could expand a little bit on the biggest methodological challenges that you’re hoping to overcome or that you have overcome. 

[00:22:13.100] – Heather L. Kosakowski 

There are a lot of methodological challenges, and many of them are out of our control. We cannot tell a baby, Oh, you need to stay still for this amount of time. A lot of methodological challenges have been improved, even since we published our paper. There’s a lot of people in the field that are working on doing better data registration, making sure we know where in the brain we’re looking when we’ve taken measurements from a brain, or the coil that we designed in collaboration with Boris Kell. That’s now being used at multiple labs across the country. Or having meetings where we collaborate or talk to other people doing infant neuroimaging to find ways to improve our methods? Like, does it matter who’s scanning the baby or does it matter what time of day the baby was being scanned? We’re looking into all of those types of questions to see if we can come up with some better ways to improve the methods. But I think that the limits are really in human patience. These studies take a lot of time and a lot of energy and dedication and effort. We need people that are willing to put in the time and effort to do this really hard work because they’re curious and want to know the answers. 

[00:23:39.160] – Heather L. Kosakowski 

I think that that is going to be our biggest limit. It’s just human curiosity. Okay. 

[00:23:49.420] – APS’ Scott Sleek 

I’m curious, how might your findings inform future research on atypical development, such as autism spectrum disorders? 

[00:24:01.100] – Rebecca R Saxe 

Yeah, that is very important and hard because… So I’ve been in social cognitive neuroscience for 25 years now. And 25 years ago, when we were first discovering cortical regions involved in social cognition, it seemed like solutions to the questions about autism were at our fingertips. Here’s a developmental disorder in which kids struggle, particularly with social interaction and social relationships. Here’s a new tool that lets us look in the brain. We look and there’s all these brain regions that play special roles in social interactions and social relationships. It seemed we were just one link away from connecting the struggles of kids with autism with the functions of these brain regions and then maybe with possible treatments. Over the last 25 years, I’ve come to think that that hope was an illusion. The seeming alignment between the challenges and the humanitarian need to study autism and the discoveries that fMRI gave us, it looked like they were so aligned, but it turns out they’re not. So autism remains a huge puzzle. And the more we understand about these social cognitive brain regions, we’re learning a huge amount about their function. But it turns out there’s no simple one-to-one relationship between things we can measure in the brain and the struggles of people with autism. 

[00:25:35.580] – Rebecca R Saxe 

And there’s a lot of possible reasons that that could be. One of them is it could be that autism is many, many different diseases. So this is something people have talked about more and more recently is that the diagnosis of autism could be given to many different children and adults for many different reasons. And if that’s true, then it might still be that these studies can give us insight into a subcategory of autism if we could find the right subcategory, that the problem has been trying to look for the cause of one disorder in an environment in a context where there are many different things going on. And of course, we should also acknowledge that it’s not only variable, it’s also variably understood. Some people who self-identify with autism understand understand themselves as not having any disorder, having just a different way of processing social relationships. That’s an important perspective to acknowledge. At the opposite end of the spectrum are children with autism who are affected not only in their social relationships, but in every aspect of their life, who can’t talk, who can’t dress themselves. So trying to understand where in the huge spectrum of things called autism, these studies of social cognitive brain regions could fit and be useful It’s proven heartbreakingly much harder and less clear than I hoped it would be when I started. 

[00:27:09.040] – Rebecca R Saxe 

Okay. 

[00:27:09.700] – APS’ Scott Sleek 

Well, tell us about the next steps in your research. Are you planning follow-up studies to say map functional differentiation over time? 

[00:27:19.940] – Heather L. Kosakowski 

Well, I’m starting my own lab at the University of Southern California, and so I am very excited about the research I’m planning. I do plan to do some follow-up studies to map functional differentiation over time, specifically a longitudinal study looking at how these brain regions respond to different face and social stimuli with well-matched controls. So yes. 

[00:27:50.240] – Rebecca R Saxe 

My lab is also still doing some work on early cortical development. One big project in the lab that we’re just wrapping up has looked at the beginnings of language language responses. So the babies that Heather studied are at the point of first recognizing just one or two words. These are tiny infants who can recognize maybe half dozen words, can’t produce any. And we’re studying now toddlers who are a little older, so one and two-year-olds, around the time that they can produce 50 to 100 words and sometimes combine them. So the question there that we’re asking is this, again, very similar. Do the brain regions involved in language processing arise slowly? Are they prepared? Do they arise in an order or all at once? And one big question for language is for most right-handed people, most of our language processing happens in our left hemisphere. You know this from, for example, stroke patients. If you get a stroke in your left hemisphere, it’s more likely you’ll lose either the ability to understand or produce language. People who have a stroke in their right hemisphere usually don’t. The language which is preserved. So that asymmetry that our left hemisphere is in charge of language, a fundamental question about human brain development, where does that asymmetry come from? 

[00:29:11.650] – Rebecca R Saxe 

Compared to all other species that we know of, we have a much bigger asymmetry, more differences between our left and right hemisphere than other animals do. Is that caused by language? Did it happen evolutionarily or does it happen in toddlerhood? To ask that question, we’re asking, when you first learn your first language at your first 100 words, is it already mostly on your left hemisphere, or does it start out balanced and become asymmetric over time? 

[00:29:39.860] – APS’ Scott Sleek 

Well, this has been a fascinating discussion. Rebecca and Heather, thank you both so much for joining us today on Under the Cortex. 

[00:29:49.040] – Heather L. Kosakowski 

Thank you for having us. 

[00:29:51.080] – APS’ Scott Sleek 

I’m Scott Sleek with the Association for Psychological Science. I’ve been speaking today with Rebecca R Saxe and Heather Kosakowski. If you want to learn more about their research, visit psychologicalscience.org. Would you like to reach us? Send your questions and suggestions to [email protected].