Scientists Map Brain’s ‘Thesaurus’ to Help Decode Inner Thoughts

Posted on April 27th, 2016

What if a map of the brain could help us decode people’s inner thoughts?

Scientists at the University of California, Berkeley, have taken a step in that direction by building a “semantic atlas” that shows in vivid colors and multiple dimensions how the human brain organizes language. The atlas identifies brain areas that respond to words that have similar meanings.

Credit: Alex Huth, UC Berkeley

Scientists map how the brain responds to different words.

The findings, published in the journal Nature and funded by the National Science Foundation (NSF), are based on a brain imaging study that recorded neural activity while study volunteers listened to stories from “The Moth Radio Hour.” They show that at least 1/3 of the brain’s cerebral cortex — including areas dedicated to high-level cognition — is involved in language processing.

Notably, the study found that different people share similar language maps.

“The similarity in semantic topography across different subjects is really surprising,” said study lead author Alex Huth, a postdoctoral researcher in neuroscience at UC Berkeley.

When spoken words fail

Detailed maps showing how the brain organizes different words by their meanings could eventually help give voice to those who cannot speak, such as people who have had a stroke, brain damage or motor neuron diseases such as ALS. While mind-reading technology remains far off on the horizon, charting language organization in the brain brings decoding inner dialogue a step closer to reality, the researchers said.

“This discovery paves the way for brain-machine interfaces that can interpret the meaning of what people want to express,” Huth said. “Imagine a brain-machine interface that doesn’t just figure out what sounds you want to make, but what you want to say.”

For example, clinicians could track the brain activity of patients who have difficulty communicating and then match that data to semantic language maps to determine what their patients are trying to express. Another potential application is a decoder that translates what you say into another language as you speak.

“To be able to map out semantic representations at this level of detail is a stunning accomplishment,” said Kenneth Whang, a program director in the NSF Information and Intelligent Systems division. “In addition, they are showing how data-driven computational methods can help us understand the brain at the level of richness and complexity that we associate with human cognitive processes.”

Huth and six other native English speakers participated in the experiment, which required volunteers to remain still inside a functional Magnetic Resonance Imaging (fMRI) scanner for hours at a time.

Each study participant’s brain blood flow was measured as they listened, with eyes closed and headphones on, to more than two hours of stories from The Moth Radio Hour, a public radio show in which people recount humorous and poignant autobiographical experiences.

The participants’ brain imaging data were then matched against time-coded, phonemic transcriptions of the stories. Phonemes are units of sound that distinguish one word from another.

The researchers then fed that information into a word-embedding algorithm that scored words according to how closely they are related semantically.

Charting language across the brain

The results were converted into a thesaurus-like map that arranged words on images of the flattened cortices of the left and right hemispheres of the brain. Words were grouped under various headings: visual, tactile, numeric, locational, abstract, temporal, professional, violent, communal, mental, emotional and social.

Not surprisingly, the maps show that many areas of the human brain represent language that describes people and social relations, rather than abstract concepts.

“Our semantic models are good at predicting responses to language in several big swaths of cortex,” Huth said. “But we also get the fine-grained information that tells us what kind of information is represented in each brain area. That’s why these maps are so exciting and hold so much potential.”

Senior author Jack Gallant, a UC Berkeley neuroscientist, said that although the maps are broadly consistent across individuals, “There are also substantial individual differences. We will need to conduct further studies across a larger, more diverse sample of people before we will be able to map these individual differences in detail.”

In addition to Huth and Gallant, co-authors of the paper are Wendy de Heer, Frederic Theunissen and Thomas Griffiths, all at UC Berkeley.

This NSF-funded project is an example of how NSF invests in theĀ frontiers of brain research.

Neuroscience Research into Dyslexia Leads to ‘Brainprints’

Posted on April 15th, 2016

A wonderful thing about basic research is its tendency to produce advances researchers hadn’t anticipated. Cognitive neuroscientist Sarah Laszlo, for instance, found her early childhood learning studies took an unexpected jump into the worlds of security and identity verification.

Credit: Sarah Laszlo, Binghamton University

Cognitive neuroscientist Sarah Laszlo of Binghamton University, State University of New York, prepares a subject to measure brain activity using electroencephalography (EEG).

Laszlo’s research at Binghamton University, State University of New York, uses electroencephalography (EEG) to measure children’s brain activity as they learn to read. Through collaboration with colleagues, however, she found the work also offered a potential breakthrough in biometrics — physical attributes, like fingerprints, that can be used to verify people’s identities.

Advancements over the past decade have revolutionized what EEG can tell researchers. Improvements in underlying technologies (e.g., size, comfort, and portability of sensors, the ability to measure the signal, and the ability to analyze large amounts of data) allow Laszlo and her colleagues to follow individual children’s development over time. Those new advantages have created opportunities to study an important area of learning development: reading.

“Previous research in this area predominantly focused on comparing groups of people, but when we are following an individual, we can begin to predict, on a child-by-child basis, who will develop problems reading in the future, at least two years in advance,” she said. “That gives us a lot of extra time to help that child before problems become noticeable.”

The ability to predict future problems with reading would provide an important tool for researching and preventing development of these issues. Research has shown that intervention can effectively help children with dyslexia and other reading disabilities — but that intervention must take place early, usually before second grade. Even attentive caregivers can have difficulty recognizing when a child has reading troubles before first or second grade.

“We are working toward developing a type of reading ability screening test that could be used how a hearing test is used now,” Laszlo said. “Having a predictive test would double or triple the time period for an effective intervention.”

Her lab studies children ages 4 to 14 who fall across the spectrum of reading ability, from gifted to dyslexic readers. She takes repeated recordings of brain activity while a child reads. Her lab is now beginning to understand characteristics that, when taken together, represent red flags discernable early enough in brain development (say in a four year old who has just barely started to understand letters) to allow for a predictive test and early intervention.

“If we can identify kids that will be dyslexic and help them before they even have a problem, it is really a big deal,” Laszlo said. “I am excited by the promise this research has to prevent problems for kids and protect them from experiencing negative life-long effects of reading problems.”

A different direction: biometrics

When her individualized brain readings over different time points caught the attention of bioengineer Zhanpeng Jin, a colleague across SUNY Binghamton’s campus, Laszlo’s research jumped in a new, direction. Jin studies biometrics and thought Laszlo’s brain activity readings could be used as a brain-based, biometric.

Using brain readings as a security measure to prevent identity theft has several advantages over other biometrics like fingerprints or retinal scans. For example, they cannot be copied surreptitiously or taken from someone who is deceased. Brain readings could prove a game-changer for the security industry. But brain readings can only be useful if they are extremely reliable; a measure that only recognizes an individual most of the time would not work as a security device because people would get locked out of their own devices and offices.

Laszlo, Jin, and their colleague, Maria V. Ruiz-Blondet, decided to explore whether they could perfect this approach by measuring brain activity in adults who were either focusing on a recurring, easily remembered, thought or looking at specific images of different foods, words, 3-D designs and celebrity faces.

By analyzing brain activity response to visual and thought stimuli, Laszlo said “We can identify the individual with 100 percent accuracy. When Zhanpeng first came to me with the idea, I honestly didn’t think it would work. It’s amazing.”

National Brain Observatory: a Phased Approach for Developing a National Research Infrastructure for Neuroscience

Posted on February 19th, 2016

Dear Colleagues:

With this Dear Colleague Letter (DCL), the National Science Foundation (NSF) is announcing the intention to foster the development of a national research infrastructure for neuroscience (National Brain Observatory) to support collaborative and team science for achieving a comprehensive understanding of the brain in action and context. Understanding the brain is one of the grand scientific challenges at the intersection of experimental, theoretical, and computational investigation in the biological, physical, social and behavioral sciences, education research, and engineering. Achieving a comprehensive understanding of the brain requires increased emphasis on systematic, multidisciplinary collaboration and team science to establish quantitative and predictive theories of brain structure and function that span levels of organization, spatial scales of study, and the diversity of species. This challenge necessitates the development of innovative, accessible, and shared capabilities, resources and cyberinfrastructure, along with the eventual organizing of these into a coherent national infrastructure for neuroscience research.

Large-scale collaborative efforts facilitated by shared instrumentation, communication, data representation, and workflow systems, and advanced computational and data resources have enabled transformative discoveries across the spectrum of scientific disciplines. In neuroscience, rapid proliferation of advanced measurement instrumentation and techniques has allowed researchers to study the brain, nervous system, cognition, and behavior at ever-finer physical and temporal scales, and generate very large datasets. However, integrative efforts in neuroscience research are hampered by a lack of systematic means for encouraging maximal utilization of existing resources, and for developing and disseminating new resources that can serve whole disciplines in collecting, managing, and analyzing large-scale data, and comparing those data to theoretical and computational models.

This multi-directorate effort is part of the NSF’s Understanding the Brain activity, including NSF’s participation in the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative ( and the National Brain Observatory (NBO) effort.

This effort will be realized through a phased approach that:

  • Fosters development and dissemination/deployment of innovative research resources and instrumentation, neurotechnologies and behavioral paradigms that can be applied across the phylogenetic spectrum, theoretical and computational frameworks, and data infrastructure resources while providing greater access to existing resources where possible and serving broad communities within the brain sciences;
  • Supports collaborative networks composed of neuroscientists, behavioral scientists, and theorists working in concert with technology and cyberinfrastructure developers on a common question or theme from a variety of perspectives; and
  • Facilitates the emergence of a coherent national infrastructure comprising the above shared and accessible tools, resources and networks that will allow rapid integration, analysis, and modeling of brain data associated with behaviors from multi-disciplinary projects and enable large-scale collaborative research efforts nationally and internationally that will advance our understanding of brain structure and function.

NSF plans to continue to release Dear Colleague Letters and Solicitations with refined guidance and specific funding opportunities aligned with each of the three phases described above, as this campaign continues into the future. NSF anticipates that this initiative will usher in a new frontier of brain exploration by empowering research communities to cooperatively collect, share, analyze, and model data across molecular, cellular, organismal, developmental, behavioral and evolutionary levels in order to reveal the fundamental principles of nervous system function and complex behavior. If you have questions concerning this DCL, please contact a program officer representing the program or solicitation of interest.


James L. Olds
Assistant Director for Biological Sciences

James Kurose
Assistant Director for Computer & Information Science & Engineering

Joan Ferrini-Mundy
Assistant Director for Education and Human Resources

Pramod Khargonekar
Assistant Director for Engineering

F. Fleming Crim
Assistant Director for Mathematical and Physical Sciences

Fay Cook
Assistant Director for Social, Behavioral, & Economic Sciences