National Science Foundation Graduate Research Fellowship Application Tips

Fall officially begins tonight, but the new school year is already in full swing. For many first and second year grad students (and some prospective grad students), that means it’s time to prepare National Science Foundation (NSF) Graduate Research Fellowship Program (GRFP) applications. It can be a stressful experience, but never fear: I am here to help with my unsolicited advice!

Each year, the NSF awards ~2,000 competitive fellowships to grad students in a variety of STEM disciplines. In addition to providing a prestigious resumé (or, as we prefer in academia, CV) boost, these fellowships pay our tuitions and a $32,000/year stipend for three years.

MoneyShower

I applied for an NSF GRF back in 2011, as I was preparing my grad school applications. I was lucky enough to receive an award that year – and I do mean lucky; grant reviewing is quite subjective, and my award came down to my ability to please 3 randomly assigned human reviewers.

I am convinced that I received a big boost because I applied so early in my research career. Reviewers expected less of me because I wasn’t a grad student yet – even though I had been more or less acting like one in my paid job as a research assistant – so my ability to write convincingly about scientific research was probably more impressive than it would have been had I been a first or second year grad student. So if you are just now applying for grad school, also apply for an NSF! Expectations will never be as low for you again.

AmyPoehler_lower-your-expectations-gif-Am-rkbp

I got loads of help when I wrote my NSF GRFP essays and did lots of Googling on best practices. Here are some tips I put together that I hope will be helpful to other NSF GRFP applicants who are furiously Googling now!

Rosa’s unsolicited NSF GRFP application advice

1) Just do it!

It’s not easy, and the whole process can kind of suck because GAH! so much work for a gamble at free money! But the NSF GRFP is probably one of the shortest major applications that you’ll ever prepare. The cost-benefit ratio is definitely worth it.

If you don’t get an award, you still get feedback on your application, which will help you improve future grant applications. If you do get an award, you have an NSF award! And you definitely won’t get an award if you don’t apply.

If you’re like I was back in 2011, trying to navigate the grad school application process and wondering if you could throw an NSF GRFP application into the mix – really just do it! I found that writing my NSF GRFP application really made me think through my research goals, which was helpful for my grad school application process as well.

I could also note on my grad school applications that I had applied for an NSF fellowship, which made me look like an organized go-getter. After I received my NSF award, I could have reached out to professors who didn’t accept me as a student to see if having my own funding changed their minds.

LookAtMeNow

2) Start early! 

Your reference letter writers are supposed to comment not only on you but also on your proposed research. That means that NSF GRFP reference letters should be more tailored than usual, so you want to give your recommenders plenty of time to do that. I started drafting my essays at the end of September and sent my reference letter writers drafts of all of my essays in mid-October, just over a month before the deadline. [When I applied, applications were due mid-November. They're now due late-October/early-November, so I suggest moving my dates up 2 weeks.]

3) Don’t be shy or humble!

The NSF GRFP application form has minimal space for you to list your accomplishments, so you have to make sure to laud yourself in your essays. This is not the time to be humble. Brag about how awesome/smart/good at research you are.

4) Focus on broader impacts and work them into all of your essays!

You’ll be rated on two criteria: Intellectual Merit and Broader Impacts. Intellectual Merit is important and comprises half your score, but you should already have plenty of practice writing to that, so it will come pretty naturally. Furthermore, it’s probably what your letter writers will focus on.

Broader Impacts are less familiar. Really addressing them requires more than one or two throwaway claims about long-term trickle-down effects of your research. My year, the NSF GRFP website had a pdf with lots of examples of what they consider to be Broader Impacts activities. Look through that list, and try to incorporate as many items as possible into your essays. [Note that the aforementioned pdf was from 2011 because I can't easily find a more recent one. I doubt the NSF's criteria for Broader Impacts has changed much, but take the list from 2011 with a grain of salt.]

If you do any sort of volunteer or outreach work, even if it’s not directly related to your project, find a way to work it into your essays. I think you can stretch (though not break) the limits of plausibility here. I wrote about volunteering with inner-city kids at a local science museum and noted that “the experience made me aware of my responsibility to disseminate information beyond the scientific community.”

GoodPerson

If you’re female and/or a minority, you can work that into your essays. I chose not to go this route because I felt like I had adequately addressed broader impacts in other ways, but I have had friends who successfully did. I could have written something about how I’ve benefitted from having strong female scientists as role models and mentors, and how I want to be a role model and mentor to others.

Another easy Broader Impacts move is to mention how much you love teaching and mentoring undergraduates, and OMG you can’t wait until you get to be a TA/professor. There’s an example of that in my personal statement. [I still love working with undergraduates, by the way!]

BeEnthusiastic

If you do it right, you will probably sound corny. You may hate yourself a little for the corny, but it’s what they’re asking for, so get over it!

5) Get help!

I discussed a few different research ideas with my boss/PI before I settled on the one that I wrote about. She also made several important suggestions for additional measures to compare and helped me clean up/fancify the language I used to describe my proposed research so that I could sound extra scientifical.

My PI also informed me that I didn’t have to do the exact project that I proposed, which I hadn’t been aware of. NSF GRFP is about funding the budding young scientist rather than the exact scientific project, so applications just need to demonstrate an ability to write about research intelligently. Don’t worry about mapping out the exact, perfect project that you will definitely do. A good project that is well-described and justified will do.

I also reached out to friends who had won NSF awards in the past and asked to see their applications. One even offered to show me her rating sheets. It was really helpful to see examples of successful applications – and it was eye-opening to see how my friends approached their essays in substantially different ways. For example, one friend’s personal statement was about how being an immigrant made her want to study cultural differences, so quite personal. Another friend’s personal statement focused more on her research experiences – less personal, but still engaging and ultimately effective.

Finally, having a few people read your essays and offer comments and suggestions will be invaluable. I recommend getting a science-minded friend who’s not exactly in your subfield to read through your writing. Your raters will be in the field, but they may not be experts in your methods or analysis, so it’s good to get a check that you remain accessible in your writing.

6) Good luck!

MotivationalPenguin

Failed replications and null results are still science

Jason Mitchell’s self-published manifesto against replication studies (studies that try to reproduce a previously published significant result) and null results (those that fail to find a significant effect) has been flying around the psychology blogosphere. Neuroskeptic, Neuropolarbear, NeuroconscienceDrug MonkeyTrue Brain, and Richard Tomasett have already weighed in with great insight, but I still thought I’d add my two cents, if only so that I can get my thoughts down and stop ranting to my friends about it over Gchat.

First, why should we care what Jason Mitchell says? For starters, he’s Professor Jason Mitchell, Ph.D., a full professor at Harvard who is influential in my field of cognitive/social neuroscience. His lab has done great work that I really admire and that has shaped how I think about and approach my own research. For example, he found that we value social things in the same way that we value money or food: people are willing to give up money to be fair or to tell a stranger something about themselves, and when they do so, reward-related regions of the brain become active.

Given what the Mitchell Lab studies, I find it interesting (ironic?) that he wrote ~5000 words stating that 1) replication studies and null results have no value and 2) replication studies are unfair to the scientists who produced the original work being replicated.

Unidentical replications – which one is the failure? From flickr user joriel.

For point 1), Professor Mitchell notes that failed replications can result from screwing up methods in some way – messing up your code, contaminating a sample, etc. And he’s right! If you follow Paper X’s reported methods and get different results, you could have made a mistake in following those methods.

BUT you also could have exactly followed Paper X’s reported methods but not done everything exactly as the authors of Paper X did. Maybe Paper X’s participants were run in the morning when they were more alert, and your participants were run in the afternoon during that post-lunch funk.

Or maybe Paper X came from a lab with mostly male experimenters, and your lab is mostly females. We recently learned that rodents are stressed by male researchers, so maybe you didn’t replicate because your testing environment was less stressful. As Neuroconscience notes, there are plenty of #methodswedontreport – things that we do but do not bother writing down in a published paper. Discovering these legitimate factors that could explain a failure to replicate is scientifically valuable and part of the scientific method.

from flickr user afagen

Furthermore, the same screw-ups that can cause a replication to fail can also cause a study to produce significant positive findings – findings that can then be published in peer-reviewed journals and treated as fact. The problem with Professor Mitchell’s case is that he argues that all positive findings are true, and all null results are false.

That simply isn’t the case. Even without intent to deceive (fraud or cherry-picking data), sometimes you get a false positive just due to random chance, as I’ve noted before.

As to point 2), I agree that reporting failures to replicate can be taken as criticisms against or even bullying of the original authors. As Professor Mitchell puts it, “You found an effect. I did not. One of us is the inferior scientist.”

Such an attitude is flat out wrong. The field should never jump to conclusions that the original authors are poor scientists (or worse, fraudulent ones). In the same vein, it also should not jump to the conclusion that the replicating authors are poor scientists.

Instead, we should follow the evidence. Given one positive finding and one negative finding, can we empirically determine what extraneous factors could have caused these different results? If not, it could be that either finding is due to random chance, so the study should be repeated until the balance of evidence tips one way or the other.

via Flickr user kxlly

In my ideal world, science would not be about egos and proving yourself right or someone else wrong. It should be about hunting down the Truth. Our main weapon is the scientific method, which is inherently based upon supporting or disproving hypotheses.

In the real world, scientists are people too. We have egos that can be bruised and feelings that can be hurt. The danger is when, in trying to protect our egos from being battered, we shift our goal from Finding the Truth to Proving Ourselves Right and shift the facts to fit our story rather than shifting the story to fit our facts.

Anticipatory regret, Bruce Springsteen, and me

A few weeks ago, I kinda, sorta, not really, almost caught a free Bruce Springsteen concert. Except Springsteen didn’t show because he was never supposed to perform in the first place. I spent five hours standing around, surrounded by drunken Duke undergrads (and carefully avoiding eye-contact with those whom I’d taught or mentored) for nothing, all because of anticipatory regret.

the desire to avoid eye contact was mutual

The desire to avoid eye contact was mutual

Let me explain. Every year, Duke undergrads celebrate their Last Day of Classes, or LDOC, with lots of partying and a free outdoor concert. Past LDOC performers have included Kanye West, B.O.B., Macklemore, and Kendrick Lamar. This year, the lineup was a bunch of artists that were not as famous (at least to me; I do not pretend to be even remotely cool about current music), leading students to wonder if the lineup was a ruse. Perhaps they had a secret, better headliner in reserve?

Here’s where Springsteen comes in. As LDOC drew near, rumors started circulating that Bruce Springsteen was going to give a surprise performance to close out LDOC. While this may sound like wild conjecture, let me present the following pieces of evidence.

  1. Springsteen’s daughter is a senior graduating from Duke this year.
  2. Springsteen was going to give a concert in nearby Raleigh the very next night and was not slated for an official performance the night of LDOC.
  3. The slogan for LDOC was “Saving the Best for Last”, and clearly Bruce Springsteen is the best!

Duke undergrads at LDOC. Photo from flickr user Theomania61

Before hearing these rumors, I had no intention of going to LDOC. Being surrounded by partying undergrads makes me feel tired and old and like I should be on the lookout for date rape. After I heard the Springsteen rumors, however, I couldn’t not go.

You see, if I didn’t go, and the rumors were true, I knew I would be devastated. I would forever kick myself about that time Bruce Springsteen surprised Duke with a free concert, and I didn’t go because I was a homebody who didn’t want to stay up past my bedtime.

In other words, I anticipated the regret that I would feel if I had passed on the possibility of seeing Springsteen and he had actually shown up (angry streaming tears face below). This regret would be more painful than the annoyed disappointment I’d feel if I went to LDOC and Springsteen didn’t show (upset tongue-out bunny below). After weighing those possible emotional outcomes, I chose to take a chance and go to LDOC to avoid an angry streaming tears situation.

AnticipatoryRegret

Springsteen photo from Bill Ebbesen on Wikimedia commons. Rosa photo by Jeff MacInnes. Emoji by Flickr user Windkoh

Anticipatory regret is a powerful motivator, and it works best when you know you’ll find out what might have been had you made a different choice. The Dutch Postcode Lottery is a famous example. It randomly picks a street or neighborhood in the Netherlands, and if you live in that selected area, you win big Euros – IF you bought a lottery ticket.

Put yourself in Dutch clogs for a second. Imagine that you didn’t buy a postcode lottery ticket, and then your neighborhood got picked to win. How much regret would you feel, knowing that you missed out those winnings? That anticipatory regret would likely compel you to buy a postcode lottery ticket, just in case.

Photo from flickr user anonneymouse1

Photo from flickr user anonneymouse1

In U.S. lotteries, if you don’t buy a ticket, you’ll probably never know how close you came to hitting the big win, and you know that you’ll never know. No anticipatory regret in that case, so less impetus to purchase U.S. lottery tickets.

We often think of decision making as being driven by the emotions we experience while we’re trying to make up our minds. Anticipatory regret is a nice reminder that it’s not just the emotions that we feel in the moment – the emotions we think we might feel alter our decisions as well.

I’m still sad that I didn’t get to see Springsteen perform. But hey, Duke Commencement is this weekend. Maybe I’ll run into him around town while he’s here to watch his daughter graduate.

ResearchBlogging.org

Marcel Zeelenberg (1999). Anticipated regret, expected feedback and behavioral decision making Journal of Behavioral Decision Making

Smartphone games teach scientists how to save lives

These days, the ubiquity of smartphones makes it easy to call up a quick game of Angry Birds or Candy Crush and spend a few fun but ultimately unproductive minutes swiping around the screen. Duke psychology professor Steve Mitroff has discovered that the massive amounts of data generated by millions of people ostensibly wasting time can actually help save lives and protect national security.

One night, while waiting for his one-year-old daughter to fall asleep, Dr. Mitroff found a smartphone game called Airport Scanner, in which players man airport x-ray scanners and search for illegal objects amongst cluttered cartoon suitcases. The game dynamics paralleled Dr. Mitroff’s research on how we find specific visual targets amid complex visual scenes. This caused him to wonder if the cartoonish smartphone game, and the big data that it generated, could serve as an avenue for scientific research.

Fig 1 from the paper: Airport Scanner game screenshots

Figure 1 from the paper: Screenshots of Airport Scanner game’s x-ray baggage displays. Can you spot the illegal items?

Fortunately for Dr. Mitroff, Ben Sharpe, the CEO of the company that made Airport Scanner, was pleased to hear that his company’s game could not only provide entertainment but also benefit scientific inquiry. Sharpe had his programmer tweak Airport Scanner’s code so that anonymized gameplay data would automatically download to a server that Dr. Mitroff’s research team could access, and a fruitful collaboration began.

CC2.5 via Wikimedia Commons user Duke

A real x-ray baggage screening display. Can you spot the illegal items? From Wikimedia Commons user Duke, via CC-BY-2.5 license

Most recently, Dr. Mitroff has used Airport Scanner to study how we search for “ultra-rare” items, or items that appear in less than 1% of searches (pdf of paper published in Psychological Science). In regular laboratory studies of visual search, the rarest testable items appear 1% of the time. Because people’s attention spans are limited, it is difficult to ask people to perform more than a few hundred searches in the lab. Consequently, items that appear 1% of the time provide just a handful of datapoints per person, and items that appear less frequently are virtually impossible to study.

Airport Scanner, on the other hand, allowed Dr. Mitroff to study millions of searches, including those with ultra-rare items that appeared as infrequently as 0.08% of the time. Even with such low rates of appearance, the big data generated by Airport Scanner players provided hundreds to thousands of instances for each ultra-rare search – enough datapoints to draw valid statistical inferences.

Figure 3 from the paper: How often targets (illegal items) were detected plotted by how frequently they appeared in the game.

Figure 3 from the paper: How often targets (illegal items) were detected plotted by how frequently they appeared in the game.

It turns out that people are disproportionately terrible at finding ultra-rare items. While items that appeared in 1% to 3.7% of searches were found 92% of the time, ultra-rare items that appeared in less than 0.15% of searches were only found 27% of the time, even by experienced players. If you are a cancer patient with an ultra-rare cancer marker, or a Transportation Security Officer (TSO) searching for an ultra-rare terrorist threat in packed luggage, this low success rate is bad news with life-threatening implications.

Dr. Mitroff’s research points to a way to remedy this poor result for ultra-rare item searches: artificially increase the frequency of ultra-rare items and thus make them no longer ultra-rare. Airport scanners already project false illegal items onto luggage x-rays for TSOs to find. This keeps TSOs vigilant and provides performance feedback. By tweaking existing algorithms to project more ultra-rare items – more fake bombs and fewer fake forgotten pocketknives – Dr. Mitroff notes that we can change “how likely [TSOs] are to see something.” Similar techniques can be applied to cancer screenings to give radiologists more practice detecting ultra-rare cancer markers.

So, the next time you find yourself wasting time or procrastinating with a smartphone game, you can reassure yourself that you are not just wasting time – you are working to generate data that could benefit science, or even save lives.

 ResearchBlogging.orgMitroff, S., & Biggs, A. (2013). The Ultra-Rare-Item Effect: Visual Search for Exceedingly Rare Items Is Highly Susceptible to Error Psychological Science, 25 (1), 284-289 DOI: 10.1177/0956797613504221

How your brain helps you get your groove on

I wrote this for a science writing competition. I did not win (here is the piece that won for the journal article I chose to explain), so I am publishing my entry here.

The paper (it’s open access, so no paywalls!): Grahn, J.A. & Rowe, J.B. (2013) Finding and feeling the musical beat: Striatal dissociations between detection and prediction of regularity. Cerebral Cortex23(4), 913-921.

The abstract: I’m going to start putting these at the end of my post because I am conceited and think you should read my explanation first.

At some point in our lives, we have likely all encountered That Guy or That Gal who just can’t seem to dance to a beat. That Guy is often found on the dance floor at a wedding reception, flailing his limbs with wild abandon but no regularity. That Gal may be in your Zumba class, perpetually a quarter to a half-second behind the music while the rest of the class is shaking their hips in sync. Why can’t That Guy and That Gal just get with the beat?

Hint: look towards the back of the room.

Did you find That Gal? Image from Brittany Carlson, Wikimedia Commons

Being able to recognize a musical beat requires two things. First, one must be able to find the beat – the steady time intervals that underlie a piece of music or rhythmic sequence. Second, one must be able to maintain an internal sense of that beat as the music continues.

The ability to find a beat lets you start dancing in time to the music, while the ability to maintain a beat lets you continue moving your limbs in a regular fashion as the music continues. Researchers at the University of Western Ontario and Cambridge University have found that we use one part of our brain, the cerebellum, to look for the beat, and another part of our brain, the basal ganglia, to maintain it.

In the study by Drs. Jessica Grahn and James Rowe, 24 healthy adults listened to a series of rhythms while their brain function was measured using magnetic resonance imaging. Some of the rhythms had underlying beats while others did not, and the rhythms with a beat were of a variety of tempos.

In a clever design twist, the different rhythms were consecutively strung together so that sometimes the beat stayed the same while the rhythm changed, sometimes the beat became faster or slower, sometimes the beat disappeared entirely, and sometimes a beat emerged from a beat-less rhythm. This design allowed the experimenters to measure how brain function changed when participants searched for a beat when there was none to find or when an existing beat was continued or altered in a new rhythm.

I also developed a strong lower back.

I developed excellent beat finding skills on my college drumline.

When participants listened to rhythms that lacked a beat, their cerebellum, a brain structure that sits right above the spinal cord, was more active than when participants listened to rhythms with a beat. This finding suggests that the cerebellum is looking for a beat before a regular rhythm can be detected. Scientists already know that the cerebellum helps coordinate our motor movements, and this study suggests that it does so in part by paying extra attention to irregular timing.

In contrast, another part of the brain called the basal ganglia was more active when listening to beat rhythms than when listening to non-beat rhythms. Furthermore, its activity level differed depending on the relative tempos of the rhythms. It was active when a beat changed tempo to become faster or slower, it was even more active when a beat maintained the same tempo but switched to a new rhythm, and it was most active when both the current beat and current rhythm stayed the same.

My, what lovely ROIs you have!

The basal ganglia (in hot colors) maintains our sense of the beat. From Figure 7 of the paper.

Finally, the basal ganglia did not differentiate between a non-beat rhythm and the first appearance of a rhythm with a beat, indicating that the basal ganglia do not pay attention to the emerging presence of a beat when there was none before. This graded level of activity in the basal ganglia suggests that it works to maintain our sense of the beat and favors continuity and similarity – it works the hardest when it meets a reliable rhythm and slacks off a bit as the rhythm changes.

Note that putamen is NOT pronounced put-a-men.

Structures in the left and right basal ganglia show no significant responses to the presence of a new beat but do show significant responses to existing beats, even if they’re changing. From Figure 7 in the paper.

Scientists already know that the basal ganglia play an important role in generating and maintaining movement, as well as in learning and reward processing. Movement disorders such as Parkinson’s disease, in which patients have difficulty initiating movement, and Huntington’s disease, in which patients have difficulty stopping and controlling movements, are both linked to damage to the basal ganglia.

Understanding exactly how different parts of the brain enable us to control and coordinate our movements helps scientists figure out what deficits to expect from people with neurological damage and how those deficits could be treated. As a result of this study, we now know that healthy basal ganglia let us maintain our sense of regular rhythm – a sense that is important for anything that involves syncing movement to a regular beat, including walking, speaking, and, of course, dancing like a champ.

The abstract: Perception of temporal patterns is critical for speech, movement, and music. In the auditory domain, perception of a regular pulse, or beat, within a sequence of temporal intervals is associated with basal ganglia activity. Two alternative accounts of this striatal activity are possible: “searching” for temporal regularity in early stimulus processing stages or “prediction’ of the timing of future tones after the beat is found (relying on continuation of an internally generated beat). To resolve between these accounts, we used functional magnetic resonance imaging (fMRI) to investigate different stages of beat perception. Participants heard a series of beat and nonbeat (irregular) monotone sequences. For each sequence, the preceding sequence provided a temporal beat context for the following sequence. Beat sequences were preceded by nonbeat sequences, requiring the beat to be found anew (“beat finding” condition), or by beat sequences with the same beat rate (“beat continuation”), or a different rate (“beat adjustment”). Detection of regularity is highest during beat finding, whereas generation and prediction are highest during beat continuation. We found the greatest striatal activity for beat continuation, less for beat adjustment, and the least for beat finding. Thus, the basal ganglia’s response profile suggests a role in beat prediction, not in beat finding.

ResearchBlogging.orgGrahn, J., & Rowe, J. (2012). Finding and Feeling the Musical Beat: Striatal Dissociations between Detection and Prediction of Regularity Cerebral Cortex, 23 (4), 913-921 DOI: 10.1093/cercor/bhs083

Off to Shanghai!

Last week, I was tweeting up a storm at the ScienceOnline Together unconference, which was a 30-minute drive away. This week, I’m jetting off to Shanghai for a neuroeconomics conference, which is 24+ hours of total travel time (blech).

I’m not so excited about the epic travel boredom, but I am excited to get to meet some of my science heroes! Paul Glimcher published what is probably the first neuroeconomics papers ever and hasn’t stopped publishing since. He’s one of the authors on the adolescent decision-making paper that I wrote about here. Elizabeth Phelps has done some amazing work on the influence of emotions on decision-making, as well as some neat stuff looking at how fear shapes memories.

Funny that they’re at NYU, but I’m going all the way to China to hear them speak. Hopefully I will learn a lot. I definitely plan to eat a lot. And I hope I don’t breathe in pollution a lot. I will not be Tweeting a lot because Twitter is banned, as is Facebook and the NY Times. Sigh, China.

Image