What to do after a rejection

Benjamin Franklin wrote, “in this world nothing can be said to be certain, except death and taxes” (though the earliest origin of that idea dates to Christopher Bullock in 1716, apparently). Most academics would agree that paper rejections also belong on that list. My 10 published papers have been rejected a total of 29 times. I also have two “archived” papers that were collectively rejected eight times before I gave up on them and four working papers that so far have been rejected seven times (two are now revise-and-resubmit, so the rate of rejection is decreasing). So I have a total of 44 rejections. I have ZERO papers that got a revise-and-resubmit at the first journal I submitted them to (= each of my papers has been rejected at least once). I’m not even counting conference and grant rejections.

Paper rejections come in many shapes and sizes: your run-of-the-mill “Nice paper, but not enough of a contribution for this journal” or “Too many little things wrong” rejections; a reviewer finding something genuinely wrong with your manuscript; boilerplate desk rejections; a half-page report from a lazy reviewer who clearly hasn’t read your paper; and the frustrating “I just don’t believe your results” rejection. Rejections don’t feel good, but given that they are inevitable, it’s important to learn how to deal with them and move past them as quickly as possible. Below, I provide some suggestions that have worked for me.

First, allow yourself to take a few days to “mourn” the decision. A few days of inaction after a rejection won’t make much of a difference. I typically don’t even read the referee reports closely until it’s been a few days because I’m not confident in my ability to take in the feedback objectively. By all means, trash-talk the referees to your colleagues (people at your institution almost surely won’t be asked to review your papers), join the “Reviewer 2 must be stopped” group on Facebook (especially if you don’t know what “Reviewer 2” refers to), have a drink or two (please drink responsibly), do some exercise, work on another paper, or binge-watch that show you’ve been waiting to see. Do be careful how you discuss your reports online or at conferences because you never know who your reviewers were or who might know who your reviewers were.

It is hard not to take rejections personally, but in the vast majority of cases, they are not. The reviewers rejected your paper, they did not reject you as a person or a researcher. Even the comments about your paper may not have anything to do with the quality of your paper. Some reviewers might strongly dislike a particular methodology or research area, others may have had a bad day or week, and some may operate in toxic environments where unnecessary harshness is disguised as “honesty”. Your reviewer may have been a graduate student doing a referee report for the first time or a senior professor drowning in service work. Almost everyone has a “Reviewer 2” story, including some of the best researchers, and you are not alone. If a reviewer seemed particularly unfair, talk to a senior colleague about appealing the decision. However, appeals are definitely not the standard way to deal with rejections.

Next comes the time for actual work. Unless the journal rejecting your paper was your last stop before you were going to abandon efforts to publish it, try to return to the reports within a week of the rejection and look at them objectively. It can be tempting to either (1) ignore the reports completely and send the paper back out as soon as possible or (2) treat the reports as a revise-and-resubmit and try to address all the reviewer’s comments. Neither approach is generally a good idea, for two reasons.

First, you may get the same reviewer again. In some fields, reviewing the same paper twice is not acceptable, so you may get a different draw in that case. But in economics and surely some other fields, it’s not uncommon for the same person to review the paper two or more times at different journals. In such a case, the best you can hope for if you didn’t change anything in your paper is that the reviewer will return the same report to the editor. But it’s also possible that the reviewer will be annoyed that you did not take into account any of the comments they worked hard to give you and treat your paper more harshly than the first time around. In short, you want to avoid giving the impression that you thought the comments so worthless that you did not address even one.

Second, even if you’re 100% sure you’re not going to get the same reviewer, it’s highly unlikely that the reviewers’ comments were completely idiosyncratic or idiotic. If you ignore a comment that you could have addressed and a subsequent reviewer has the same concern, your paper could end up rejected again for avoidable reasons. Despite all the “Reviewer 2” stories out there, I think the overall peer review process is far from completely broken, so it’s also very unlikely that all the comments are useless and wrong. In short, the best way to treat the reviewer reports following a rejection is as an opportunity to make your paper better.

When deciding whether to address a particular comment, I ask myself two things: (1) How likely is this comment to come up again? and (2) How easy is this for me to address? The higher the comment is on this two-dimensional likelihood-ease scale, the more you should jump at the chance to address it. Whether something is likely to come up again or not is the hardest question to answer. Here, thinking about comments you’ve gotten at conferences or asking colleagues for their feedback on a particular comment can be really helpful. Rigorous self-honesty helps too: with some introspection, most of us will be able to identify comments where the reviewer really does have a point. Once you’ve identified all the relevant comments, start addressing them one by one. Where to stop can be difficult to tell, but if you start with the comments that rank high on ease and/or likelihood, you can stop at any point with the knowledge that you’ve addressed the most important ones. For me, a good rule of thumb is that the paper should be ready to go back out within 1-3 months or less of not-full-time work (this is probably equivalent to about 1-3 weeks of full-time for me). Anything more than that is likely to be excessive in most circumstances.

I’ll wrap up with two specific suggestions. If a reviewer comment makes it seem like she or he misunderstood something about what you’re doing, try to see if you can make that part of the paper clearer. You have the privilege of knowing your paper better than anyone else, so what seems clear to you may not be to the average reader. If there is a comment that seems likely to come up again but would be really difficult to address, you have a few options. You can add a brief explanation as to why doing X would be difficult, possibly as a footnote, possibly as a suggested avenue for future research. This signals to reviewers that you are aware of X. Relatedly, you can hint that you could do X but it’s outside of the scope of the current paper. That allows a persistent reviewer to insist on seeing X in a revision but reduces the likelihood that they reject the paper because you didn’t already do X.

In the end, these steps don’t necessarily make rejections more pleasant, but they do move your paper closer to published!

How to pick which journal to submit your manuscript to

There are many high-quality journals out there and choosing which one to submit your paper to can be a daunting task. Below, I offer some suggestions.

A great starting point is your manuscript’s reference section. Identify the papers most closely related to yours and tabulate the journals that published them. Think of a few more distantly related papers that may not have made it to your references and add the journals they are published in to your list. If your list is short or, by contrast, there are too many options, look at your advisors’ CVs for guidance. If you are still not happy, look at CVs of colleagues working in related fields.

Once you have identified at least five potential journals, go to each journal’s website and ask yourself: how often does this journal publish work similar to mine (in terms of subject, research methods, etc) relative to other journals on my list? The less frequently a journal publishes papers in your research area, the lower your chances of acceptance. If the journal rarely publishes related papers, you may want to try somewhere else first.

Are there exceptions to this rule? Yes. If the journal has a new editor who works in your area, your chances are probably higher than historical publication information may indicate. Browsing the editorial board of a journal can thus help you assess your publication chances as well. The best sign is if your paper cites the work of at least one of the editors in a positive light. Not only does this mean they are more likely to handle your paper, but they are also likely to view it more favorably than someone outside the field. (Unless you’ve tried to disguise a bad paper as a good paper. But you wouldn’t do that, of course.)

Next, you want to ask yourself, how quickly does each journal process the average submission and how much time do you have? If you’re approaching a milestone like tenure or if there are many other people working on the same topic and you’re worried about being scooped, you may want to prioritize journals that turn papers around quickly. Sometimes journals publish these statistics (e.g., number of days to first decision). Other times, you may have to ask colleagues about their experiences with particular journals.

Another obvious consideration is the ranking/visibility of the journals on your list and your own goals for the paper. If you are trying to get tenure, prioritize the journals that are more valued by departments where you could plausibly get tenure (my suggestion is to not put all your eggs in one basket and ignore the idiosyncratic preferences of your current department unless you have a really really good reason for doing so). If you’re trying to maximize the impact of your work, consider which journals are most respected by people in your field. These are often correlated with general journal rank, but there may be some divergence.

Finally, think about how much rejection you want to take. On average, the more competitive the journal you have chosen, the longer it will take to publish the paper and the more likely you are to receive negative feedback. I’m personally of the opinion that the rejection process can be used productively and would thus recommend toughing it out if you have time, but this approach may not be right for everyone. For better or worse, rejections are inevitable for all of us, and learning how to deal with them is part of the academic career. More on that in another post!

Has it gotten harder to publish in top economics journals?

In a fascinating series of graphs, David Card and Stefano DellaVigna document that submissions to the top 5 economics journals have gone up from about 3,000 per year in the early 1990s to over 7,000 per year in the mid-2010s, while acceptance rates have decreased from 10-15% to 2.4-5.6%. A natural interpretation of these trends is that it has gotten much harder to publish in top journals, all else equal. But could there be more to this story? There are (at least) three reasons things are not as clear as they seem at first glance.

First, submitting a paper to a journal has certainly become easier since the 1990s. It used to be (or so I’m told) that you had to physically print out the paper, put it into an envelope, send it somewhere and wait for a return envelope, which meant that even desk rejections took a lot longer than they do now, on average. The whole process should have been inherently harder, which meant people may have thought twice before trying for a Hail Mary at a top journal. With more and more people having internet access and virtually all top journals accepting electronic submissions, I would predict that a lot more people are willing to shop their papers around for a while in hopes of hitting the best journal possible.

Second, the emphasis on having a top-5 publication seems to have increased steadily over the past 20 years or so. Of course, there could be reverse causality here – if publishing a paper in the top 5 is harder, then success should be valued more, on average. But I think it’s likely that the emphasis on the top 5 journals has caused more people to try to publish papers there that they would otherwise send someone else, driving submissions up and acceptance rates down. For example, authors may now try for three of the five journals instead of just one (I may or may not have done this myself), so a single paper will show up as three submissions in the statistics above.

Third, the graphs show that the number of authors per paper has gone up too, so conditional on the total number of published papers being constant, the total number of people with a top 5 publication should be going up!

But shouldn’t the first two trends mean it’s gotten harder to publish in top journals, as the original statistics imply? This is true only if the marginal papers are about as good as the average papers. And that is very unlikely to be the case. Given the high rates of desk rejections at many journals, is seems more likely that there are many people trying for a very unlikely outcome. In fact, it’s possible that the probability of a good paper getting into a top journal is essentially unchanged because each marginal paper has an incredibly tiny probability of acceptance. Of course, it’s hard to believe that the publishing process is that accurate, so it’s likely to have gotten at least somewhat more competitive at the top journals. But I highly doubt that the level of competition is anywhere close to 2-5 times higher, as the statistics imply.

The real question we should be asking is, are there more high-quality papers per high-quality journal (somehow controlling for general progress in methods, computing power, and data)? Or, if we’re interested in the likelihood of a person having a top-5 publication, are there more high-quality academics per high-quality journal? Both of these questions are really difficult to answer, but I found no statistics suggesting that there have been significant increases in the number of academic economists over this time period. For example, this website from 2006 says that “the number of new PhDs who intend to pursue careers in the US has declined.”. This report indicates that there were 24,886 AEA members and subscribers in 1999 and this one says there were 23,170 in 2017. My overall conclusion is that the decline in acceptance rates should not be interpreted as really bad news for good papers trying to get into good journals.

Just how significant are publication lags?

Generally, Academic Sequitur finds papers as soon as they are posted online. Increasingly, journals are posting papers as soon as they are accepted and correctly formatted (some even before then!), which means that when the “official” new issue is announced, the papers it consists of could have been hanging out on the web for months without being publicized. This month, I checked just how large this lag can be by seeing when Academic Sequitur found papers that were included in journals’ most recent issues.

I checked the top 5 economics journals: American Economic Review, Quarterly Journal of Economics, Journal of Political Economy, Econometrica, and Review of Economic Studies. For the January 2019 issue of AER, the included articles were all added to our database between July 10 and August 27, 2018 (corresponding to the dates they were posted). For the February 2019 issue of QJE, we found all the articles between August 20 and October 25, 2018. For the December 2018 issue of Econometrica, articles were found between June 4and December 12, 2018. For the December issue of the Journal of Political Economy, we located all articles between August 2 and November 21, 2018. Finally, for Review of Economic Studies, articles from the January 2019 issue were located between January 28, 2018 (yes, almost a year early!) and November 28, 2018.

Our users find out about articles at the time they are posted, not when they get grouped into an issue after languishing online for months. And I think that’s a huge plus!