A new way of ranking journals*

Journal rankings can be controversial. At the same time, the quality of a journal in which one’s research is published is generally thought to be very important for a researcher’s career, and many researchers are thus rightly concerned about it.

Here at Academic Sequitur, we came up with a new way (to the best of our knowledge) to think about how a journal is perceived by the profession. This blog post focuses on the field of economics. We start with the premise that the top 5 economic journals (AER, Econometrica, JPE, QJE, and ReStud) are really the best in the profession, on average. Then we calculate what percent of authors who published in another journal have at least one “top 5” publication. The higher than number is, the more likely it is to be a high-quality journal. In non-top-5 journals, we considered all articles published since 2018 (results are similar if we start in 2009, when the AEJs were started). For calculating whether an author has a top-5 publication, we used top 5 articles since the year 2000.

Before I show you the results, it’s important to note that one thing which can affect this ranking is the topic of the journal under consideration. If a journal’s focus is not “sexy” enough for a top 5 journal, that’s likely going to lower its ranking. Whether this is a feature or a bug I will let you decide.

So with that, here are journals that come out on top, based on the overall proportion of authors publishing at least one article in any top 5 (the rest of the columns show the journal-specific proportions).

Four interesting things about these rankings: First, the shares are really high for eight out of ten of these journals, with about half the authors having at least one top 5. To me, this pattern suggests that we definitely shouldn’t overlook the non-top-5-journals when looking for quality articles. Second, these eight journals are pretty close to each other according to this metric, suggesting that quality differences between them are not large. Third, this metric largely aligns with what I think are the general perceptions of applied microeconomists in North America, with one exception: we seem to be giving the Journal of the European Economic Association less credit than it deserves. Fourth, the rankings would definitely change if we used specific journals for comparison rather than the overall top-5 metric.

Here’s the next set of journals. Keep in mind that we didn’t perform these calculations for all journals in our database (this is just a blog post, after all). So if you don’t see your favorite journal, that doesn’t mean it’s ranked lower than these. It just means we didn’t calculate a ranking for it. But if you’d like, leave us a comment and we’ll tell you how your favorite journal ranks!

While this metric is unlikely to be perfect, it is also unlikely to be worse than citation impact measures. And its benefit for economics specifically is that it isn’t as affected by publication lags as a citation-based measure. What do you think?

36 thoughts on “A new way of ranking journals*

  1. That’s a very good article. The way journals are ranked should be changed. Lots of articles published in non-high ranked journals are often of high quality, contribution & significance. Let alone the indexing issue.

      1. Top 5 econ journals. JPAM is now classified as an Econ journal by RePEC and ISI so I’m interested in how it does using this econ-centered metric.

        1. Better than I guessed (given the research focus, not the quality)! 15% of 2018-2019 article authors have a top 5 since 2000; 10% AER, 0.8% Econometrica, 5% JPE, 8% QJE, 1.6% ReStud.

          1. I agree, this could create some undesirable incentives. One way to minimize it is to rank based on subsequent top 5 publications. Whether or not journals would game something like this depends on how easy it is to predict who will publish in a top 5. Ultimately, I think the best way to use a ranking like this is to create a giant matrix of each journal relative to one another. That would allow the user to benchmark against one journal against another journal she considers “good”.

  2. I’m interested in the Marketing journals by this metric (compared to top 5 econ journals):

    Marketing Science
    Quantitative Marketing & Economics
    Journal of Marketing Research
    Management Science

    1. Marketing Science. Any top 5: 15%, AER: 12%, Econometrica: 1.3%, JPE: 5.3%, QJE: 4%, ReStud: 1.3%
      Quantitative Marketing & Economics. Any top 5: 20%; AER: 20%, Econometrica: 8.6%, JPE: 14.3%, QJE: 5.7%, ReStud: 11.4%
      Journal of Marketing Research. Any top 5: 5%, AER: 2.6%, Econometrica: 0%, JPE: 1.7%, QJE: 0.9%, ReStud: 0.9%
      Management Science. Any top 5: 17%, AER: 11.8%, Econometrica: 3.5%, JPE: 4.1%, QJE: 3.7%, ReStud: 5.7%

  3. Excellent post — I encourage you to make it into a paper (if only a short one).
    I am interested in the following list of good journals:
    Journal of Economic Theory
    International Economic Review
    Review of Economic Dynamics
    Journal of Money Credit and Banking
    Journal of Economic Dynamics and Control
    Also, I would be curious to see how “invited” outlets do (J. of Econ Literature and NBER Macro Annual for instance).
    Thank you!

    1. Thank you! I’ve thought about formalizing this in a paper, we’ll see whether the schedule allows it.
      Journal of Economic Theory. Any top 5: 43%, AER: 24.6%, Econometrica: 23.2%, JPE: 12.5%, QJE: 9.8%, ReStud: 22.6%
      International Economic Review. Any top 5: 38%, AER: 26%, Econometrica: 13%, JPE: 11%, QJE: 9.1%, ReStud: 20.1%
      Review of Economic Dynamics. Any top 5: 29%, AER: 20.5%, Econometrica: 7.6%, JPE: 12.9%, QJE: 7%, ReStud: 19.7%
      Journal of Money, Credit and Banking. Any top 5: 19%, AER: 14.4%, Econometrica: 2.2%, JPE: 4.6%, QJE: 2.8%, ReStud: 3.7%
      Journal of Economic Dynamics and Control. Any top 5: 21%, AER: 12.2%, Econometrica: 5.1%, JPE: 5.6%, QJE: 5.9%, ReStud: 11%
      Journal of Economic Literature. Any top 5: 75%, AER: 63.5%, Econometrica: 22.2%, JPE: 28.6%, QJE: 50.8%, ReStud: 39.7%

      So JEL is killing it (consistent with it being ranked #2 on RePEc: https://ideas.repec.org/top/top.journals.simple.html). Don’t have NBER Macro Annual, unfortunately.

      Updated table with all these & more journals forthcoming in a future blog post!

  4. What about if you looked at the overlap with political science? I.e. to what are generically considered the top political science journals: American Political Science Review (APSR), American Journal of Political Science (AJPS), Journal of Politics (JOP), Quarterly Journal of Political Science (QJPS)? This matters for those of us who do political economy work–and would be superhelpful!

  5. Interesting idea. There are two separate questions here:
    1) How much does the authorship of a journal overlap with that of the top 5 (which some note could just indicate how clubby it is)
    2) Is the journal a place where top-5 authors are also sending their papers?

    For 2) the numerator may be of more interest than the ratio. I would be interested in a statistic like the following (e.g. for development economics).
    – Take all top 5 articles published in development economics (e.g. with JEL field code O). Get this set of authors.
    – Then rank other journals by the number of articles published in them by that set of authors. E.g. perhaps 12 articles in the JDE in the past year were published by this set of top-5 authors, versus 7 in World Development and 4 in EDCC.

    This would tell you which journals the top-5 authors are most likely to also publish in.

    1. We could do something like that, although the calculation of what is a development article would be a bit different (we don’t collect JEL codes). But one issue is that some journals publish more articles than others, which is why I normalized by the total number of article-authors in a journal. What’s the benefit of not normalizing?

Leave a Reply

Your email address will not be published. Required fields are marked *