Five Laws of Choosing Referees

For the last few years I've been editing a mathematical journal - the Journal of Integer Sequences, which is a true open-access journal. Neither authors nor readers are charged; it is completely free.

Much of my time is spent dealing with referees: choosing them, trying to get them to agree to referee a paper, reading their reports, and sending them to authors. Along the way, I've had a number of interesting experiences: like the time I kept pursuing a referee despite the fact that he was dead, and the referee who sent me a report on a completely different paper that I had not sent.

I think I now have enough data to list five laws of choosing referees:

1. In general, famous mathematicians make lousy referees. (Of course, there are exceptions.) Famous mathematicians have a lot of demands on their time, which means they probably won't even answer your request to referee a paper. If they do respond, they'll probably say no, because they're so busy doing important work. If they do accept, they'll probably take a long time. Because they're famous, they're probably really bright - much smarter than me and probably the author of the paper - so if they do write a report, it tends to be really short, snarky, and dismissive of the results.

2. Graduate students make good referees in one respect: they tend to read the paper really carefully, and are good at spotting sections where the argument is incorrect or unclear. But they rarely have the mastery of the literature needed to know if something is new or original.

3. If you have to ask a potential referee several times whether they're willing to referee a paper, then don't bother phoning them or making much more effort to contact them. If they're so disorganized and impolite that they refuse to tell you yes or no quickly, then they'll never produce a report.

4. Referees from Asia tend - generally speaking - to write extremely short reports that are rarely helpful. Whether this is a function of the culture, or whether they're embarrassed by their English skills, or something else, I don't know. In contrast, eastern Europeans generally write good and helpful reports, despite their lousy English skills.

5. The worst referees of all are the ones that agree to referee the paper and then keep stringing you along for month after month, each time claiming that they're almost done the report, and it will be coming shortly. I once wasted 9 months pursuing a referee who kept claiming it would be there next week. So if you can't get a report from the referee within a month of the time they originally promised, give up - and tell the referee why you're giving up on them.