Good reports should
- put the paper in context - is the subject well-studied? Or is it a backwater where people haven't worked in years? Will people want to read it?
- evaluate the paper - Is it a real breakthrough in the area, or just one in a series of similar results? Does the author introduce some new useful technique?
- evaluate the writing - is it clear? How could it be improved? Can arguments be restructured to be simpler and clearer? Are too many important subresults left to the reader?
- evaluate the bibliography - is it complete enough, or (in the other direction) are many irrelevant papers cited?
Here is an example of a really bad report:
This paper is of absolutely no interest. I showed it to my colleague, Professor X, and she agrees. I recommend rejection.
A good referee report should be useful to the author. This report doesn't tell the author anything that he/she can use to improve the paper. Is it bad because the problem addressed is too trivial? Or because the results are already known? What is an author expected to do after receiving a report like this? Commit suicide?
Here's another example of a bad report:
Tiling problems have been studied for many years. They are of great interest in combinatorics and logic. This paper is a good contribution to the subject, and I recommend acceptance.
A good referee report should be useful to the editor, too. This report doesn't tell the editor anything useful! Are the results really deep and novel? Or is it just another in a series of similar small results? Not only that, a report like this suggests strongly that the referee didn't really read the paper with care, and just skimmed the paper in a few minutes. Are there really no papers that the author missed citing? Are all the equations really correct in all respects? Is there nothing that could be improved?