Your job as reviewer is part of the larger JEDM editorial process described here.
A scientific paper can be distilled into four parts:
The author should clearly communicate items 1-3 to justify the claims they are making.
In your role as reviewer, you are responsible for three tasks:
You can only accomplish these tasks to a reasonable standard if the paper is in your area of expertise, you agree to adhere to JEDM’s adopted ethical guidelines for reviewing, and you have the time to perform the review.
Your review will have three parts. The comments to the authors, the comments to the editor, and a recommendation.
The bulk of your review will be comments to the authors; it should have the following components:
The summary should be in your own words and describe the motivation, methods, and results in about a paragraph. The purpose of the summary is to position the rest of your review, so restating the abstract is inappropriate.
Major issues are issues that you feel must be addressed in order for the paper to be scientifically sound. A major issue is:
You should point to specific figures, paragraphs, or results for each major issue and be concrete about the problems. Vague criticisms are unacceptable. When possible, use references from previous literature to back up your claims. Your summary of major issues can be a list or in narrative form.
Additionally, we appreciate it if you can give comments regarding fairness, equity, and positive social impacts of the submission when appropriate, see the guidelines worked out for the EDM 2022 conference.
Minor issues and typos should be a bulleted list. Minor issues are issues that make the paper less useful/effective in an objective or subjective sense.
Objective minor issues include poor grammar, problems with figure axes, misnumbering of figures, table formatting, consistency of notation in equations, or slightly incorrect citations (e.g., off by one year).
Subjective minor issues include wordy/terse sections of text, choice of a particular notation in equations (e.g., subscript should be p rather than k), and certain elements of presentation (which analysis to present first).
All minor issues should be mentioned, even ones you think are subjective. However, as you describe minor issues, you may wish to use different language for objective vs. subjective issues, e.g., “The authors should…” vs. “The authors might…”
Although it is not your job to point out every typo, pointing out typos is useful as JEDM does not have funding for professional copyediting. It is expected that all authors will use spelling/grammar checking software before submission, so any typos you see should have already passed these checks.
Here are some things that your comments to the authors should not contain:
Remember that this is a professional document. They are anonymous for the authors (you don't have to sign your name) but the Action Editor and Editor will see the review and your reputation may be affected by the quality of the work you perform in a positive or negative way. There is no reason to be rude, competitive, or snarky in a review.
If you think you have covered everything in your comments to the authors you may leave this field blank. If you do put any text in, it should be no more than one paragraph. It should not contain any criticism of methods/results that you did not put in your comments to the authors. It may include a statement of how interesting you think the paper is and how appropriate it is for the journal readership if it helps justify your decision.
If you believe the authors have violated JEDM’s ethical guidelines for publication, you should describe your concerns in this section. However, you are also encouraged to contact the Editor immediately to discuss your concerns.
You have three basic options for the recommendation:
Accept if you think the paper is ready for publication in its current state. This typically only happens after one or more rounds of review.
Choose Revisions required if you think the authors can address your major issues (Major revisions) or minor issues (Minor revisions). Do not select revisions required if you think it is unlikely the authors can address your issues. Do not select revisions required if you would not accept the paper even if all your issues were addressed. You should only choose revisions required if you would agree to review the revised paper.
Reject if you think major issues probably cannot be addressed or the paper does not advance the field in a meaningful way. If you think the idea might have promise but you’re not sure if the authors can bring it about, then you may Reject with an invitation to resubmit. You will not be asked to review the resubmission. If you don’t think the idea has promise, you should fully Reject.
Occasionally a paper will receive a recommendation of Major revisions in a first round and a Reject in a second round. While this is sometimes unavoidable, it should only happen when a fatal flaw is discovered during revisions rather than because the initial recommendation was off.
Remember that your recommendation is one of several. If all reviewers make the same recommendation, the Action Editor’s job is easy. But sometimes reviewers make very different recommendations, and the Action Editor’s job is more difficult. If you think that other reviewers will disagree with your recommendation (e.g., you really like a paper but doubt other reviewers will like it so much), you should consider a longer justification for your recommendation that incorporates these other points of view. This justification could go into comments for the editor. Because different reviewers may give different recommendations, you should never put your recommendation in comments to the author. Your recommendation is for the Action Editor, and the Action Editor/Editor make the editorial decision.
The review should be 1-2 pages long, depending on the issues you find. If you take great pains to proofread a paper (which is not your job) then the review would correspondingly be longer.
You should be:
Unless the paper was outright rejected or accepted, the authors will have a chance to respond to your review. If you have followed the guidelines above, it should make the re-review process more straightforward:
The difference between objective and subjective minor issues becomes more relevant during re-review. If the authors make a reasonable argument for retaining what you perceive to be a subjective minor issue, then that minor issue could reasonably be considered resolved.
New major or minor issues can arise during re-review. It may be that the author made a clarification in revision that makes a new issue apparent, or it may be that another reviewer has identified an issue that you are also concerned about. This sometimes happens; just describe the new issue and explain why you didn’t mention it before. Again, it is possible to reject in a later round if a fatal flaw is revealed in revision.
Remember that your recommendations during re-review, as in review, are to the editorial team and not to the author. In the narrative portion of you review, you may write “The authors have successfully/fully addressed my comments,” but please do not write something like, “I recommend accepting the paper.”
These guidelines are based in part on https://github.com/jtleek/reviews and are accordingly released under the GNU General Public License v2.0.