Ross Girshick: writing good research papers

[210112] Ross Girshick: writing good research papers

website: https://zhuanlan.zhihu.com/p/104279421

What did I learn from your paper?

• A paper should be about a single focused idea or question.
• “Idea” usually means method. What should I learn?
  • Under what conditions does it work?
  • When does it work?
  • If the idea has multiple components, which is the most important one?
  • Which implementations are the most important?
• I seldom care if “your idea” + “unrelated ideas/tricks” --> S.O.T.A results
  • my first priority: to learn something interesting about your ideas.

one table, one message

Ross 用他的 Mask R-CNN 为例，论文中有很多表格，每张表都只表达1个信息。

Support all of your claims

所有的 claim 应该用 citation 或者 experiment 来支持。否则，不要出现不合格的表达，比如：

• “Intuitively, increasing X is important for Y…”
  This statement is your intuition (not fact), the reviewer may disagree!
• “Increasing X may lead to improved Y…”
  Expresses uncertainty or that some conditions may apply
  Ross举了一个例子，在Mask R-CNN论文中，论文中说alignment非常重要，因此提出了 RoIAlign，为了支持这个观点，有一个实验结果表格来支撑。

speed/accuracy claims

与当前算法的比较经常是不可控的（uncontrolled）。主要表现为如下五个方面：

• 使用不同参数时，算法准确率不同
• 不同参数及硬件环境中，算法速度不同
• inference details 不同时，算法速度不同
• 有些人是以优化算法速度为生的！他们能做到10-100倍速度提升
• ?
  因此，我们对于算法准确率及速度的提升保持怀疑态度（taken with a large grain of salt）。而我们在做实验时，注意下面四项即可：
• Making training settings as similar as possible
• Making inference settings as similar as possible
• Ensuring low-level optimization fairness
• Using the same hardware for all methods

implement all methods in one Codebase

Many good Codebase now for object detection: mmdetection, simpledet, detectron2, etc.
Use the same Codebase to the greatest extent possible.

avoid big tables of historical comparisons

With today’s best practice, the historical results has no meaning to be compared.