What's the correct way to report that B is x% better than A? I see 2 ways:

  • Way 1) x = (B/A - 1)
  • Way 2) x = (1 - A/B)

In case the goal is to maximize the value (for example: calculations per second):

  • A = 100
  • B = 130 (better)
  • Way 1) x = (B/A - 1) => B is 30% better than A
  • Way 2) x = (1 - A/B) => B is 23% better than A
  • C = 500 (best)
  • Note: C is 5 times better than A
  • Way 1) C is 400% better than A
  • Way 2) C is 80% better than A

In case the goal is to minimize the value (for example: number of seconds run):

  • A = 100
  • B = 70 (better)
  • Way 1) x = (B/A - 1) => B is 30% better than A
  • Way 2) x = (1 - A/B) => B is 42% better than A
  • C = 20 (best)
  • Note: C is 5 times better than A
  • Way 1) C is 80% better than A
  • Way 2) C is 400% better than A

Due to writing this down, it seems clear that way 1) is the way to report this. This is also they way I 've always used thus far. But is that accepted as the canonical way in the scientific research community? Or is there discussion/variance on how to calculate that something is x% better?

asked 13 Feb '14, 06:26

Geoffrey%20De%20Smet's gravatar image

Geoffrey De ... ♦
3.6k32764
accept rate: 6%

edited 13 Feb '14, 07:17


I would report your first example as either "B is 30% better than A" or "A is 23% worse than B", mainly because my brain hears "than" as "base case/standard follows".

link

answered 13 Feb '14, 15:08

Paul%20Rubin's gravatar image

Paul Rubin ♦♦
14.6k412
accept rate: 19%

2

I was about to share http://en.wikipedia.org/wiki/Relative_change_and_difference#Percentage_change – but the first sentence of @Ehsan's answer already covered that. It's basically a question of what you define as 'reference'.

(13 Feb '14, 15:32) fbahr ♦

I agree with Paul: the than implies that the right hand side is the reference. Although Ehsan's answer is probably reality (so you can't know what "B is 30% better than A" means if you read it in a paper)...

(14 Feb '14, 02:53) Geoffrey De ... ♦
1

@Geoffrey: While I understand your concern, I don't think the current practice of reporting academic results is that bad. First, reviewers almost always ask for your definition of gap if it is not available. Second, authors usually provide results of both methods A and B along the gaps. So even if they fail to mention gap formula, you could still determine it for yourself. Finally, if a paper is just reporting method B and the gaps without enough information on gap formula and method A, the paper lacks proper research methodology and transparency and probably not worth reading anyway.

(14 Feb '14, 03:18) Ehsan ♦

@Ehsan Newspapers's headlines (nor even articles) don't include the definition of the gap. Same goes for blogs, tweets, ...

(14 Feb '14, 03:28) Geoffrey De ... ♦

@Geoffrey: Well that's another story. I agree with you on that part.

(14 Feb '14, 03:47) Ehsan ♦

I think way #1 is the more commonly-used measure as it is saying "how far away is your solution from its competitor". Essentially, way #2 is saying "how far away is your best competitor from your solution". So, in option #2 you are somehow being more fair to your competitor. Another point is that way #2 is the measure mostly employed by optimization solvers such as CPLEX. I don't know why, but perhaps they do so to reduce display space. Maybe, people working on solvers (e.g., @Bo or @jfpuget) could elaborate on that.

In business settings, I suppose way #1 would be a better option for presentation (you employers would love the fact that you improved their profit by 400%, not that they were doing 80% worse). Based on my experience, in academia what different people might use is based on the solution methodology they usually use. I've seen that people working on heuristics usually use the first one, while the people working on exact methods use the second one (as they usually compare with solvers, they use the same criterion as solvers use). That's my observation, so its generalization might not be true. My final recommendation is to select the one more suited to your audience (i.e., way #1 for comparing heuristics and business settings, while way #2 for comparing exacts methods).

link

answered 13 Feb '14, 07:10

Ehsan's gravatar image

Ehsan ♦
4.8k31122
accept rate: 16%

edited 13 Feb '14, 07:13

That's depressing. But thanks for the very interesting answer :)

(13 Feb '14, 07:15) Geoffrey De ... ♦

Why depressing?

(13 Feb '14, 07:17) Ehsan ♦

Presume you read an article that states "B uses 80% less memory than A". So if A is 100, what is B? It can be 20, or it can be 55, depending on how they calculate. So it can be 5 times better or less than 2 times better.

(13 Feb '14, 07:24) Geoffrey De ... ♦

The main reason this is depressing is that opportunistic researchers can switch ways depending on the property to inflate the gain: use way 1 if they need to maximize the goal value and way 2 if they need to minimize the goal value.

(13 Feb '14, 07:26) Geoffrey De ... ♦
1

Against common belief, academic publishing relies heavily on presentation. The best theoretical or applied work with weak presentation is hardly accepted, while a shady work with great presentation has a better chance for acceptance. I think it is up to the reviewers and readers to notice such details and determine whether the research deserve recognition.

(13 Feb '14, 07:55) Ehsan ♦

We (at CPLEX) say B is 1.3x better than A, and C is 5x better than A.

Usually we use 'better' for 'faster' ;)

A related question is about the computation of the gap. Here each MP solver seems to have a slightly different way of computing it. It may be worth looking at this given it provides a percentage that says how far is the current best solution from the best dual bound.

link

answered 14 Feb '14, 04:39

jfpuget's gravatar image

jfpuget
2.5k310
accept rate: 8%

edited 14 Feb '14, 04:41

Agreed that avoiding the use of "%" avoids the ambiguity :)

(14 Feb '14, 04:53) Geoffrey De ... ♦

IMHO, it's more accurate to say:

  • B is 0.3x better than A [or: B is 1.3x as good as A], and
  • C is 4x better than A [or: C is 5x as good as A].

[ I. e.,

  • n times better than ⇒ A+n*A;
  • n times as good as ⇒ n*A ]
(14 Feb '14, 05:23) fbahr ♦

@fbahr On this one, I disagree (and therefore agree with @jfpuget): "1.3x as good" doesn't parse well in my brain.

(14 Feb '14, 07:49) Geoffrey De ... ♦

@Geoffrey: That's why people _use_ "%" (change). [You probably won't have problems parsing a similar statement incl. 'many' and its comparative 'more': Fry has twice as many eyes as Leela, but he does _not_ have two times more (than Leela has).]

(14 Feb '14, 10:12) fbahr ♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×8

Asked: 13 Feb '14, 06:26

Seen: 8,193 times

Last updated: 14 Feb '14, 12:33

OR-Exchange! Your site for questions, answers, and announcements about operations research.