Igor Spasić 10 年之前 Nice, i love JMH - finally a good microbenchmark library. However, one thing I would do differently is NOT using loops in the tests (imho thats a benchmark-anti-pattern. For example, benchmark "serviceTracker_getServices()" have a iterator loop to also process 4 events. Is this really what you want to measure or **just** call to getService()? If it is just a call to getService() then remote the loop and use eg BlackHole to consume the returned service. Cheers! 请登录以投票。 以……回复 取消 Ray Augé Igor Spasić 10 年之前 The point of the test is to determine the performance of obtaining the "collection" of event handlers, and also the relative performance of their iterator implementations.Not all iterators are equal.In order to eliminate the effects a loop would have on the result, the loop size is fixed.Finally, getService is specifically _not_ what I wanted to test since that is only to be used when you want to get the "one single" matching service rather than "all" matching, which is what you want for event handlers (we need to process every tracked event handler).I wanted to compare it against our existing implementation of EventsProcessorUtil.process() method. You can't do that unless you account for iteration over the collection.HTH 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Got it. Still, I would not use the `for` loop in the benchmark tests. Loop size is not the only thing that affect the benchmark; JVM might optimize it with the loop unrolling. For example, that is why all previous benchmark tools that used the for loop for iterating benchmark code (like eg google caliper) are not correct. Instead, I would manually unroll the loop or use just one element in collection and explicitly get the first one, without any looping. That is all what I wanted to say ;) 请登录以投票。 以……回复 取消 Ray Augé Igor Spasić 10 年之前 But it must use a loop! Otherwise I cannot compare directly to the origjnal implementation (which internally uses a loop). Also, it must also return "more than one" event listener otherwise the collection behavior and iterators are not compared. If I want to have benchmarks which compare directly against the original impl they must emulate the exact behaviour.I completely understand the loop issue. But in this test (I did ask directly the developer of JMH if the design was a problematic, and he said "No") I really need to include the loop but how the looping issue is eliminated is to ensure each loop is identically oriented such that it's unrolling does not affect the outcome of the test (each having identical unrolling means that cost is constant and therefore does not negatively impact the test). 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Ok, all clear now, thank you! 请登录以投票。 以……回复 取消
Ray Augé Igor Spasić 10 年之前 The point of the test is to determine the performance of obtaining the "collection" of event handlers, and also the relative performance of their iterator implementations.Not all iterators are equal.In order to eliminate the effects a loop would have on the result, the loop size is fixed.Finally, getService is specifically _not_ what I wanted to test since that is only to be used when you want to get the "one single" matching service rather than "all" matching, which is what you want for event handlers (we need to process every tracked event handler).I wanted to compare it against our existing implementation of EventsProcessorUtil.process() method. You can't do that unless you account for iteration over the collection.HTH 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Got it. Still, I would not use the `for` loop in the benchmark tests. Loop size is not the only thing that affect the benchmark; JVM might optimize it with the loop unrolling. For example, that is why all previous benchmark tools that used the for loop for iterating benchmark code (like eg google caliper) are not correct. Instead, I would manually unroll the loop or use just one element in collection and explicitly get the first one, without any looping. That is all what I wanted to say ;) 请登录以投票。 以……回复 取消 Ray Augé Igor Spasić 10 年之前 But it must use a loop! Otherwise I cannot compare directly to the origjnal implementation (which internally uses a loop). Also, it must also return "more than one" event listener otherwise the collection behavior and iterators are not compared. If I want to have benchmarks which compare directly against the original impl they must emulate the exact behaviour.I completely understand the loop issue. But in this test (I did ask directly the developer of JMH if the design was a problematic, and he said "No") I really need to include the loop but how the looping issue is eliminated is to ensure each loop is identically oriented such that it's unrolling does not affect the outcome of the test (each having identical unrolling means that cost is constant and therefore does not negatively impact the test). 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Ok, all clear now, thank you! 请登录以投票。 以……回复 取消
Igor Spasić Ray Augé 10 年之前 Got it. Still, I would not use the `for` loop in the benchmark tests. Loop size is not the only thing that affect the benchmark; JVM might optimize it with the loop unrolling. For example, that is why all previous benchmark tools that used the for loop for iterating benchmark code (like eg google caliper) are not correct. Instead, I would manually unroll the loop or use just one element in collection and explicitly get the first one, without any looping. That is all what I wanted to say ;) 请登录以投票。 以……回复 取消 Ray Augé Igor Spasić 10 年之前 But it must use a loop! Otherwise I cannot compare directly to the origjnal implementation (which internally uses a loop). Also, it must also return "more than one" event listener otherwise the collection behavior and iterators are not compared. If I want to have benchmarks which compare directly against the original impl they must emulate the exact behaviour.I completely understand the loop issue. But in this test (I did ask directly the developer of JMH if the design was a problematic, and he said "No") I really need to include the loop but how the looping issue is eliminated is to ensure each loop is identically oriented such that it's unrolling does not affect the outcome of the test (each having identical unrolling means that cost is constant and therefore does not negatively impact the test). 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Ok, all clear now, thank you! 请登录以投票。 以……回复 取消
Ray Augé Igor Spasić 10 年之前 But it must use a loop! Otherwise I cannot compare directly to the origjnal implementation (which internally uses a loop). Also, it must also return "more than one" event listener otherwise the collection behavior and iterators are not compared. If I want to have benchmarks which compare directly against the original impl they must emulate the exact behaviour.I completely understand the loop issue. But in this test (I did ask directly the developer of JMH if the design was a problematic, and he said "No") I really need to include the loop but how the looping issue is eliminated is to ensure each loop is identically oriented such that it's unrolling does not affect the outcome of the test (each having identical unrolling means that cost is constant and therefore does not negatively impact the test). 请登录以投票。 以……回复 取消 Igor Spasić Ray Augé 10 年之前 Ok, all clear now, thank you! 请登录以投票。 以……回复 取消