Questions tagged [benchmarking]
The benchmarking tag has no usage guidance.
40
questions
3
votes
1
answer
209
views
Benchmarking C# code - results
I watch this YouTuber's mostly excellent videos on .NET features (let's call him Nick). He more often than not uses Benchmark.net to demonstrate the differences in performance of different approaches ...
- 18.3k
29
votes
5
answers
4k
views
Benchmarking, why discard lowest time?
I've quite frequently seen benchmarks where the tester discarded the highest and the lowest time out of N runs.
Discarding the highest time I understand; it's probably high because of some other ...
- 499
3
votes
1
answer
304
views
What is the expected performance of While loops using `array.pop()` assignment vs other methods
Recently I was asked to refactor some code that leverages JavaScript's array.reduce() method because other developers felt the code hard to read. While doing this I decided to play around with some ...
- 149
-3
votes
1
answer
121
views
Methods for reliably benchmarking own program's run time
I'm writing a basic password cracker in C as an introduction to multithreaded programming. I've implemented this already using a 'parallelization' approach that spins up a set number of threads and ...
- 109
2
votes
3
answers
4k
views
Comparison of build times on various hardware; why non-linear results?
Why do the compile times not vary significantly between different era CPUs, even though disk (NVMe vs. HDD) and CPU benchmarks vary significantly in performance?
Why does disabling hyperthreading ...
- 345
2
votes
0
answers
75
views
How to write disk benchmarking software?
I am coding a disk (HDD/SSD) benchmarking software. I have a few questions on how a disk should be benchmarked (regardless of programming language used).
This is my pseudo code for testing sequential ...
- 175
3
votes
2
answers
1k
views
How to benchmark software performance under same conditions (RAM, CPU caching etc)?
I am writing a software that splits big files into smaller files and I have coded several solutions.
I am measuring the execution time of each of my solutions. (with threads, go routines, MPI etc) ...
- 247
-3
votes
1
answer
82
views
Handling different versions of a program and benchmarking with OO
Note: I'm not sure if the way I phrase the question initially will fit 100% on this board, so please help me to focus on the main point.
Scenario: You develop an algorithm or a neural network that ...
- 11
2
votes
1
answer
268
views
How do I triage a website effectively?
I've inherited responsibilities for maintaining a section of enterprise intranet, and my manager has tasked me with, "Clean up the code, please. We're probably going to get lots of change requests ...
- 207
3
votes
1
answer
271
views
How to compare C and CUDA performance fairly?
I have developed a CUDA code that is a reimplementation of a (CPU only) C code.
I would like to benchmark the performance of these two programs. I am wondering how to do benchmarking well in this ...
- 593
1
vote
2
answers
1k
views
Can I use statistical methods to compare benchmark performance figures?
Imagine I am working on a C compiler. I have made some changes to the compiler's code, and am interested to know what effect these changes have on the performance of generated binaries.
I have a ...
- 167
4
votes
4
answers
887
views
How to manage code experiments meant to stay permanently with git?
I have an asm/C code which implements some image filters. The objective of this project is experimentation with different implementations, benchmarking, plotting and reporting data to write a paper ...
- 143
5
votes
2
answers
120
views
Estimate if benchmark sampling difference is significative
In order to estimate the fastest function in various cases in a non fully deterministic context, I'm running an experiment calling one or the other at random and recording the duration of the ...
- 1,434
3
votes
1
answer
2k
views
Load testing HTTP server with large number of concurrent connections
I'm trying to load test/benchmark a http server with a very large number of simultaneous connections (10-100k). What is a good procedure for doing this? On linux I've seen that both the client and ...
- 131
1
vote
1
answer
570
views
When do micro-benchmarks make sense? [closed]
I've been researching micro-benchmarks (e.g. using JMH or Caliper, though not necessarily limited to Java). Now, there are many articles which describe how to write a benchmark but I haven't found ...
- 187
3
votes
5
answers
15k
views
Why Java is a factor 2-3 slower than equivalent C++ program? [closed]
I know there is an opinion that programs written in Java and running under JVM as as fast as C++ programs, after introducing just-in-time (JIT). I see many cases when Java is terribly slow for simple ...
- 181
7
votes
2
answers
2k
views
Where is the evidence for the performance of Node.js?
A lot of developers, especially ones who haven't used Node.js in production, seem to believe that Node.js is faster than other interpreted languages such as PHP, Python, and Ruby.
These claims ...
- 1,164
3
votes
3
answers
752
views
How does a software developer know if the target hardware is fast enough to run the software? [duplicate]
How do software developers manage to know if their software will run on the target hardware they are developing to? Many developers today work on machines much faster than the intended platform. What ...
- 41
1
vote
2
answers
192
views
Should benchmarks (to compare processors performance) be compiled with optimization CFLAGS?
I need to compare performance of 2 processors and I concluded I should benchmark then with several various tests.
I'm currently using
linpack (HPL) (because it's still well known and used for ...
- 113
3
votes
2
answers
1k
views
Dual pivot quicksort in face of expensive swaps
I was told this is better place to ask this
TLDR
Has anyone tested dual pivot quicksort performance with expensive-to-swap elements? It seems that in this case, it should massively underperform ...
- 133
0
votes
1
answer
261
views
Huge long addition overhead compared to Native call?
A run a simple test JDK 1.7.0_45 (Windows 7, 64bit):
Test 1:
long start = System.nanoTime();
for (int i=0; i < 1000000; i++) {
System.currentTimeMillis();
}
elapsed = System.nanoTime() - ...
- 11
1
vote
4
answers
276
views
Track performance numbers
I have a (large) software project in a git repository. There is a benchmark suite that spits out lots of numbers (time per benchmark, allocations done per benchmark) I’d like to track.
What is the ...
- 119
1
vote
1
answer
12k
views
Why execution time is different each time in java?
I am trying to make a program in Java which measure the complexity of a specific program with respect to its run time. Let's take a problem and code that problem in all possible ways with respect to ...
- 11
-1
votes
1
answer
109
views
What do I benchmark before updating a web site? [closed]
I am volunteering my time to update a website for a local business. The current website they have is not up to date and it fails quite a few "web tests".
For example, the pages are too large, the ...
- 511
3
votes
0
answers
431
views
Repeatability of CPU-intensive benchmarks on NUMA hardware
Consider a benchmark of a C++ application. It does little or no I/O in proportion to its overall runtime -- it is compute-intensive. It is a single-threaded program. It pages/reads in all of its data ...
- 1,698
19
votes
3
answers
20k
views
Fastest Functional Language
I've recently been delving into functional programming especially Haskell and F#, the prior more so. After some googling around I could not find a benchmark comparison of the more prominent functional ...
- 309
6
votes
11
answers
3k
views
If python compiles to assembly and an OS is written in it, will it compete favorably with C in benchmarks?
Ok, I have used the word python in the question, but it well could be language agnostic in that: If a language X has a well optimised compiler targeting assembly and an OS is written in that language, ...
- 1,317
4
votes
2
answers
2k
views
How to test the render speed of my solution in a web browser?
Ok, I need to test the speed of my solution in a web browser, but I have some problems, there are 2 versions of the web solution, the original one that is on server A and the "fixed" version that is ...
- 535
5
votes
3
answers
466
views
Is running "milli"-benchmarks a good idea?
I just came across the Caliper project and it looks very nice.
Reading the introduction to microbenchmarks, one gets the feeling that the developers would not suggest to use the framework if the ...
- 153
4
votes
2
answers
460
views
Benchmarking CPU processing power
Provided that many tools for computers benchmarking are available already, I'd like to write my own, starting with processing power measurement.
I'd like to write it in C under Linux, but other ...
- 1,413
5
votes
1
answer
213
views
Use of a profiler tool to aid in the analysis of a brute force algorithm in Java
I was asked to profile (using some tool, such as YourKit or JVisualVM) a couple of implementations of the Traveling Salesman Problem (find the minimum path that visits all the given set of cities), in ...
- 592
4
votes
3
answers
687
views
Question about a "Benchmark" regarding Java and .NET
To get that out of the way - i am not asking about which platform is faster or try to start a flame war.
Question: Why is the .NET version slower 5x in the first test ?
The results i got were:
.NET ...
- 1,539
14
votes
4
answers
6k
views
Benchmarking databases
I see lots of discussions flying around about performance of db 'x' or that moving from 'x' to 'y' improved our site performance.
I'm yet to see proper benchmarking that works across different types ...
- 11.2k
2
votes
4
answers
2k
views
Measures of Javascript engine performances over time? [closed]
Since the beginning of the Javascript race -- which I would situate around Google Chrome launch in 2008 -- the improvement in the Javascript engine performances have been impressive.
The web is ...
- 131
8
votes
5
answers
1k
views
Does profiling without benchmarking lead to micro-optimization?
Read first: For a definition of the two terms ("profiling" and "benchmarking") and the need for distinguishing them, please read this answer to a previous question.
I have to admit that until I saw ...
- 16.7k
1
vote
1
answer
219
views
benchmarks for quasi real-time software
I'm wondering if there is any benchmarks which can help to understand real-time characteristics (response time) of non-real-time software. For instance C# application on Windows or MSSQL Server.
I ...
- 475
23
votes
3
answers
16k
views
Does anyone do hardware benchmarks on compiling code? [closed]
I've seen a bunch of sites that benchmark new hardware on gaming performance, zipping some files, encoding a movie, or whatever. Are there any that test the impact of new hardware (like SSDs, new CPUs,...
- 536
3
votes
2
answers
871
views
What will be a good python script (or your favorite language goes here) to test a system's performance and capabilities?
Let's say you're in a computer store looking at 10 laptops, you want to really compare the system's capabilities. What will be an efficient "your fav language goes here" script that will allow you to ...
- 702
2
votes
2
answers
423
views
Which benchmarking software would you use to evaluate the performance of your development rig?
I have been tasked with evaluating a new PC for our dev team that is spec'd only slightly better than our current models.
Could anyone recommend appropriate benchmarking software that would be ...
- 129
3
votes
2
answers
182
views
Do system background services affect languages benchmarking?
Today I was looking at some blogs on Ruby and Python. A Python loyalist wrote a few lines of code with a loop to show how Python is faster than Ruby, with a similar code. I also found a blog of a Ruby ...
- 4,378