By Jewe | March 30, 2015
I have done some benchmarking, using different compilers to build the same runtime environment. It showed some interesting results.
There is a small demo-application in JewelScript’s SVN repository. It’s called “JILRunOnly”. I made this to show developers how to embed JewelScript in any application. It’s a simple command-line program written in ANSI-C, so it can be compiled on any platform using any compiler.
Having just added a new project for Microsoft’s VS Express 2013 to the repository, I came up with the idea to compare the outputs of all compilers I have access to. The compilers used are TDM-GCC 4.8.1 (via DevC++ 5.9.2), Microsoft VS Express 2008 and VS Express 2013.
I made fully optimized release builds of “JILRunOnly”, targeting 32-bit in VS 2008 and 32/64-bit in the other two compilers. Then I ran a synthetic benchmarking script and compared their execution times and virtual machine instructions per second.
Here are the results:
|TDM-GCC 32||3.287 s||1.594 s||0.922 s||0.42 s||91.18 M|
|TDM-GCC 64||3.105 s||1.513 s||0.905 s||0.421 s||95.52 M|
|VS2008 32||2.98 s||1.466 s||0.952 s||0.483 s||96.55 M|
|VS2013 32||3.619 s||2.356 s||1.138 s||0.609 s||73.38 M|
|VS2013 64||2.98 s||1.373 s||0.982 s||0.422 s||98.63 M|
alloc: Allocates, initializes and destroys 10 million script objects. Basically it measures memory allocation, deallocation and copying performance.
loop: Simply runs through six nested loops, each of them counting to 18, resulting in a total number of 34,012,224 iterations. This measures raw byte-code execution performance.
fib: Recursively calculates the Fibonacci sequence for N=32, resulting in 7,049,155 recursions. This mainly measures function call and variable stack performance.
ack: Recursively calculates the Ackerman sequence for M=3 N=8, resulting in 2,785,999 recursions. Measures function call and variable stack performance as well.
IPS: The average number of executed virtual machine instructions per second calculated by the script.
The arithmetic / recursive benchmarks perform better with the GCC output, while the memory / iteration based benchmarks perform better with the Microsoft compiler. The 32-bit output of the VS 2013 compiler is a different story though. I have no idea why the 32-bit build would be so much slower than the 64-bit build of the same compiler, or the 32-bit build from VS 2008 for that matter.
It seems building 32-bit applications with VS 2013 is not recommended. If you have both VS 2008 and 2013 installed, I would recommend using VS 2008 to build 32-bit versions and VS 2013 for 64-bit versions.
Comments are closed.