In a previous blog entry, I did some extremely informal benchmarking with Lush, OCaml, Python and C. I’ve now added two new Python tests: one with Psyco, a JIT-like solution that takes almost no effort to add to existing code, and one with Pyrex, where one can code extension modules in a language that looks just like Python but has types. These modules are then translated to C and compiled into Python usable extension libaries.
With Psyco, one only has to add the lines “import psyco; psyco.full()” to one’s Python code. The performance gains can be huge, but in general are quite modest. It takes far less effort than Pyrex though.
The updated benchmarks table is shown below:
| Language | Time (seconds) |
|---|---|
| Octave 2.1.57 | 7.894 |
| Python 2.3.4 | 0.459 |
| OCaml 3.08 interpreted | 0.229 |
| Python 2.3.5 with Psyco | 0.148 |
| Python 2.3.5 with Pyrex | 0.064 |
| OCaml 3.08 compiled | 0.021 |
| gcc 3.3.4 | 0.017 |
The new implementations are available at the usual place.
You should try to code according to each language’s strengths.
Octave (and Matlab) strength is vector manipulation. I can rewrite your Octave code replacing tight for loops with vector operations (which are shorter and easier in my opinion).
Doing that leaves us with the following version of the code:
## $Revision: 1.3 $
## run with octave lushCompare.m
function z = harmonic (n)
z = sum(1./(1:n));
endfunction
## 100 loop iterations is more than enough; this is not so slow
numberOfLoops = 100;
cstart = time();
for i = 1:numberOfLoops
printf(“%f – %d\n”, harmonic (1000000), i);
endfor
cend = time();
printf (“%f seconds.\n”, (cend – cstart) / numberOfLoops)
I get 0.049810 seconds (for comparison, the C version with the -O3 switch gets 0.020000 seconds).
However, if we avoid the function call (these are also expensive and for such a short snippet of code it’s not worthy), we get the following code:
## $Revision: 1.4 $
## 300 loop iterations is more than enough; this is NOT slow
numberOfLoops = 300;
cstart = time();
for i = 1:numberOfLoops
printf(“%f – %d\n”, sum(1./(1:1000000)), i);
endfor
cend = time();
printf (“%f seconds.\n”, (cend – cstart) / numberOfLoops)
And run time is now 0.012111 seconds. Less than the C version (the C version can be vectorized using the Intel compiler and runs in 0.016667 seconds, still slower).
When comparing languages, compare apples to apples. This is definitely a biased test.
Hi there Octave fan. :)
My purpose was not to put octave in a bad light. I wanted to compare apples to apples: for loops to for loops in this case. I know that octave (and matlab) are not strong in this regard.
Thanks for the code in anycase!
Hi,
I know you knew they are bad at loops :-), that’s why this is a biased benchmark.
Moreover, each language has its own idiosyncrasies, and you are supposed to program according to them. For example: it is certainly possible to program procedurally in Lisp or Scheme. Of course, it is not what they are intended for, and possibly the performance will be awful.
That’s what I meant with apples to apples.