Comparing implementations of the Monkey Language X: An update on performance for Go, Kotlin, Crystal, Python and Ruby

Previously

In the last episode, we wrote a TypeScript implementation for the Monkey Language.

Let's review some numbers, shall we?

Since I started this series, most languages and runtimes have released new versions. On the other hand, I did some modifications to a couple of implementations.

Go

With Go, I saw a significant increase in performance from 1.16 to 1.17, which I covered in a previous issue. The update to 1.19 doesn't seem to do anything performance-wise. No significant updates on my codebase, only the use of generics on a couple of tests but nothing major in the interpreter or VM.

The VM implementation with Go is still the fastest one. But the interpreted version is now the slowest of the compiled implementations and even lies (not by far) behind interpreted TypeScript running on the Bun runtime.

The general feedback from the community is that my Go implementation isn't as optimised as my TypeScript one. And that can be true, but I lack the knowledge to go further. If someone wants to have a look, I'll be more than happy to accept a PR to my repo.

Kotlin

I made several modifications to the Kotlin AST and interpreter. I simplified the AST hierarchy, which allowed me to reduce the recursion in the evaluator loop. So, now my interpreter is faster than my VM in JDK 11.0.17 and 19.0.1

But not on the GraalVM 22.3-19.0.1

Crystal

Takes first place in the performance improvement championship 2022. My interpreter compiled with Crystal 1.6.2 is 2.44 times faster than the same interpreter compiled with Crystal 1.2.2

Leaving Go far behind.

The VM version had a more modest improvement of "just" 8%.

Python

The original idea of this article was to discuss the performance improvements that Python 3.11 would have brougth to my interpreter.

It was theorised that some applications would not see an increase by switching to 3.11 and, others would have worst performance.

In Linux, I can see an increase with Python 3.11 on some days, while on other days go to Python 3.10. Is inconclusive.

Running the same benchmark again on MacOS gave an advantage to Python 3.10.

Let's see and wait for the improvements Python 3.12 will bring.

Ruby

Ruby MRI itself didn't have any new major releases, but there was a major release in the Ruby world. JRuby 9.4.0.0 was released with support for Ruby 3.1.0.

For those too young to remember, in the good ol' days, JRuby was faster than Ruby. Not anymore. The focus on performance that the Ruby community took in the last few years paid dividends, and Ruby is now faster than JRuby (not even counting YJIT).

Furthermore, there is an alternative Ruby runtime, TruffleRuby, that runs on the GraalVM. TruffleRuby isn't a drop-in replacement for JRuby, but there are migrations guides if needed.

TruffleRuby tops any other implementation, followed by YJIT (included in the MRI distribution), MRI and lastly JRuby.

Conclusion

Go is still the best on VM implementation, but it drags behind other Interpreter implementations. As a Go rookie, there isn't a lot that I can do.

Kotlin keeps improving, mainly due to the changes in JVM versions.

Crystal is amazing. I'm surprised how this small team can produce a language that rivals giants in the industry. I know it lacks the popularity to make it mainstream, but the infrastructure is there. I hope that more people will give Crystal a chance.

The update to Python 3.11 didn't bring any conclusive performance benefits. Alternative runtimes are yet to support Python 3.1x features to be able to make a proper comparison.

I like Ruby as a language. Ruby YJIT is an engineering triumph, and it'll get better in Ruby 3.2. TruffleRuby is impressive, and I see many companies and a big community working on improving Ruby. So perhaps the news that Ruby is dead are greatly exaggereted.