Ruby 2.7 to 4.0 Benchmarks: How Performance Evolved Over Time

This time, we tested the latest matching Docker image for every Ruby release line from 2.7 through 4.0. We combined a ten-minute synthetic application with smaller standard-library benchmarks to see both the full application and the individual operations inside it.

Versions tested

We used these exact Ruby Docker images:

Ruby 3.0.6 and 3.1.6 were the newest matching Docker Official Image tags available for those release lines.

What the synthetic application does

Each operation parses deterministic JSON, derives a cache key, reads or updates a shared bounded Hash, calculates SHA-256 on cache misses, Base64-encodes the result, generates a small JSON response, and creates additional allocation pressure.

Ten Ruby worker threads run the same code in every version.

Workload profiles

The light profile uses 1KiB payloads and 256 bytes of additional allocation per operation. The heavy profile uses 8KiB payloads and 4KiB of additional allocation. These profiles change the amount of work, they are not memory limits.

Synthetic application results

Throughput

Average application throughput in millions of operations per second.

Mean latency

Average time spent on each application operation.

Total GC pause time

Average time attributed to garbage collection during the measurement window.

CPU use

Average process CPU utilization as a percentage of all ten host cores.

Resident memory

Average resident set size in MiB.

Live heap slots

Average number of live Ruby heap slots reported during measurement.

Application behavior over time

These light-profile charts show one-minute points from the recorded process samples.

Resident memory over time

Resident set size during the ten-minute application run.

Smoothing window Smooths the line using a centered moving average across nearby minutes. 1m shows raw values.

3m

Live heap slots over time

Live Ruby heap slots during the same application run.

Microbenchmark results

Light and Heavy do not always mean different work in these charts. For benchmarks that process the full payload, Light uses 1KiB and Heavy uses 8KiB. The extra allocation setting is only used by the synthetic application. Some microbenchmarks use the same input in both profiles, which is why their two bars can be nearly identical.

JSON parsing

Parses the full deterministic JSON payload: 1KiB for Light and 8KiB for Heavy.

JSON generation

Generates JSON from the same small object in both profiles.

SHA-256

Calculates a SHA-256 digest from a 1KiB Light payload or an 8KiB Heavy payload.

Note: Ruby 2.7 is a clear outlier in this chart. Its Digest::SHA256 could use the OpenSSL implementation, while newer Ruby versions on Linux use Ruby's bundled SHA-256 implementation. This is primarily an implementation and backend difference, not evidence of a Ruby VM performance regression.

Base64

Encodes and decodes the full 1KiB Light payload or 8KiB Heavy payload.

Regular expressions

Extracts the same user field near the beginning of each payload, so the work is effectively the same in both profiles.

Integer sorting

Sorts the first 256 integers from the payload. Both profiles use the same first 1KiB of data.

Hash churn

Inserts, reads, and deletes entries in a Ruby Hash. This test does not use the profile payload, so Light and Heavy run the same work.

Marshal

Serializes and deserializes the same small object in both profiles using Ruby Marshal.

zlib compression

Compresses and decompresses a 1KiB Light payload or an 8KiB Heavy payload with zlib.

How the tests were performed

Source code

Conclusion

The full application and the smaller benchmarks do not always move together. That is the main reason we run both.

These results are a useful reference, but your own request paths, payload sizes, gems, and allocation patterns will matter more for your Ruby service.

What should the next Ruby benchmark add to the mix? Send us your ideas.
Happy Benchmark Thursday!