2023 Bitcoin Node Performance Tests

As I’ve noted many times in the past, backing your bitcoin wallet with a fully validating node gives you the strongest security model and privacy model that is available to Bitcoin users. Five years ago I started running an annual comprehensive comparison of various implementations to see how well they performed full blockchain validation. Now it's time to see what has changed over the past year!

The computer I use as a baseline was high-end off-the-shelf hardware back in 2018. It cost about $2,000 at the time.

Note that no Bitcoin implementation strictly fully validates the entire chain history by default. As a performance improvement, most of them don’t validate signatures before a certain point in time. This is considered safe because those blocks and transactions are buried under so much proof of work. In order for someone to create a blockchain that had invalid transactions before that point in time would cost so much mining resources, it would fundamentally break certain security assumptions upon which the network operates.

For the purposes of these tests I need to control as many variables as possible; some implementations may skip signature checking for a longer period of time in the blockchain than others. As such, the tests I'm running do not use the default settings - I change one setting to force the checking of all transaction signatures and I often tweak other settings in order to make use of the higher number of CPU cores and amount of RAM on my machine.

Also, in order to ensure that the bandwidth of peers on the public network is not a bottleneck and potential source of inconsistency, I now sync between two nodes on my local network.

The amount of data in the Bitcoin blockchain is relentlessly increasing with every block that is added, thus it's a never-ending struggle for node implementations to continue optimizing their code in order to prevent the initial sync time for a new node from becoming obscenely long. After all, if it becomes unreasonably expensive or time consuming to start running a new node, most people who are interested in doing so will chose not to, which weakens the overall robustness of the network.

Last year's test was for syncing to block 760,000 while this year's is syncing to block 819,000. Due to the rising popularity of inscriptions, there was a larger than usual data increase of 22% from 434GB to 530GB. As such, we should expect implementations that have made no performance changes to take about 22% longer to sync than 1 year ago.

What's the absolute best case syncing time we could expect if you had limitless bandwidth and disk I/O? Since you have to perform 2.8 billion ECDSA verification operations in order to reach block 819,000 and it takes my machine about 4,430 nanoseconds per operation via libsecp256k1... it would take my machine 3.45 hours to verify the entire blockchain if bandwidth and disk I/O were not bottlenecks. Note that last year it took my machine 4,500 nanoseconds per ECDSA verify operation; the secp256k1 library keeps being further optimized.

On to the results!

Bcoin v2.2.0

There was no new release of Bcoin in the past year. Syncing Bcoin 2.2.0 to height 819,000 used:

• 850 MB disk reads
• 4.7 TB disk writes
• 1.8 GB RAM

My bcoin.conf:

only:<local node IP address>
network:main
checkpoints:false
cache-size:10000
sigcache-size:1000000
max-files:5000

Bitcoin Core 26.0

The full sync used:

• 16.7 GB RAM
• 60 MB disk reads
• 561 GB disk writes

My bitcoind.conf:

connect=<local node IP address>
assumevalid=0
dbcache=24000
disablewallet=1

Bitcoin Knots 25.1

Bitcoin Knots is effectively Bitcoin Core plus a set of patches, so it's reasonable to expect the performance to be similar. Worth noting that Knots 26.0 was not yet released, so the comparison is not as close as it could be.

The full sync used:

• 17.8 GB RAM
• 234 MB disk reads
• 561 GB disk writes
• 510 GB download bandwidth

My bitcoind.conf:

assumevalid=0
dbcache=24000
maxmempool=1000

btcd v0.23.3

BTCD hasn't had a release since last year's test, despite being actively developed.

The full sync used:

• 4.1 GB RAM
• 803 GB disk reads
• 13.1 TB disk writes
• 518 GB download bandwidth

I noted that it still doesn't really use more than half of the available CPU cycles, so I think there's still some low hanging fruit to grab. I'm told that next year we'll see a release with massive performance improvements that include parallelization of block processing.

My btcd.conf:

nocheckpoints=1
sigcachemaxsize=1000000
connect=<local node IP address>

Gocoin 1.10.3-pre

Gocoin has not tagged a release in over 30 months despite being actively developed. I compiled commit hash 403d87f...

I made several changes to get maximum performance according to the documentation:

• Installed secp256k1 library and built gocoin with sipasec.go
• set several new configs noted below

My gocoin.conf:

LastTrustedBlock: 00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048
AllBalances.AutoLoad:false
UTXOSave.SecondsToTake:0
Stats.NoCounters:true
Memory.CacheOnDisk:false
ConnectOnly:<local node IP address>

Gocoin still has the best dashboard of any node implementation.

Gocoin used:

• 30.2 GB RAM
• 530 GB download bandwidth
• 8.4 GB disk reads
• 424 GB disk writes

Gocoin was only 4 minutes slower than Bitcoin Core - if my benchmark machine had more RAM, it wouldn't have done as much disk I/O and I bet it would dethrone Core for the #1 spot!

Libbitcoin Server 3.6.0

Note that this software has remained the same for 4 years; there was a 3.8.0 release a few months ago, but from looking at the code changes they were only changing the build system and node the node logic itself. Unfortunately I struggled to get 3.8.0 to build due of dependencies on specific versions of gcc and libboost. They have been working on an overhauled 4.0 release for a while; hopefully it will be ready for next year's tests!

My libbitcoin config:

[network]
peer = <local network node IP address>:8333
outbound_connections = 1
[blockchain]
# I set this to the number of virtual cores since default is physical cores
cores = 12
checkpoint = 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f:0
[database]
cache_capacity = 100000

During the source of syncing, Libbitcoin Server used:

• 463 TB disk reads
• 7.9 TB disk writes
• 3.7 GB RAM
• 960 GB download bandwidth

I'm not sure what's going on with Libbitcoin since it took over twice as long to sync as last year. Last year it did:

• 77 TB disk reads
• 5.5 TB disk writes

It looks like the reason for the extremely high disk reads is that it very quickly starts to use my hard drive's swap partition for its UTXO cache even though my machine had tons of RAM available. But this year's result was so much slower that I ran the week-long sync test twice to be sure. It's possible something got screwed up when I failed to compile the recent release, though the binary I ran was the same one I compiled in 2020. It's a shame that they don't offer precompiled binaries so that I could remove that uncertainty. In short, it looks like Libbitcoin really needs to get their 4.0 release out because the current architecture is not scaling well.

Mako

Mako is a newer implementation by Chris Jeffrey that's written in C. In order to create a production optimized build I built the project via:

cmake . -DCMAKE_BUILD_TYPE=Release

And ran it with node -checkpoints=0 -connect=<LAN node> -maxconnections=1

Mako hasn't changed since last year's tests. CPU usage was only around 50% - the bottleneck here is clearly disk I/O, as I observed mako constantly writing 100 MB/S. This is because mako has not yet implemented a UTXO cache and thus has to write UTXO set changes to disk every time it processes a transaction.

During the course of syncing, Mako used:

• 402 MB RAM
• 145 MB disk reads
• 4.9 TB disk writes
• 517 GB download bandwidth

Performance Rankings

1. Bitcoin Core 26.0: 8 hours, 38 minutes
2. Gocoin 1.10.3: 8 hours, 42 minutes
3. Bitcoin Knots 25.1: 8 hours, 49 minutes
4. Mako 3d8a5180: 1 day, 3 hours, 46  minutes
5. Bcoin 2.2.0: 1 day, 14 hours, 32 minutes
6. BTCD 0.23.3:  3 days, 7 hours, 56 minutes
7. Libbitcoin Server 3.6.0: 6 days, 16 hours, 36 minutes

Delta vs Last Year's Tests

Remember that the total size of the blockchain has grown by 22% since my last round of tests, thus we would expect that an implementation with no new performance improvements or bottlenecks should take ~22% longer to sync.

1. Bitcoin Knots 25.1: +1 hour, 26 minutes (19.4% longer)
2. Gocoin 1.10.3: +1 hour, 27 minutes (20% longer)
3. Bitcoin Core 26.0: +1 hour, 28 minutes (20.5% longer)
4. Mako 3d8a5180:  +4 hours, 2 minutes (17% longer)
5. Bcoin 2.2.0: +6 hours, 24 minutes (13.7% longer)
6. BTCD 0.23.3: +11 hours, 29 minutes (18.5% longer)
7. Libbitcoin Server 3.6.0: +3 days, 16 hours, 9 minutes (167% longer)

As we can see, most of the implementations took less than the expected 22% longer to sync. It's unclear if Libbitcoin's slowdown is due to changes in blockchain data this year or my own fault as a result of my failed attempt to compile the latest release. Shout-out to Gocoin for continuing to inch toward flipping Bitcoin Core!

Exact Comparisons Are Difficult

While I ran each implementation on the same hardware and synced against a local network peer to keep those variables as consistent as possible, there are other factors that come into play.

1. Not all implementations have caching; even when configurable cache options are available it’s not always the same type of caching.
2. Not all nodes perform the same indexing functions. For example, Libbitcoin Server always indexes all transactions by hash — it’s inherent to the database structure. Thus this full node sync is more properly comparable to Bitcoin Core with the transaction indexing option enabled.
3. Your mileage may vary due to any number of other variables such as operating system and file system performance.

Conclusion

Given that the strongest security model a user can obtain in a public permissionless crypto asset network is to fully validate the entire history themselves, I think it’s important that we keep track of the resources required to do so.

We know that due to the nature of blockchains, the amount of data that needs to be validated for a new node that is syncing from scratch will relentlessly continue to increase over time. Thus far the tests I run are on the same hardware each year, but on the bright side we do know that hardware performance per dollar will also continue to increase each year.

It's important that we ensure the resource requirements for syncing a node do not outpace the hardware performance that is available at a reasonable cost. If they do, then larger and larger swaths of the populace will be priced out of self sovereignty in these systems.