5 min read

Running Bitcoin Core v0.7 and Earlier

An exploration into eccentricities of trying to sync extremely old versions of Bitcoin Core.
Running Bitcoin Core v0.7 and Earlier

The following is a result of my research into the historical syncing performance of all Bitcoin Core releases and the subsequent challenges of building really old releases for which it's difficult to find precompiled binaries.

Note that if you're planning on syncing these versions of Bitcoin Core, you're going to have a bad time. I'm not the first person to look into this; during my research I found that Sjors Provoost ran similar experiments in 2017 and wrote about it.

Historical Bitcoin Core Client Performance – Sjors Provoost

Investigating Syncing Failures

None of the versions prior to v0.8.0 will be able to complete a sync to chain tip for various reasons.

v0.3: stopped syncing after block 124,275 with this error:

ERROR: ConnectInputs() : fb0a1d8d34 VerifySignature failed
InvalidChainFound: invalid block=0000000000004939267f 
height=124276  work=6613870563198902508

At first glance this doesn't look like a particularly weird transaction. However, if we start looking into the history of whomever was spending funds locked by that Bitcoin address, we can see that they crafted several odd transactions with hundreds of "0 value" outputs. It wouldn't be a stretch to assume this was a fairly technical person who was trying to break the network.

Eventually, something that jumped out at me was the size of this input's signature. 75 bytes is on the extreme upper bound whereas most P2PKH spends are 71 - 73 bytes.

You can read an excellent deep dive into bitcoin transaction signature history here; it notes that if you try to validate the early chain with a newer version of OpenSSL (1.0.0p / 1.0.1k) then you'll get failures because the DER validation is stricter and will reject certain types of encodings.

Evolution of the signature size in Bitcoin
Digital signatures are an essential building block of the Bitcoin protocol andaccount for a large part of the data stored on the blockchain. We detail how thesize of the encoded ECDSA signatures reduced multiple times over the last yearsand how the proposed Schnorr signature compares to the lengt…

The workaround is to either build Bitcoin Core with an older version of OpenSSL or to manually apply this code patch before building. I was confused at first because gitian builder uses Ubuntu 10.04 virtual machines as build environments, which I figured should be using the older (compatible) version of OpenSSL... however, Andrew Chow pointed out that they backported this OpenSSL patch in 2015.

v0.4 and v0.5 both stopped syncing after block 258,354 with error

EXCEPTION: 11DbException       
Db::put: Cannot allocate memory       
bitcoin in ProcessMessage()       

ProcessMessage(block, 901212 bytes) FAILED
received block 0000000000000023e872

This is notable because it looks like block 258355 is the first block that is 900KB in size; before this height it looks like almost all mined blocks were hitting the default "soft limit" of 250KB at the time.

v0.6 stopped syncing after block 364,670 with error

EXCEPTION: 11DbException       
Db::put: Cannot allocate memory       
bitcoin in ProcessMessages()       

ProcessMessage(block, 999787 bytes) FAILED
received block 000000000000000001d3

This is similarly notable because it looks like block 364,671 is the first block that hit 1MB in size.

v0.7 failed at the same block but with a different error logged.

received block 00000000000000000221
ERROR: ConnectBlock() : UpdateTxIndex failed
InvalidChainFound: invalid block=00000000000000000221  height=364671
ERROR: SetBestChain() : SetBestChainInner failed
ERROR: AcceptBlock() : AddToBlockIndex failed
ERROR: ProcessBlock() : AcceptBlock FAILED

I suspect that the v0.4 - v0.7 stalls are all due to the same issue, but what is it? Is this a result of the "BDB locks issue" that caused the unintentional chain split in 2013? I tried creating a file ~/.bitcoin/DB_CONFIG as per these instructions:

set_lg_dir database
set_lk_max_locks 537000

But each node version still stalls at the same block height as before. It turns out that you ALSO need to configure set_lk_max_objects AND the original recommended value of 537000 isn't high enough if you want to get all the way to block 700,000. The following ~/.bitcoin/DB_CONFIG values worked for me:

set_lg_dir database
set_lk_max_locks 1000000
set_lk_max_objects 1000000

Performance Results

You can find my raw syncing data here.

Though you can hardly even see the data for v0.3 since it's nearly the same as v0.4 and v0.5 and ends abruptly at height 124,000. Here's a log scale version to zoom in a bit more.

It is noteworthy that there's an inflection point near block 190,000 at which point v0.4 starts performing better than v0.5. My best guess is that this is a result of checkpoints. Bitcoin 0.3.2 introduced a mechanism called checkpoints to prevent denial-of-service attacks during initial block download by ensuring that new full nodes couldn’t be tricked into spending excessive amounts of effort validating alternative blockchains that were different than the best-known chain at certain points in time.

Bitcoin 0.5.0 built upon those checkpoints to speed up syncing by skipping verification of signatures in blocks that were earlier in the block chain than the most recent checkpoint. "But wait," you might be thinking, "I thought you forced the nodes to validate all historical signatures by setting assumevalid=0 in the config?" I sure do, but that setting doesn't have any effect for Bitcoin Core versions before v0.14 when it was first introduced.

As such, I'd say that v0.5 is generally slower than v0.4 and the early syncing performance is actually cheating / an unfair comparison.

Legacy vs Modern Performance

How much more performant is the latest v22 release compared to these old releases?

Syncing to block 124,000:

  • v22 is 17X faster than v0.3

These blocks are all empty, which is why the performance difference is far less than for nodes that sync more of the blockchain.

Syncing to block 258,000:

  • v22 is 77X faster than v0.4
  • v22 is 83X faster than v0.5

Syncing to block 364,000:

  • v22 is 40X faster than v0.6
  • v22 is 38X faster than v0.7

Forward Compatibility is Hard

You may often hear claims that one can run the very first version of the Bitcoin software and it would be compatible with the current network. Naturally, the true answer is much more complicated and nuanced. In order to get really old versions of Bitcoin software to sync to chain tip, some changes are needed. It's additionally tricky because we can see that not all consensus critical code is necessarily code written by Bitcoin developers - sometimes it's in third party libraries that have changed over time, thus making the software build process itself a potential point of consensus failure!