I am making an attempt to create a Bitcoin deal with database utilizing btcrecover and Google BigQuery information. Whereas the Ethereum database works advantageous, my Bitcoin database fails to return any addresses, though take a look at addresses and random addresses from the blockchain exist within the dataset.
Right here’s what I attempted:
1.Created the database utilizing:
python create-address-db.py --inputlistfile C:UserstestDesktopbtc-addresses-db-20250816 --dbfilename btc-addresses-db-20250816.db --dblength 31
2.Checked addresses utilizing:
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresslist ./addressdb-checklists/BTC.txt
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresses bc1qxy2kgdygjrsqtzq2n0yrf2493p83kkfjhx0wlh
Observations:
- Ethereum DB works with –dblength 29.
- After I create the Bitcoin DB with –dblength 31 utilizing the complete dataset (~2000 BigQuery recordsdata, 16 GB DB), it finds no addresses.
- A smaller subset (4 BigQuery recordsdata out of 2000) works appropriately, with each –dblength 27 (1GB DB) and –dblength 31 (16GB DB).
- Splitting the dataset into halves or subsets results in inconsistent outcomes: some deal with ranges are discovered, others usually are not, even throughout the similar database.
- Tried a number of Python variations (3.9–3.13) and btcrecover variations (1.6.0, 1.12.0, grasp) with the identical outcomes.
- 64-bit system, all dependencies put in, Rust put in, digital environments used.
- Official Bitcoin database from the writer (addresses-BTC-2011-to-2021-03-31.zip) works appropriately.
Instance of inconsistent outcomes from subsets:
| File Vary | Addresses Discovered? |
|---|---|
| 500–520 | Sure |
| 500–599 | No |
| 500–570 | No |
| 550–570 | No |
| 550–559 | Sure |
| 560–569 | Sure |
| 570–579 | Sure |
| 563–579 | Sure |
As you possibly can see, some databases efficiently returned addresses, whereas others didn’t. Initially, I believed the issue may be attributable to file 570. Nonetheless, in a subsequent take a look at, the database that included file 570 labored appropriately.
In one other take a look at, I created a database from recordsdata 550–570 and examined addresses from every file as samples for this database. The outcomes had been as follows:
- Pattern deal with from file 570: discovered
- Pattern deal with from file 553: not discovered
- Pattern deal with from file 563: discovered
So even throughout the similar database, some addresses had been efficiently retrieved whereas others weren’t. This sample means that the problem shouldn’t be merely with a single file, however may be associated to how sure addresses are listed or saved within the database.
I’ve tried this on each a Home windows server and my very own PC (with 32GB of
RAM). I even rebuilt the complete database a number of instances and verified the
hashes afterward to examine for any storage or corruption points, each
database recordsdata had similar hashes, so it doesn’t appear to be a saving
or file corruption downside. It’s price mentioning that I constructed the
Ethereum database utilizing the identical course of, and that one works completely
advantageous, the problem solely occurs with the Bitcoin database.
Query:
Has anybody skilled comparable points creating giant Bitcoin databases with btcrecover? May this be a dataset problem, a bug in btcrecover, or an issue with how addresses are listed when –dblength is giant?
I am making an attempt to create a Bitcoin deal with database utilizing btcrecover and Google BigQuery information. Whereas the Ethereum database works advantageous, my Bitcoin database fails to return any addresses, though take a look at addresses and random addresses from the blockchain exist within the dataset.
Right here’s what I attempted:
1.Created the database utilizing:
python create-address-db.py --inputlistfile C:UserstestDesktopbtc-addresses-db-20250816 --dbfilename btc-addresses-db-20250816.db --dblength 31
2.Checked addresses utilizing:
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresslist ./addressdb-checklists/BTC.txt
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresses bc1qxy2kgdygjrsqtzq2n0yrf2493p83kkfjhx0wlh
Observations:
- Ethereum DB works with –dblength 29.
- After I create the Bitcoin DB with –dblength 31 utilizing the complete dataset (~2000 BigQuery recordsdata, 16 GB DB), it finds no addresses.
- A smaller subset (4 BigQuery recordsdata out of 2000) works appropriately, with each –dblength 27 (1GB DB) and –dblength 31 (16GB DB).
- Splitting the dataset into halves or subsets results in inconsistent outcomes: some deal with ranges are discovered, others usually are not, even throughout the similar database.
- Tried a number of Python variations (3.9–3.13) and btcrecover variations (1.6.0, 1.12.0, grasp) with the identical outcomes.
- 64-bit system, all dependencies put in, Rust put in, digital environments used.
- Official Bitcoin database from the writer (addresses-BTC-2011-to-2021-03-31.zip) works appropriately.
Instance of inconsistent outcomes from subsets:
| File Vary | Addresses Discovered? |
|---|---|
| 500–520 | Sure |
| 500–599 | No |
| 500–570 | No |
| 550–570 | No |
| 550–559 | Sure |
| 560–569 | Sure |
| 570–579 | Sure |
| 563–579 | Sure |
As you possibly can see, some databases efficiently returned addresses, whereas others didn’t. Initially, I believed the issue may be attributable to file 570. Nonetheless, in a subsequent take a look at, the database that included file 570 labored appropriately.
In one other take a look at, I created a database from recordsdata 550–570 and examined addresses from every file as samples for this database. The outcomes had been as follows:
- Pattern deal with from file 570: discovered
- Pattern deal with from file 553: not discovered
- Pattern deal with from file 563: discovered
So even throughout the similar database, some addresses had been efficiently retrieved whereas others weren’t. This sample means that the problem shouldn’t be merely with a single file, however may be associated to how sure addresses are listed or saved within the database.
I’ve tried this on each a Home windows server and my very own PC (with 32GB of
RAM). I even rebuilt the complete database a number of instances and verified the
hashes afterward to examine for any storage or corruption points, each
database recordsdata had similar hashes, so it doesn’t appear to be a saving
or file corruption downside. It’s price mentioning that I constructed the
Ethereum database utilizing the identical course of, and that one works completely
advantageous, the problem solely occurs with the Bitcoin database.
Query:
Has anybody skilled comparable points creating giant Bitcoin databases with btcrecover? May this be a dataset problem, a bug in btcrecover, or an issue with how addresses are listed when –dblength is giant?
