add labs notes debugger

add labs to build

build: Make $(package)_env available to all $(package)_cmds

build: Move environment variables into $(package)_config_env

bugfix: Strict type checking for RPC boolean parameters

refactor: Replace isTrue with get_bool

This makes the code more robust, see previous commit.

In general replacing isTrue with get_bool is not equivalent because get_bool can throw exceptions, but in this case, exceptions won't happen because of RPCTypeCheck() and isNull() checks in the preceding code.

Merge bitcoin/bitcoin#24611: Add fish completions

ccba4fe7e3113f5d3bb2fbe54147b3a17e1bfa24 doc: Add completion subdir to contrib/README.md (willcl-ark) 7075848f9646a5b37044beb78c48e57e1f32b6d0 script: Add fish completions (willcl-ark) a27a445b71dbee41f81c7786e667eaa3d9813028 refactor: Sub-folder bash completions (willcl-ark)

Pull request description:

The completions are dynamically generated from the respective binary help pages.

Completions should be sourced into the shell or added to $XDG_CONFIG/fish/completions. See where to put completions for more information.

As the completions are auto-generated they should only require as much maintenance as the bash equivalents, which is to say very little!

ACKs for top commit: achow101: ACK ccba4fe7e3113f5d3bb2fbe54147b3a17e1bfa24 josibake: ACK https://github.com/bitcoin/bitcoin/commit/ccba4fe7e3113f5d3bb2fbe54147b3a17e1bfa24

Tree-SHA512: fe6ed899ea1fe90f82970bde7739db11dd0c845ccd70b65f28ad5212f75b57d9105a3a7f70ccdff552d5b21fa3fe9c697d128fb10740bae31fe1854e716b4b8b

Merge bitcoin/bitcoin#25934: wallet, rpc: add label to listsinceblock

4e362c2b7269ae0426010850c605e5c1d0d58234 doc: add release note for 25934 (brunoerg) fe488b4c4b7aa07fb83d528e2942ef914fd188c0 test: add coverage for label in listsinceblock (brunoerg) 722e9a418d078ed34aedd1ca55c1ae104f29a7d3 wallet, rpc: add label to listsinceblock (brunoerg) 852891ff98cffd37a74b9cb96394f43b2e6ca30e refactor, wallet: use optional for label in ListTransactions (brunoerg)

Pull request description:

This PR adds label parameter to listsinceblock to be able to fetch all incoming transactions having the specified label since a specific block.

It's possible to use it in listtransactions, however, it's only possible to set the number of transactions to return, not a specific block to fetch from. getreceivedbylabel only returns the total amount received, not the txs info. listreceivedbylabel doesn't list all the informations about the transactions and it's not possible to fetch since a block.

ACKs for top commit: achow101: ACK 4e362c2b7269ae0426010850c605e5c1d0d58234 w0xlt: ACK https://github.com/bitcoin/bitcoin/pull/25934/commits/4e362c2b7269ae0426010850c605e5c1d0d58234 aureleoules: ACK 4e362c2b7269ae0426010850c605e5c1d0d58234

Tree-SHA512: fbde5db8cebf7a27804154fa61997b5155ad512e978cebb78c17acab9efcb624ea5f39d649899d12e5e675f80d4d0064cae8132b864de0d93a8d1e6fbcb9a737

Merge bitcoin/bitcoin#26308: rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second

d7f61e7d5909da7227c9e34be06ce9eb872ba074 rpc: reduce LOCK(cs_main) scope in gettxoutproof (Andrew Toth) 4d92b5aabaf371225cdc37a7b114adc040bf4da5 rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats (Andrew Toth) efd82aec8a2dd0fca8f2597c3f84cefe057d1243 rpc: reduce LOCK(cs_main) scope in blockToJSON (Andrew Toth) f00808e932c1f67bc76702a3f05778074b10025c rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock (Andrew Toth) 7d253c943f44612431be894b198ffb49ff988fff zmq: remove LOCK(cs_main) from NotifyBlock (Andrew Toth) c75e3d2772b00acc3850f72a8cb733a0345aa773 rest: reduce LOCK(cs_main) scope in rest_block (Andrew Toth)

Pull request description:

Picking up from #21006.

After commit https://github.com/bitcoin/bitcoin/commit/ccd8ef65f93ed82a87cee634660bed3ac17d9eb5 it is no longer required to hold cs_main when calling ReadBlockFromDisk. This can be verified in master at https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L755. Same can be seen for UndoReadFromDisk https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L485.

The first commit moves ReadBlockFromDisk outside the lock scope in rest_block, where we can see a huge performance improvement when fetching blocks with multiple threads.

My test setup, on an Intel i7 with 8 cores (16 threads):

1. Start a fully synced bitcoind, with this bitcoin.conf:

    rest=1
    rpcthreads=16
    rpcworkqueue=64
    rpcuser=user
    rpcpassword=password

2. Run ApacheBench: 10000 requests, 16 parallel threads, fetching block nr. 750000 in binary:

    ab -n 10000 -c 16 "http://127.0.0.1:8332/rest/block/0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e.bin"

Time per request (mean) 183 ms on master 30 ms this branch

So this can process 6.1 times as many requests, and saturates all the cores instead of keeping them partly idle waiting in the lock. With 8 threads the mean times were 90 ms on master and 19 ms on this branch, a speedup of 4.7x.

Big thanks to martinus for finding this and the original PR.

The second commit is from a suggestion on the original PR by jonatack to remove the unnecessary LOCK(cs_main) in the zmq notifier's NotifyBlock.

I also found that this approach could be applied to rpcs getblock (including verbosity=3), getblockstats, and gettxoutproof with similar very good results. The above benchmarks steps need to be modified slightly for RPC. Run the following ApacheBench command with different request data in a file named data.json:

ab -p data.json -n 10000 -c 16 -A user:password "http://127.0.0.1:8332/"

For getblock, use the following in data.json:

{"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e"]}

master - 184 ms mean request time branch - 28 ms mean request time

For getblock with verbosity level 3, use the following in data.json:

{"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", 3]}

This verbosity level fetches an undo file from disk, so it benefits from this approach as well. However, a lot of time is spent serializing to JSON so the performance gain is not as severe. master - 818 ms mean request time branch - 505 ms mean request time

For getblockstats, use the following in data.json:

{"jsonrpc": "1.0", "id": "curltest", "method": "getblockstats", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", ["minfeerate","avgfeerate"]]}

This request used a lock on reading both a block and undo file, so the results are very good. master - 244 ms mean request time branch - 28 ms mean request time

ACKs for top commit: MarcoFalke: re-ACK d7f61e7d5909da7227c9e34be06ce9eb872ba074 💫 hebasto: ACK d7f61e7d5909da7227c9e34be06ce9eb872ba074, I have reviewed the code and it looks OK. Did not make benchmarking though.

Tree-SHA512: 305ac945b4571c5f47646d4f0e78180d7a3d40b2f70ee43e4b3e00c96a465f6d0b9c750b8e85c89ed833e557e2cdb5896743f07ef90e4e53d4ad85452b545886

test: Fix backwards compatibility intermittent failure

rpc: Enable wallet import on pruned nodes

Co-authored-by: João Barbosa joao.paulo.barbosa@gmail.com Co-authored-by: Ryan Ofsky ryan@ofsky.org

test: Avoid intermittent timeout in feature_assumevalid.py

test: Use last release in compatibility tests

Merge bitcoin/bitcoin#26513: Make static nLastFlush and nLastWrite Chainstate members

07dfbb5bb8115c680621f361c65d9cde2f8c52f2 Make static nLastFlush and nLastWrite Chainstate members (Aurèle Oulès)

Pull request description:

Fixes #22189.

The static std::multimap<uint256, FlatFilePos> mapBlocksUnknownParent; referenced in the issue was already fixed by #25571. I don't believe Chainstate references any other static variables.

ACKs for top commit: jamesob: ACK 07dfbb5bb8115c680621f361c65d9cde2f8c52f2 (jamesob/ackr/26513.1.aureleoules.make_static_nlastflush_a) theStack: Concept and code-review ACK 07dfbb5bb8115c680621f361c65d9cde2f8c52f2

Tree-SHA512: 0f26463c079bbc5e0e62707d4ca4c8c9bbb99edfa3391d48d4915d24e2a1190873ecd4f9f11da25b44527671cdc82c41fd8234d56a4592a246989448d34406b0

rpc: Prevent unloading a wallet when rescanning

doc: Add release notes for #26618

Merge bitcoin/bitcoin#26378: refactor: Pass reference to last header, not pointer

fa579f306363ed03c1138121415b67b9b36b4d53 refactor: Pass reference to last header, not pointer (MacroFake)

Pull request description:

It is never a nullptr, otherwise an assertion would fire in UpdatePeerStateForReceivedHeaders.

Passing a reference makes the code easier to read and less brittle.

ACKs for top commit: john-moffett: ACK fa579f3 aureleoules: ACK fa579f306363ed03c1138121415b67b9b36b4d53

Tree-SHA512: 9725195663a31df57ae46bb7b11211cc4963a8f3d100f60332bfd4a3f3327a73ac978b3172e3007793cfc508dfc7c3a81aab57a275a6963a5ab662ce85743fd0

Merge bitcoin/bitcoin#24279: build: Make $(package)_*_env available to all $(package)_*_cmds

affbf58a1e52a8e60c830be6a9e0347e0ff4c28e build: Move environment variables into $(package)_config_env (Hennadii Stepanov) d44fcd3c976572883bbf7f386bc88e2610dc1a58 build: Make $(package)_env available to all $(package)_cmds (Hennadii Stepanov)

Pull request description:

On master (1e7564eca8a688f39c75540877ec3bdfdde766b1) the depends build system, which is based on pure GNU Make, works, but it lacks robustness, and in some corner cases it fails. For example, see bitcoin/bitcoin#22552.

Another bug in the depends build system has already become a problem at least two times in the past (https://github.com/bitcoin/bitcoin/pull/16883#issuecomment-683817472 and https://github.com/bitcoin/bitcoin/pull/24134). Each time the problem was solved with other means.

The initial solution had some discussion. Also it was discussed on the IRC meeting in #bitcoin-core-builds channel. This PR, actually, is a resurrection of it, as the bug silently struck pretty recently.

The bug is well described in bitcoin/bitcoin#22719.

Here is another, a bit simpler description, which requires only basic shell (bash, dash etc) experience. After creating targets by this code:https://github.com/bitcoin/bitcoin/blob/1e7564eca8a688f39c75540877ec3bdfdde766b1/depends/funcs.mk#L280 a "draft" line of recipe like $($(1)_config_env) $(call $(1)_config_cmds, $(1)) becomes a shell command sequence VAR1=foo VAR2=bar command1 && command2 which is supposed to be executed in a new sub-shell.

Please note that VAR1=foo VAR2=bar part is visible for the first command1 only (for details see shell docs). Example:

$ VAR1="foo" VAR2="bar" echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
begin
$ echo $?
1

Using the export command is a trivial solution:

$ export VAR1="foo" VAR2="bar"; echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
begin
foo
bar
end
$ echo $?
0

As a new sub-shell is invoked for each line of the recipe, there are no side effects of using export. It means this solution should not be considered invasive.

Fixes bitcoin/bitcoin#22719.

Also this PR removes no longer needed crutch from qt.mk.

ACKs for top commit: fanquake: ACK affbf58a1e52a8e60c830be6a9e0347e0ff4c28e

Tree-SHA512: 0ce2cf82870a7774bdf1592fac50857126ae47da902e349f1092d50109223be9d6a8efd5e92ec08c2ca775b17516482aabaf232378950ade36484a883acc177b

wallet: if only have one output type, don't perform "mixed" coin selection

there is nothing to mix.

Merge bitcoin/bitcoin#26658: test: Fix backwards compatibility intermittent failure

c29bff5b91061b565076093b7245dfb1717e97b7 test: Fix backwards compatibility intermittent failure (Aurèle Oulès)

Pull request description:

Fixes #24400.

See https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1341531696 to reproduce the failure.

As MarcoFalke suggested in https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1342282165, it can happen that the wallet is not fully flushed before being copied over to the other node. Fixed by unloading the wallet before copying the file.

ACKs for top commit: Sjors: utACK c29bff5

Tree-SHA512: ea93b859878873d07fa44ccb1c5b4c1b1014be8568dc2b9bbef02bb7a9230fefa7a71b149eb553870d3f4e986263342ea4f996fa0221098a2244f38952100471

Merge bitcoin/bitcoin#26660: test: Use last release in compatibility tests

fabb24cbef6ccccf5e82ac52ca2aafd47c34455a test: Use last release in compatibility tests (MarcoFalke)

Pull request description:

In compatibility tests it makes sense to always use the last release without the new feature, as it is likely more in use than any even older previous release.

ACKs for top commit: Sjors: utACK fabb24c

Tree-SHA512: beb854f4d28ba313282e1e0303abb0e09377828b138bde5a3e209337210b6b4c24855ab90a68f8789387001e4ca33b15cc37dbc9b7809929f4e7d1b69833a527

test: Move feature_backwards_compatibility.py to wallet_backwards_compatibility.py

Update dnf

Create module-haskell.hs

Create function.hs

Create debugger-ghci.hs

Update memory-swap.marcosptf

Update memory-swap.marcosptf

Update memory-swap.marcosptf

Create memory-swap.marcosptf