change by price – Shedding changed transactions when watching the mempool

I’m attempting to seize all of the changed transactions. I’ve complied Bitcon Core with tracing enabled and hearken to the mempool:changed as steered on this query.

This methodology works as anticipated and I can get the changed transactions. Nevertheless, in some instances (about 20% of all of the transactions I seize) the tracepoint alerts me of the transaction being changed, however then I’m not capturing the substitute of that transaction.

I can consider two explanation why that is taking place:

1. It may very well be that a few of the replacements don’t enter my mempool. But when that’s the case, how does the tracepoint is aware of that the transactions is being changed? Additionally, it’s bizarre that this may occur with so many transactions (20% as defined earlier).
2. The second choice is that the script that course of the transaction just isn’t contemplating some instances, which leads to dropping transactions. The script is a modified model of this instance. I’ve modified it so I can seize the youngsters of the changed transactions as properly. The info I collect with this script is later saved in a pkl file, which is why it has a shared checklist.

I do not know why I’m dropping so many transactions, and I used to be questioning if somebody may assist see what I’m doing unsuitable.

Listed below are the modified features:

       def handle_replaced(_, knowledge, measurement):
          # Set up a brand new RPC connection for this occasion
          rpc_connection_replaced = connection()
          occasion = bpf["replaced_events"].occasion(knowledge)
          hash_replaced = bytes(occasion.replaced_hash)[::-1].hex()
          hash_new = bytes(occasion.replacement_hash)[::-1].hex()
          tx_time = get_timestamp()
          
          hex_tx = rpc_connection_replaced.getrawtransaction(hash_new)
          new_tx = rpc_connection_replaced.decoderawtransaction(hex_tx)
          new_tx['hex'] = hex_tx
          # Decide guardian transactions which might be nonetheless within the mempool and should not the transaction itself
          mother and father = set([x['txid'] for x in new_tx['vin'] if x['txid'] in mempool.keys() and x['txid'] != new_tx['txid']])
          # Retrieve the previous transaction information from the mempool 
          previous = mempool.get(hash_replaced, None)
          if previous just isn't None:  # Some transactions could also be lacking initially
            mempool.pop(hash_replaced, None)
            # Share the changed occasion particulars by way of list_shared queue
            list_shared.put((previous, [new_tx, tx_time, parents]))
            # Discover all youngster transactions that reference the brand new transaction as a guardian
            childs = [i[0]['txid'] for i in mempool.values() if new_tx['txid'] in i[2]]
            
            # Recursive operate to gather youngster transaction IDs
            def youngster(ll):
              if len(ll) == 0:
                return []
              new_childs = [i[0]['txid'] for i in mempool.values() if ll - i[2] != ll]
              return checklist(ll) + new_childs + youngster(set(new_childs))
            
            if len(childs) > 0:
              childs = youngster(set(childs))
              # Share the connection between the brand new transaction and the primary youngster
              list_shared.put(([new_tx, tx_time, parents], mempool[childs[0]]))
              # Share connections between subsequent youngster transactions
              for i in vary(len(childs)-1):
                list_shared.put((mempool[childs[i]], mempool[childs[i+1]]))

          # Add the brand new transaction to the mempool with its timestamp and guardian set
          mempool[new_tx['txid']] = [new_tx, tx_time, parents]
          logger.information('-----------')
          logger.information('New RBF!')        
          
      def handle_added(_, knowledge, measurement):
          rpc_connection_added = connection()
          occasion = bpf["added_events"].occasion(knowledge)
          hash_new = bytes(occasion.hash)[::-1].hex()
          hex_tx = rpc_connection_added.getrawtransaction(hash_new)
          tx_raw = rpc_connection_added.decoderawtransaction(hex_tx)
          tx_raw['hex'] = hex_tx
          mother and father = set([x['txid'] for x in tx_raw['vin'] if x['txid'] in mempool.keys()])
          mempool[tx_raw['txid']] = [tx_raw, get_timestamp(), parents]
          
      def handle_removed(_, knowledge, measurement):
        occasion = bpf["removed_events"].occasion(knowledge)
        if occasion.purpose != b'changed':
            txid_rem = bytes(occasion.hash)[::-1].hex()
                    
            keys = mempool.keys()
            if txid_rem in keys:
              mempool.pop(txid_rem) 
              logger.information('-----------')
              logger.information(f'Eliminated. Motive:{occasion.purpose}')

I’m attempting to seize all of the changed transactions. I’ve complied Bitcon Core with tracing enabled and hearken to the mempool:changed as steered on this query.

This methodology works as anticipated and I can get the changed transactions. Nevertheless, in some instances (about 20% of all of the transactions I seize) the tracepoint alerts me of the transaction being changed, however then I’m not capturing the substitute of that transaction.

I can consider two explanation why that is taking place:

1. It may very well be that a few of the replacements don’t enter my mempool. But when that’s the case, how does the tracepoint is aware of that the transactions is being changed? Additionally, it’s bizarre that this may occur with so many transactions (20% as defined earlier).
2. The second choice is that the script that course of the transaction just isn’t contemplating some instances, which leads to dropping transactions. The script is a modified model of this instance. I’ve modified it so I can seize the youngsters of the changed transactions as properly. The info I collect with this script is later saved in a pkl file, which is why it has a shared checklist.

I do not know why I’m dropping so many transactions, and I used to be questioning if somebody may assist see what I’m doing unsuitable.

Listed below are the modified features:

       def handle_replaced(_, knowledge, measurement):
          # Set up a brand new RPC connection for this occasion
          rpc_connection_replaced = connection()
          occasion = bpf["replaced_events"].occasion(knowledge)
          hash_replaced = bytes(occasion.replaced_hash)[::-1].hex()
          hash_new = bytes(occasion.replacement_hash)[::-1].hex()
          tx_time = get_timestamp()
          
          hex_tx = rpc_connection_replaced.getrawtransaction(hash_new)
          new_tx = rpc_connection_replaced.decoderawtransaction(hex_tx)
          new_tx['hex'] = hex_tx
          # Decide guardian transactions which might be nonetheless within the mempool and should not the transaction itself
          mother and father = set([x['txid'] for x in new_tx['vin'] if x['txid'] in mempool.keys() and x['txid'] != new_tx['txid']])
          # Retrieve the previous transaction information from the mempool 
          previous = mempool.get(hash_replaced, None)
          if previous just isn't None:  # Some transactions could also be lacking initially
            mempool.pop(hash_replaced, None)
            # Share the changed occasion particulars by way of list_shared queue
            list_shared.put((previous, [new_tx, tx_time, parents]))
            # Discover all youngster transactions that reference the brand new transaction as a guardian
            childs = [i[0]['txid'] for i in mempool.values() if new_tx['txid'] in i[2]]
            
            # Recursive operate to gather youngster transaction IDs
            def youngster(ll):
              if len(ll) == 0:
                return []
              new_childs = [i[0]['txid'] for i in mempool.values() if ll - i[2] != ll]
              return checklist(ll) + new_childs + youngster(set(new_childs))
            
            if len(childs) > 0:
              childs = youngster(set(childs))
              # Share the connection between the brand new transaction and the primary youngster
              list_shared.put(([new_tx, tx_time, parents], mempool[childs[0]]))
              # Share connections between subsequent youngster transactions
              for i in vary(len(childs)-1):
                list_shared.put((mempool[childs[i]], mempool[childs[i+1]]))

          # Add the brand new transaction to the mempool with its timestamp and guardian set
          mempool[new_tx['txid']] = [new_tx, tx_time, parents]
          logger.information('-----------')
          logger.information('New RBF!')        
          
      def handle_added(_, knowledge, measurement):
          rpc_connection_added = connection()
          occasion = bpf["added_events"].occasion(knowledge)
          hash_new = bytes(occasion.hash)[::-1].hex()
          hex_tx = rpc_connection_added.getrawtransaction(hash_new)
          tx_raw = rpc_connection_added.decoderawtransaction(hex_tx)
          tx_raw['hex'] = hex_tx
          mother and father = set([x['txid'] for x in tx_raw['vin'] if x['txid'] in mempool.keys()])
          mempool[tx_raw['txid']] = [tx_raw, get_timestamp(), parents]
          
      def handle_removed(_, knowledge, measurement):
        occasion = bpf["removed_events"].occasion(knowledge)
        if occasion.purpose != b'changed':
            txid_rem = bytes(occasion.hash)[::-1].hex()
                    
            keys = mempool.keys()
            if txid_rem in keys:
              mempool.pop(txid_rem) 
              logger.information('-----------')
              logger.information(f'Eliminated. Motive:{occasion.purpose}')