Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Composer V2 #2873

Merged
merged 67 commits into from
Jan 3, 2025
Merged
Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
00a2fa3
Introduce composer.py
Ouziel Sep 27, 2024
b4f2300
Add unit tests
Ouziel Sep 30, 2024
c6e0438
Merge branch 'develop' into composer
Ouziel Oct 3, 2024
106405c
fix test
Ouziel Oct 3, 2024
32a168d
tweaks
Ouziel Oct 3, 2024
5a8ea0a
Migrate to 'bitcoinutils' which supports taproot addresses
Ouziel Oct 3, 2024
45fa9fc
migrate get vsize
Ouziel Oct 3, 2024
d3a9a70
Merge branch 'develop' into composer
Ouziel Oct 3, 2024
59ab8ab
more unit tests
Ouziel Oct 3, 2024
0289a65
Merge branch 'develop' into composer
Ouziel Dec 19, 2024
d577e98
fix pytest
Ouziel Dec 19, 2024
b9ec24e
Composer v2 progress
Ouziel Dec 23, 2024
9c1b06d
composer v2 progress
Ouziel Dec 23, 2024
3e0d203
Add utxo locks
Ouziel Dec 24, 2024
641bc8a
Add more_outputs params
Ouziel Dec 24, 2024
920a193
Tweaks
Ouziel Dec 24, 2024
9f55b5f
ensure unspent_list is complete
Ouziel Dec 24, 2024
ff04ff7
tweaks
Ouziel Dec 24, 2024
88a6953
Correctly set has_segwit parameter
Ouziel Dec 24, 2024
511e84b
backward compatibilities
Ouziel Dec 24, 2024
c14dd45
tweaks
Ouziel Dec 25, 2024
5ab194a
Merge branch 'develop' into composer
Ouziel Dec 25, 2024
466b77f
fixes
Ouziel Dec 25, 2024
3d3326a
More fixes; Regtest OK
Ouziel Dec 26, 2024
c74f83e
Merge branch 'develop' into composer
Ouziel Dec 26, 2024
23dfc8d
More fixes; property tests OK
Ouziel Dec 26, 2024
53f9500
Cleaning; no need to pass inputs_set and keys
Ouziel Dec 26, 2024
65cb56e
Delete old code; progress in fixing tests
Ouziel Dec 27, 2024
1a1e8dd
Merge branch 'develop' into composer
Ouziel Dec 27, 2024
327f762
Merge branch 'develop' into composer
Ouziel Dec 28, 2024
b654b2c
pytest fixing progress
Ouziel Dec 28, 2024
feda13e
fix typo; pytest fixing progress
Ouziel Dec 28, 2024
b8a0676
progress
Ouziel Dec 28, 2024
1fc79e6
pytest fixing progress
Ouziel Dec 29, 2024
15a44f6
fix existing pytest
Ouziel Dec 29, 2024
64f25d8
Add function to check transaction sanity
Ouziel Dec 29, 2024
15d2cd6
cleaning
Ouziel Dec 29, 2024
c4a120a
More fixes, more tweaks
Ouziel Dec 30, 2024
95cf95e
fixes; Clean construct_params; add warnings
Ouziel Dec 30, 2024
7e07197
restore pubkeys parameters
Ouziel Dec 30, 2024
4d8c3cf
restore and fix test with p2sh source
Ouziel Dec 30, 2024
5c014e7
clean debug
Ouziel Dec 30, 2024
5c84f45
fix get pubkeys
Ouziel Dec 30, 2024
3f7a66e
tweak error messges for invalid utxos
Ouziel Dec 30, 2024
58bb8da
fix typo
Ouziel Dec 31, 2024
586b87a
tweak with balance checking
Ouziel Dec 31, 2024
e2b3745
re-enable and fix COMPOSER_VECTOR; fix regtest; fixes and tweaks
Ouziel Dec 31, 2024
419fe65
tweaks
Ouziel Dec 31, 2024
16d874b
exclude silently utxo with balances when inputs_set is not provided; …
Ouziel Dec 31, 2024
05caf3e
fix missing params and name in compose result
Ouziel Dec 31, 2024
2abe695
Don't retry RPC calls on API requests
Ouziel Dec 31, 2024
44a6c2b
fix pytest
Ouziel Dec 31, 2024
15ff5ac
Merge branch 'develop' into composer
Ouziel Dec 31, 2024
405a506
more tests; more fixes
Ouziel Dec 31, 2024
0d30d8f
More tests
Ouziel Jan 1, 2025
92409e7
more tests
Ouziel Jan 1, 2025
574cb0c
last unit tests for composer.py
Ouziel Jan 2, 2025
2d527ae
update release notes
Ouziel Jan 2, 2025
b3aa413
fix pytest
Ouziel Jan 2, 2025
3f56246
Insert dummy signatures before needed fee calculation
Ouziel Jan 2, 2025
9c3fc90
Calculate fees with adjusted vsize
Ouziel Jan 3, 2025
59348a8
fix tests
Ouziel Jan 3, 2025
d1ed85d
Add utxo_value parameter for attach and move
Ouziel Jan 3, 2025
5854ac3
Tweak docs; more tests; fix regtest
Ouziel Jan 3, 2025
acfbf06
more sigops tests
Ouziel Jan 3, 2025
aa4fca3
fix typo
Ouziel Jan 3, 2025
35d1a46
fix typo
Ouziel Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add utxo locks
Ouziel committed Dec 24, 2024
commit 3e0d2030302d7929ab6698dcb55346fb42822047
40 changes: 38 additions & 2 deletions counterparty-core/counterpartycore/lib/composer.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import binascii
import time
from collections import OrderedDict

from bitcoinutils.keys import P2pkhAddress, P2wpkhAddress, PublicKey
from bitcoinutils.script import Script, b_to_h
@@ -51,7 +53,7 @@
encoding = "opreturn"
else:
encoding = "multisig"
if encoding not in ("multisig", "opreturn"):
raise exceptions.TransactionError(f"Not supported encoding: {encoding}")
return encoding

@@ -73,7 +75,7 @@
try:
PublicKey.from_hex(pubkey).get_address(compressed=True).to_string()
return True
except Exception:
return False


@@ -158,6 +160,33 @@
################


class UTXOLocks(metaclass=util.SingletonMeta):
def __init__(self):
self.locks = OrderedDict()
self.max_age = config.UTXO_LOCKS_MAX_AGE
self.max_size = config.UTXO_LOCKS_MAX_ADDRESSES

def lock(self, utxo):
self.locks[utxo] = time.time()
if len(self.locks) > self.max_size:
self.locks.popitem(last=False)

def locked(self, utxo):
if utxo not in self.locks:
return False
if time.time() - self.locks[utxo] > self.max_age:
del self.locks[utxo]
return False
return True

def filter_unspent_list(self, unspent_list):
return [utxo for utxo in unspent_list if not self.locked(f"{utxo['txid']}:{utxo['vout']}")]

def lock_inputs(self, inputs):
for input in inputs:
self.lock(f"{input.txid}:{input.txout_index}")


def list_unspent(source, allow_unconfirmed_inputs):
min_conf = 0 if allow_unconfirmed_inputs else 1

@@ -169,7 +198,7 @@
pass

# then try with Electrs
if len(unspent_list) == 0 and config.ELECTRS_URL is None:
raise exceptions.ComposeError(
"No UTXOs found with Bitcoin Core and Electr is not configured, use the `inputs_set` parameter to provide UTXOs"
)
@@ -258,7 +287,7 @@
return new_unspent_list


def filter_utxos_with_balances(db, unspent_list, construct_params):
use_utxos_with_balances = construct_params.get("use_utxos_with_balances", False)
if use_utxos_with_balances:
return unspent_list
@@ -268,7 +297,7 @@
for utxo in unspent_list:
str_input = f"{utxo['txid']}:{utxo['vout']}"
utxo_balances = ledger.get_utxo_balances(db, str_input)
with_balances = len(utxo_balances) > 0 and any(
[balance["quantity"] > 0 for balance in utxo_balances]
)
if exclude_utxos_with_balances and with_balances:
@@ -303,6 +332,10 @@
if f"{utxo['txid']}:{utxo['vout']}" not in exclude_utxos_list
]

# excluded locked utxos
if not construct_params.get("disable_utxo_lock", False):
unspent_list = UTXOLocks().filter_unspent_list(unspent_list)

# exclude utxos with balances if needed
filter_utxos_with_balances(db, unspent_list, construct_params)

@@ -395,7 +428,7 @@
# if change is enough for exact_fee, add change output and break
if exact_fee is not None:
change_amount = change_amount - exact_fee
if change_amount > config.REGULAR_DUST_SIZE:
change_outputs.append(TxOutput(change_amount, get_script(source)))
break

@@ -407,7 +440,7 @@
# if change is enough for needed fee, add change output and break
if change_amount > needed_fee:
change_amount = change_amount - needed_fee
if change_amount > config.REGULAR_DUST_SIZE:
change_outputs.append(TxOutput(change_amount, get_script(source)))
break
# else try with more inputs
@@ -416,7 +449,7 @@
return inputs, btc_in, change_outputs


"""
+encoding
+validate
+verbose
@@ -431,11 +464,11 @@
+exclude_utxos
+use_utxos_with_balances
+exclude_utxos_with_balances
disable_utxo_lock
+disable_utxo_lock

+mutlisig_pubkey

more_outputs
-more_outputs

regular_dust_size (removed)
multisig_dust_size (removed)
@@ -471,13 +504,16 @@
source, outputs, unspent_list, construct_params
)

if not construct_params.get("disable_utxo_lock", False):
UTXOLocks().lock_inputs(inputs)

# construct transaction
tx = Transaction(inputs, outputs + change_outputs)
btc_out = sum(output.nValue for output in outputs)
btc_change = sum(change_output.nValue for change_output in change_outputs)
verbose = construct_params.get("validate", False)

if verbose:
unsigned_tx_hex = tx.serialize().hex()
return {
"btc_in": btc_in,
Loading