-
Notifications
You must be signed in to change notification settings - Fork 113
Channel splicing support #677
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
👋 Thanks for assigning @wpaulino as a reviewer! |
|
🔔 1st Reminder Hey @tnull @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
2 similar comments
|
🔔 1st Reminder Hey @tnull @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
|
🔔 1st Reminder Hey @tnull @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
tnull
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Looks mostly good, just a few comments. This will need bindings support, too, and unfortunately also need a rebase by now. Sorry!
| } else { | ||
| log_error!( | ||
| self.logger, | ||
| "Channel not found for user_channel_id: {:?} and counterparty: {}", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pre-existing, but we should probably implement Display for UserChannelId, now opened #681
| Ok(txid) | ||
| } | ||
|
|
||
| pub(crate) fn select_confirmed_utxos( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be cool to be able to reuse this for impl CoinSelectionSource while dropping impl WalletSource going forward. What do we think the necessary changes would be that would allow for this? Probably just making claim_id optional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc: @wpaulino
Hmm... are you saying have LDK Node's Wallet implement CoinSelectionSource directly? Would we implement UTXO locking there? Or are you assuming BDK's wallet will eventually implement this?
To re-use this code, yeah we can extract the Utxo from each FundingTxInput to use in CoinSelection. Though we need to expose that or have a conversion function. Currently, FundingTxInput::utxo is pub(super).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... are you saying have LDK Node's
WalletimplementCoinSelectionSourcedirectly?
Yes, it might be a bit more efficient than the detour over WalletSource, and would allow us to reuse most code, IIUC.
Would we implement UTXO locking there? Or are you assuming BDK's wallet will eventually implement this?
We could, although I still prefer to wait until BDK ships it soon hopefully. Might be debatable if we need to implement it manually in the meantime. However, the UTXO locking discussion seems orthogonal to whether or not to use CoinSelectionSource, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could, although I still prefer to wait until BDK ships it soon hopefully. Might be debatable if we need to implement it manually in the meantime. However, the UTXO locking discussion seems orthogonal to whether or not to use
CoinSelectionSource, no?
IIUC, seems we'd replace the lightning::events::bump_transaction::Wallet parameterization in BumpTransactionEventHandler with LDK Node's Wallet. But the former has some sort of locking already. Or at least it prefers not to double spend when possible.
We may be able to maintain a list of OutPoints to pass to TxBuilder::unspendable (filtered by claim id), attempting with and without the list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may be able to main a list of
OutPoints to pass toTxBuilder::unspendable(filtered by claim id), attempting with and without the list.
Yeah, though note that would require us to track a list of locked OutPoints whenever we generate any Transaction. We'd then need to drop entries whenever they are spent and re-add them in case of a reorg, etc. This might become very complicated real quick, if we want to do it 'properly'.
I think an alternative approach would be to just always act as if we already saw the transaction in the mempool, i.e., call Wallet::apply_unconfirmed_txs, and then the Wallet should do the right thing and eventually mark the transaction as evicted if it wouldn't end up in the mempool afterall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, maybe it would even make sense to go with the last described approach in this PR (also for non-splicing funding transactions). If we do, we might however finally need to handle DiscardFunding, too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think an alternative approach would be to just always act as if we already saw the transaction in the mempool, i.e., call
Wallet::apply_unconfirmed_txs, and then theWalletshould do the right thing and eventually mark the transaction as evicted if it wouldn't end up in the mempool afterall.
Hmmm... would we do this when CoinSelectionSource::sign_psbt is called? We don't have the transaction in CoinSelectionSource::select_confirmed_utxos.
Hmm, maybe it would even make sense to go with the last described approach in this PR (also for non-splicing funding transactions). If we do, we might however finally need to handle
DiscardFunding, too.
Which approach?
src/config.rs
Outdated
| /// users should be aware of the backwards compatibility risk prior to using the functionality. | ||
| /// | ||
| /// [`Node`]: crate::Node | ||
| pub reject_inbound_splices: bool, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIUC, contrary to the docs here and in LDK, this flag is necessary to still support downgrades, i.e., forwards, not backwards compatibility (we might want to fix this/make this more clear in the LDK docs).
However, we currently don't support downgrades in LDK Node anyways, so here we can just omit this flag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think CI is still failing because reject_inbound_splices is still here even though it hasn't been added to the bindings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I'm not sure how consistent we've been with using forward and backward compatibility correctly. For instance, IIUC, should this have said backward compatibility?
And these like reject_inbound_splices should have said forward compatibility?
https://github.com/lightningdevkit/rust-lightning/blob/17f7858f01f29ec904c5bddda799365f4f9ff71a/lightning/src/util/config.rs#L915
https://github.com/lightningdevkit/rust-lightning/blob/17f7858f01f29ec904c5bddda799365f4f9ff71a/lightning/src/util/config.rs#L941
Maybe it would be best to avoid "backward" / "forward" and just say "compatibility" when referencing an earlier version. Seems these could be confusing if it is unclear if the term is relative to older or newer version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think CI is still failing because
reject_inbound_splicesis still here even though it hasn't been added to the bindings.
Yup, gonna drop the commit, so that will fix CI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this have said backward compatibility
I think so, yes.
And these like
reject_inbound_splicesshould have said forward compatibility?
Yes, I think so, too.
Maybe it would be best to avoid "backward" / "forward" and just say "compatibility" when referencing an earlier version. Seems these could be confusing if it is unclear if the term is relative to older or newer version.
Yeah, that sounds reasonable. Seems there often is confusion around it and 'compatibility with versions prior' is much more precise anyways.
tests/integration_tests_rust.rs
Outdated
|
|
||
| #[test] | ||
| fn splice_channel() { | ||
| macro_rules! expect_splice_pending_event { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Usually we'd add this to common/mod.rs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it wasn't being used there, having it there would give a build warning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point about the build warning.
To resolve that and utilize the macro more broadly, what do you think about adding a test for splice-in/splice-out assertions into do_channel_full_cycle? We could use the new expect_splice_pending_event! macro there, which would justify having it in common/mod.rs and make our full-cycle test more robust for splicing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Started looking into this. @tnull The channel funding transaction is recognized as an outbound on-chain payment for A -- presumably because we are spending from the wallet. If B were to splice out paying A (or whomever), should the splice funding transaction be considered an outbound on-chain payment for B? It does not currently since it spends from the funding txo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Started looking into this. @tnull The channel funding transaction is recognized as an outbound on-chain payment for A -- presumably because we are spending from the wallet. If B were to splice out paying A (or whomever), should the splice funding transaction be considered an outbound on-chain payment for B? It does not currently since it spends from the funding txo.
Hmm, yeah, unfortunately that will be the case prior to 0.3 when we get lightningdevkit/rust-lightning#3566. I think having splice-out counting as an outbound onchain transaction makes sense until then. Might be worth noting somewhere that this is the case though, and that PaymentKinds will change in the future. (Tbh. one reason why I think it would make sense to flag splicing support alpha/experimental, simply as the classification API is unstable currently).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... so currently we don't have the Txid until we see an LdkEvent::FundingTransactionReadyForSigning, but we've lost the context of the transaction at that point. We can assume it is a splice (could be dual-funding later), but it might difficult to determine if it is a splice-out. Maybe simply seeing that none of the inputs belong to our wallet would be sufficient? We should only see the event if we have any contributions / are the initiator, IIRC.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... so currently we don't have the
Txiduntil we see anLdkEvent::FundingTransactionReadyForSigning, but we've lost the context of the transaction at that point. We can assume it is a splice (could be dual-funding later), but it might difficult to determine if it is a splice-out. Maybe simply seeing that none of the inputs belong to our wallet would be sufficient? We should only see the event if we have any contributions / are the initiator, IIRC.
Okay. That makes me wonder even more if we should wait for LDK#3566 above to be able to classify them properly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, maybe let's just stick with adding splice-ins (which will be marked outbound payments) for now, while noting this behavior on the splice_in/splice_out methods under an 'Experimental API' section or similiar?
982ce34 to
57ad1c8
Compare
jkczyz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to rebase still.
tests/integration_tests_rust.rs
Outdated
|
|
||
| #[test] | ||
| fn splice_channel() { | ||
| macro_rules! expect_splice_pending_event { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it wasn't being used there, having it there would give a build warning.
|
🔔 2nd Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 2nd Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
57ad1c8 to
281ebce
Compare
chuksys
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm really excited to see Splicing being added to LDK-Node - this is a fantastic feature! I learned a lot reviewing this PR.
I've left two minor, non-blocking comments regarding the CI failure and a test organization suggestion (to utilize the new macro in do_channel_full_cycle).
Great work!
src/config.rs
Outdated
| /// users should be aware of the backwards compatibility risk prior to using the functionality. | ||
| /// | ||
| /// [`Node`]: crate::Node | ||
| pub reject_inbound_splices: bool, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think CI is still failing because reject_inbound_splices is still here even though it hasn't been added to the bindings.
tests/integration_tests_rust.rs
Outdated
|
|
||
| #[test] | ||
| fn splice_channel() { | ||
| macro_rules! expect_splice_pending_event { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point about the build warning.
To resolve that and utilize the macro more broadly, what do you think about adding a test for splice-in/splice-out assertions into do_channel_full_cycle? We could use the new expect_splice_pending_event! macro there, which would justify having it in common/mod.rs and make our full-cycle test more robust for splicing.
281ebce to
adb59c8
Compare
|
🔔 3rd Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 3rd Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
|
🔔 4th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 4th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
|
🔔 5th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 5th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
have some weirdness with events and tx history that could be fixed or at least documented
- spice outs never show up in my history
- splice ins do but do not show the fees
- would be nice if the
ChannelReadyevent could clarify between spliced channels and new channels. New channels means i have a new balance that is usable, this isn't always the case with splices though
| locked_wallet | ||
| .tx_details(txin.previous_output.txid) | ||
| .map(|tx_details| tx_details.tx.deref().clone()) | ||
| .map(|prevtx| FundingTxInput::new_p2wpkh(prevtx, txin.previous_output.vout)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this scared me at first but then found the wallet only uses p2wpkh, should add a comment saying that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added debug assertions.
| /// This provides for increasing a channel's outbound liquidity without re-balancing or closing | ||
| /// it. Once negotiation with the counterparty is complete, the channel remains operational | ||
| /// while waiting for a new funding transaction to confirm. | ||
| pub fn splice_in( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one issue I am having with this is that there is no way to see how much we spend on fees doing this. Would be nice if this function returned it or if it was somehow included in the PaymentDetails
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tnull How should we expose this? Or should we wait until the payment API changes are addressed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tnull How should we expose this? Or should we wait until the payment API changes are addressed?
Yes, I'd be in favor of postponing it for now. How to expose fees for onchain transactions in general is also still an open question (see #578), so we might want to come up with a simliar API design for all cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did some digging on this, and found the issue is actually because we don't have the original channel funding utxo in the wallet when calculating the fee here
Line 194 in 3505dfb
| let fee = locked_wallet.calculate_fee(&wtx.tx_node.tx).unwrap_or(Amount::ZERO); |
can do something like this to fix, not the prettiest though as we have a dummy spk
benthecarman@a51e4b5
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Chatted with @benthecarman offline. Yeah, it sounds like this is what we need to do. If we want the actual txout in the wallet, we'd need a way to access from LDK, possibly just through ChannelDetails. @TheBlueMatt @tnull Would it be worth adding that in LDK 0.2?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summarizing some offline discussions:
- Because the BDK wallet doesn't know about the funding transaction, the payment store will show
Amount::ZEROfee for the splice-in, which it categorized as an outbound on-chain payment - This can be fixed by having the BDK wallet track the funding txo as @benthecarman shows, but requires using fake pubkeys since LDK doesn't expose the channel's funding pubkeys
- Using fake pubkeys may not be ideal because we'd persist these when writing the BDK wallet.
Options:
- Keep the current behavior of showing
Amount::ZEROfor the splice-in fees until it can be done properly in Add APIs to classify channel-related transaction types rust-lightning#3566 - Use fake pubkeys and live with having inaccurate txouts in the persisted BDK wallet
- Expose the funding pubkeys upstream either as part of
ChannelDetailsor in theChannelPendingevent
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I think I'd like to avoid adding fake pubkeys, because I am not fully sure what the consequences would be for BDK-internals. In particular, I fear we we could run into issues in case of reorgs/evictions, as the wallet wouldn't detect anything for the fake pubkey during syncing?
- Expose the funding pubkeys upstream either as part of
ChannelDetailsor in theChannelPendingevent
That would be my preferred option. If we can't get that in 0.2 still, we could just show Amount::ZERO for now, especially given that we'll need to add a huge disclaimer around payment store misclassification anyways. Note however that I don't see how this issue is connected to lightningdevkit/rust-lightning#3566, given that the solution we currently envision there is to simply add a enum BroadcastType to BroadcasterInterface that would allow us to intercept and curate a mapping of <Txid, BroadcastType>...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, right, the fees would still be inaccurate. Any preference to ChannelDetails or ChannelPending? Both are serializable, so we can't avoid an Option. Though ChannelDetails is only persisted when used with phantom payments, IIUC.
adb59c8 to
a715e63
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems tests are still failing:
thread 'splice_channel' panicked at tests/integration_tests_rust.rs:1021:5:
assertion `left == right` failed
left: 4399013
right: 4399056
Huh, I couldn't reproduce that locally (macos) even after rebasing. And re-running the canceled macos job shows it passing. So seem limited to ubuntu. Any idea why that would be? |
When a channel is spliced, the existing funding transaction's output is spent and a new funding transaction output is formed. Once the splice is considered locked by both parties, LDK will emit a ChannelReady event which will include the new funding_txo. Additionally, the initial ChannelReady event now includes the original funding_txo. Include this data in LDK Node's ChannelReady event.
When the interactive-tx construction protocol completes in LDK during splicing (and in the future dual-funding), LDK Node must provide signatures for any non-shared inputs belonging to its on-chain wallet. This commit implements this when handling the corresponding FundingTransactionReadyForSigning event.
Extract the funds availability checking logic from open_channel_inner into a separate method so that it can be reused for channel splicing.
Instead of closing and re-opening a channel when outbound liquidity is exhausted, splicing allows to adding more funds (splice-in) while keeping the channel operational. This commit implements splice-in using funds from the BDK on-chain wallet.
Instead of closing and re-opening a channel when on-chain funds are needed, splicing allows removing funds (splice-out) while keeping the channel operational. This commit implements splice-out sending funds to a user-provided on-chain address.
Since LDK Node does not support downgrades, there's no need to have a Config parameter for accepting inbound splices. Instead, enable it by default.
a715e63 to
913b968
Compare
Ok, my theory is that this is off because LDK will produce a Pushed some fixups to mine a block after receiving the payment as a way to hopefully mitigate this. Not sure if there's a better way. |
|
Doing this in orange so we can have splice outs in our history, not really sure a better way around this. Maybe we could eventually watch the funding txo with bdk but that would complicate calculating balance https://github.com/lightningdevkit/orange-sdk/commit/5e649e9dd6c3f27e3a2e422f52223679a31ed780 |
|
🔔 6th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 6th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
|
🔔 7th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 7th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
|
🔔 8th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
1 similar comment
|
🔔 8th Reminder Hey @TheBlueMatt @wpaulino! This PR has been waiting for your review. |
LDK 0.2 added beta support for splicing. This PR exposes it in LDK Node as two new
Nodemethods:splice_inandsplice_out. Funds used for splicing in are taken from the on-chain BDK wallet. When splicing out, any on-chain address can be provided as the destination.Two new events are provided:
SplicePendingandSpliceFailed. The former is emitted once the new funding transaction has been broadcast. The latter indicates that any contributed inputs may be re-used, though this is not currently exposed. It should be used internally to unlock UTXOs that were intended to be spent.