Skip to content

Conversation

@nipatel-jump
Copy link
Contributor

@nipatel-jump nipatel-jump commented Oct 20, 2025

New Solcap Version

@nipatel-jump nipatel-jump force-pushed the nishk/solcap_new branch 2 times, most recently from 8608f30 to 3b04448 Compare October 24, 2025 18:14
@nipatel-jump nipatel-jump changed the title [WIP] solcap: v2 - shared buffer, capture tile, new solcap format [WIP] solcap: version ULONGMAX Oct 24, 2025
@nipatel-jump nipatel-jump force-pushed the nishk/solcap_new branch 10 times, most recently from 7377f26 to 2a98888 Compare October 29, 2025 16:08
@nipatel-jump nipatel-jump changed the title [WIP] solcap: version ULONGMAX [WIP] solcap v2: updates to solcap Oct 29, 2025
@nipatel-jump nipatel-jump changed the title [WIP] solcap v2: updates to solcap solcap v2: updates to solcap Oct 30, 2025
@nipatel-jump nipatel-jump force-pushed the nishk/solcap_new branch 5 times, most recently from 6c695ee to f9377e0 Compare October 30, 2025 19:39
@github-actions
Copy link

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.076021 s 0.078395 s 3.123%
backtest mainnet-368528500-perf snapshot load 2.69 s 2.76 s 2.602%
backtest mainnet-368528500-perf total elapsed 76.020816 s 78.39545 s 3.124%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

@github-actions
Copy link

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.073266 s 0.075495 s 3.042%
backtest mainnet-368528500-perf snapshot load 2.645 s 2.92 s 10.397% ⚠️
backtest mainnet-368528500-perf total elapsed 73.265757 s 75.494839 s 3.042%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

@github-actions
Copy link

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.072983 s 0.07554 s 3.504%
backtest mainnet-368528500-perf snapshot load 2.674 s 2.973 s 11.182% ⚠️
backtest mainnet-368528500-perf total elapsed 72.983168 s 75.540167 s 3.504%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

@github-actions
Copy link

github-actions bot commented Nov 4, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.052719 s 0.054349 s 3.092%
backtest mainnet-368528500-perf snapshot load 1.72 s 1.775 s 3.198%
backtest mainnet-368528500-perf total elapsed 52.719006 s 54.349027 s 3.092%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

Copy link
Contributor

@ibhatt-jumptrading ibhatt-jumptrading left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic looks good, just need to clean up the code

i don't have enough expertise to review the stem changes

@github-actions
Copy link

github-actions bot commented Nov 4, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.076078 s 0.075953 s -0.164%
backtest mainnet-368528500-perf snapshot load 2.667 s 2.713 s 1.725%
backtest mainnet-368528500-perf total elapsed 76.078491 s 75.953372 s -0.164%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.061401 s 0.050514 s -17.731%
backtest mainnet-368528500-perf snapshot load 1.766 s 1.562 s -11.552%
backtest mainnet-368528500-perf total elapsed 61.401327 s 50.513764 s -17.732%
firedancer mem usage with mainnet.toml 974.13 GiB 962.13 GiB -1.232%

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.063654 s 0.06385 s 0.308%
backtest mainnet-368528500-perf snapshot load 2.165 s 2.4 s 10.855% ⚠️
backtest mainnet-368528500-perf total elapsed 63.65425 s 63.849559 s 0.307%
firedancer mem usage with mainnet.toml 974.15 GiB 962.15 GiB -1.232%

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.053016 s 0.052553 s -0.873%
backtest mainnet-368528500-perf snapshot load 1.679 s 1.762 s 4.943%
backtest mainnet-368528500-perf total elapsed 53.016322 s 52.553258 s -0.873%
firedancer mem usage with mainnet.toml 974.15 GiB 962.15 GiB -1.232%

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.052727 s 0.052739 s 0.023%
backtest mainnet-368528500-perf snapshot load 1.762 s 1.746 s -0.908%
backtest mainnet-368528500-perf total elapsed 52.727425 s 52.738707 s 0.021%
firedancer mem usage with mainnet.toml 974.15 GiB 962.15 GiB -1.232%

"type": "struct",
"fields": [
{ "name": "lamports", "type": "ulong" },
{ "name": "rent_epoch", "type": "ulong" },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did this change?

/* Add the capture tile to topo */
/**********************************************************************/
if (solcap_enabled) {
fd_topob_wksp( topo, "captur" );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to capture anything in backtest?

.slot = (uint32_t)msg_hdr->slot,
.txn_idx = msg_hdr->txn_idx
};
fwrite( &int_hdr, sizeof(fd_solcap_chunk_int_hdr_t), 1UL, file );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need error code checking everywhere for stuff like this, even just FD_TEST the result is not err

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is in one of the commits on the other branch which i will squash into this commit after fixing comments from this review.

sizeof(fd_solcap_chunk_ftr_t));

/* Serialize account chunk metadata */
uint32_t block_len = (uint32_t)((unaligned_block_len + 3UL) & ~3UL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
uint32_t block_len = (uint32_t)((unaligned_block_len + 3UL) & ~3UL);
uint block_len = (uint)((unaligned_block_len + 3UL) & ~3UL);

Use fd types everywhere please


ctx->gui_enabled = fd_topo_find_tile( topo, "gui", 0UL )!=ULONG_MAX;

if( FD_UNLIKELY( strcmp( "", tile->replay.solcap_capture ) ) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is any of this for? It does not look like it's used

fd_solcap_write_ftr( ctx->capture_ctx->capture, ctx->block_len );

if (ctx->msg_set_sig == SOLCAP_WRITE_BANK_PREIMAGE) {
fflush(ctx->file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check errors

writer->account_idx = 0UL;
return 0;
FILE*
fd_solcap_file_verify( fd_solcap_writer_t * writer ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this


ctl = fd_frag_meta_ctl( 0UL, 0UL, is_last ? 1UL : 0UL, 0UL );

fd_mcache_publish( buf->mcache, buf->depth, buf->seq, 0UL, buf->chunk, msg_sz, ctl, 0UL, 0UL );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can just send the sz here

ulong slot,
uchar const * data,
ulong data_sz) {
ctx->capture_link->vt->write_account_update(ctx, txn_idx, key, info, slot, data, data_sz);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FD_TEST( ctx );

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants