⚡️ Speed up method xt.parse_trade by 11%
#49
+4,686
−2,563
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄 11% (0.11x) speedup for
xt.parse_tradeinpython/ccxt/async_support/xt.py⏱️ Runtime :
2.21 milliseconds→1.98 milliseconds(best of62runs)📝 Explanation and details
The optimized code achieves an 11% speedup by implementing several key performance optimizations focused on reducing dictionary lookups and method call overhead:
Primary Optimizations:
Fast-path dictionary access in
safe_stringmethods: The original code always calledExchange.key_exists()which involveshasattr()checks and exception handling. The optimized version adds a fast path for dict objects using directdictionary[key]access with try-catch, avoiding the expensivekey_existsmethod in the common case.Inlined
safe_string_2logic: Instead of callingsafe_either()which creates additional function call overhead, the optimized version directly implements the fallback logic, reducing call stack depth.Optimized
safe_string_nandsafe_integer_n: For dictionary objects (the most common case), these now use directdict.get()calls and inline the iteration logic rather than calling the genericget_object_value_from_key_list()helper.Faster
iso8601formatting: Replaced the expensivestrftime('%Y-%m-%dT%H:%M:%S.%f')[:-6] + format()pattern with separatestrftimeand f-string concatenation, avoiding substring operations and redundant formatting.Optimized
safe_bool: Added fast path for dict objects usingdict.get()and direct type checking, avoiding the more expensivesafe_bool_ncall.Local variable caching in
safe_market: Storesself.markets_by_idin a local variable and uses directdict.get()access patterns to reduce repeated attribute lookups.Streamlined currency code extraction in
parse_trade: Extracts fee currency codes once and reuses the result, and moves expensivenumber_to_string(market['contractSize'])outside conditional logic.Performance Impact:
The test results show consistent 9-17% improvements across various trade parsing scenarios, with the largest gains in contract trades (11.7-12.9% faster) where dictionary access patterns are most intensive. The optimizations particularly benefit workloads with frequent trade parsing, as evidenced by all test cases showing measurable improvements while maintaining identical functionality.
These optimizations focus on the hot path identified by the profiler - dictionary access operations that account for over 40% of total runtime in the original implementation.
✅ Correctness verification report:
🌀 Generated Regression Tests and Runtime
import pytest
from ccxt.async_support.xt import xt
--- Unit tests for xt.parse_trade ---
@pytest.fixture
def xt_instance():
return xt()
----------------------- Basic Test Cases -----------------------
def test_spot_public_trade_basic(xt_instance):
# Basic spot trade from fetchTrades
trade = {
"i": 203530723141917063,
"t": 1678227505815,
"p": "22038.81",
"q": "0.000978",
"v": "21.55395618",
"b": True
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 66.7μs -> 61.1μs (9.26% faster)
def test_spot_ws_trade_basic(xt_instance):
# Spot trade from watchTrades
trade = {
"s": "btc_usdt",
"i": "228825383103928709",
"t": 1684258222702,
"p": "27003.65",
"q": "0.000796",
"b": True
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 73.0μs -> 66.0μs (10.6% faster)
def test_spot_mytrade_basic(xt_instance):
# Spot trade from fetchMyTrades
trade = {
"symbol": "btc_usdt",
"tradeId": "206906233569974658",
"orderId": "206906233178463488",
"orderSide": "SELL",
"orderType": "MARKET",
"bizType": "SPOT",
"time": 1679032290215,
"price": "25703.46",
"quantity": "0.000099",
"quoteQty": "2.54464254",
"baseCurrency": "btc",
"quoteCurrency": "usdt",
"fee": "0.00508929",
"feeCurrency": "usdt",
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 77.4μs -> 70.6μs (9.68% faster)
def test_contract_public_trade_basic(xt_instance):
# Contract trade from fetchTrades
trade = {
"t": 1678227683897,
"s": "btc_usdt",
"p": "22031",
"a": "1067",
"m": "BID"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 64.8μs -> 58.0μs (11.7% faster)
def test_contract_mytrade_basic(xt_instance):
# Contract trade from fetchMyTrades
trade = {
"orderId": "207260566170987200",
"execId": "207260566790603265",
"symbol": "btc_usdt",
"quantity": "13",
"price": "27368",
"fee": "0.02134704",
"feeCoin": "usdt",
"timestamp": 1679116769838,
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 69.8μs -> 61.8μs (12.8% faster)
----------------------- Edge Test Cases -----------------------
def test_missing_optional_fields(xt_instance):
# Missing optional fields (fee, orderType, takerMaker, etc.)
trade = {
"s": "eth_usdt",
"i": "123456789",
"t": 1684258222702,
"p": "1800.50",
"q": "0.5",
"b": False
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 73.2μs -> 65.5μs (11.6% faster)
def test_missing_side_and_maker_fields(xt_instance):
# No side, no takerMaker, no isMaker, but has m field
trade = {
"t": 1678227683897,
"s": "btc_usdt",
"p": "22031",
"a": "1067",
"m": "ASK"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 65.1μs -> 57.9μs (12.5% faster)
def test_contract_trade_missing_quantity(xt_instance):
# Contract trade with missing 'quantity', using 'a'
trade = {
"t": 1678227683897,
"s": "btc_usdt",
"p": "22031",
"a": "1067",
"m": "BID"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 64.7μs -> 58.1μs (11.5% faster)
def test_fee_currency_none(xt_instance):
# Fee currency is None
trade = {
"symbol": "btc_usdt",
"tradeId": "206906233569974658",
"orderId": "206906233178463488",
"orderSide": "SELL",
"orderType": "MARKET",
"bizType": "SPOT",
"time": 1679032290215,
"price": "25703.46",
"quantity": "0.000099",
"fee": "0.00508929",
"feeCurrency": None,
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 78.5μs -> 71.4μs (9.94% faster)
def test_fee_is_zero_string(xt_instance):
# Fee is "0"
trade = {
"symbol": "btc_usdt",
"tradeId": "206906233569974658",
"orderId": "206906233178463488",
"orderSide": "SELL",
"orderType": "MARKET",
"bizType": "SPOT",
"time": 1679032290215,
"price": "25703.46",
"quantity": "0.000099",
"fee": "0",
"feeCurrency": "usdt",
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 76.4μs -> 69.4μs (10.1% faster)
def test_missing_price_and_quantity(xt_instance):
# Missing price and quantity
trade = {
"symbol": "btc_usdt",
"tradeId": "206906233569974658",
"orderId": "206906233178463488",
"orderSide": "SELL",
"orderType": "MARKET",
"bizType": "SPOT",
"time": 1679032290215,
"fee": "0.00508929",
"feeCurrency": "usdt",
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 67.5μs -> 61.1μs (10.6% faster)
def test_all_fields_none(xt_instance):
# All fields are None
trade = {
"symbol": None,
"tradeId": None,
"orderId": None,
"orderSide": None,
"orderType": None,
"bizType": None,
"time": None,
"price": None,
"quantity": None,
"fee": None,
"feeCurrency": None,
"takerMaker": None
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 36.1μs -> 33.4μs (7.90% faster)
def test_ws_swap_trade(xt_instance):
# Websocket swap trade with isMaker field
trade = {
'fee': '0.04080840',
'isMaker': False,
'marginUnfrozen': '0.75711984',
'orderId': '376172779053188416',
'orderSide': 'BUY',
'positionSide': 'LONG',
'price': '3400.70',
'quantity': '2',
'symbol': 'eth_usdt',
'timestamp': 1719388579622
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 70.6μs -> 62.5μs (13.0% faster)
----------------------- Large Scale Test Cases -----------------------
def test_large_batch_spot_trades(xt_instance):
# Test parsing a large batch of spot trades
trades = []
for i in range(1000):
trades.append({
"i": str(i),
"t": 1678227505815 + i,
"p": str(22000 + i),
"q": str(0.001 * i),
"b": i % 2 == 0
})
results = [xt_instance.parse_trade(trade) for trade in trades]
for i, result in enumerate(results):
pass
def test_large_batch_contract_trades(xt_instance):
# Test parsing a large batch of contract trades
trades = []
for i in range(1000):
trades.append({
"t": 1678227683897 + i,
"s": "btc_usdt",
"p": str(22000 + i),
"a": str(i),
"m": "BID" if i % 2 == 0 else "ASK"
})
results = [xt_instance.parse_trade(trade) for trade in trades]
for i, result in enumerate(results):
pass
def test_large_batch_missing_fields(xt_instance):
# Large batch with missing optional fields
trades = []
for i in range(1000):
trades.append({
"i": str(i),
"t": 1678227505815 + i,
"p": str(22000 + i),
"q": str(0.001 * i),
# missing 'b', 'fee', etc.
})
results = [xt_instance.parse_trade(trade) for trade in trades]
for i, result in enumerate(results):
pass
codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
import pytest
from ccxt.async_support.xt import xt
--- Function to test: xt.parse_trade ---
Minimal stub to allow test execution, imports above are assumed available.
class DummyMarket(dict):
def init(self, symbol, type='spot', contractSize=1):
super().init()
self['symbol'] = symbol
self['type'] = type
self['contractSize'] = contractSize
from ccxt.async_support.xt import xt
--- Unit Tests ---
@pytest.fixture
def xt_instance():
return xt()
1. Basic Test Cases
def test_parse_trade_spot_fetch_trades_basic(xt_instance):
# Spot trade with all basic fields
trade = {
"i": "203530723141917063",
"t": 1678227505815,
"p": "22038.81",
"q": "0.000978",
"v": "21.55395618",
"b": True
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 66.9μs -> 60.9μs (9.89% faster)
def test_parse_trade_spot_watch_trades_basic(xt_instance):
# Spot trade with string trade id and missing some fields
trade = {
"s": "btc_usdt",
"i": "228825383103928709",
"t": 1684258222702,
"p": "27003.65",
"q": "0.000796",
"b": True
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 73.0μs -> 65.9μs (10.8% faster)
def test_parse_trade_spot_watch_my_trades_basic(xt_instance):
# Spot trade with orderId
trade = {
"s": "btc_usdt",
"t": 1656043204763,
"i": "6316559590087251233",
"oi": "6216559590087220004",
"p": "30000",
"q": "3",
"v": "90000"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 73.2μs -> 66.1μs (10.7% faster)
def test_parse_trade_contract_fetch_trades_basic(xt_instance):
# Contract trade with contractSize
trade = {
"t": 1678227683897,
"s": "btc_usdt",
"p": "22031",
"a": "1067",
"m": "BID"
}
market = DummyMarket('BTC/USDT', type='contract', contractSize=0.001)
codeflash_output = xt_instance.parse_trade(trade, market); result = codeflash_output # 64.8μs -> 57.3μs (12.9% faster)
def test_parse_trade_spot_fetch_my_trades_basic(xt_instance):
# Spot trade with fee and currencies
trade = {
"symbol": "btc_usdt",
"tradeId": "206906233569974658",
"orderId": "206906233178463488",
"orderSide": "SELL",
"orderType": "MARKET",
"bizType": "SPOT",
"time": 1679032290215,
"price": "25703.46",
"quantity": "0.000099",
"quoteQty": "2.54464254",
"baseCurrency": "btc",
"quoteCurrency": "usdt",
"fee": "0.00508929",
"feeCurrency": "usdt",
"takerMaker": "TAKER"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 77.6μs -> 70.6μs (10.00% faster)
def test_parse_trade_contract_fetch_my_trades_basic(xt_instance):
# Contract trade with feeCoin
trade = {
"orderId": "207260566170987200",
"execId": "207260566790603265",
"symbol": "btc_usdt",
"quantity": "13",
"price": "27368",
"fee": "0.02134704",
"feeCoin": "usdt",
"timestamp": 1679116769838,
"takerMaker": "TAKER"
}
market = DummyMarket('BTC/USDT', type='contract', contractSize=1)
codeflash_output = xt_instance.parse_trade(trade, market); result = codeflash_output # 69.4μs -> 62.3μs (11.3% faster)
2. Edge Test Cases
def test_parse_trade_missing_fields(xt_instance):
# Missing price, quantity, fee
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"time": 1679032290215,
"orderSide": "BUY"
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 64.0μs -> 57.4μs (11.4% faster)
def test_parse_trade_fee_currency_not_in_currencies(xt_instance):
# Fee currency not in currencies_by_id
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"fee": "0.001",
"feeCurrency": "doge",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 69.5μs -> 62.1μs (12.0% faster)
def test_parse_trade_fee_coin(xt_instance):
# Fee coin instead of feeCurrency
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"fee": "0.002",
"feeCoin": "usdt",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 69.0μs -> 62.0μs (11.3% faster)
def test_parse_trade_side_bid_or_ask(xt_instance):
# Side from 'm' field
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"m": "ASK",
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 64.7μs -> 58.0μs (11.6% faster)
trade['m'] = "BID"
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 33.1μs -> 28.4μs (16.7% faster)
def test_parse_trade_is_maker_field(xt_instance):
# isMaker field
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"isMaker": True,
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 65.2μs -> 57.8μs (12.7% faster)
trade['isMaker'] = False
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 32.5μs -> 28.6μs (13.5% faster)
def test_parse_trade_contract_no_quantity_a_field(xt_instance):
# Contract trade, no quantity, but 'a' field
trade = {
"symbol": "eth_usdt",
"a": "5",
"price": "2000",
"m": "BID",
"timestamp": 1679116769838
}
market = DummyMarket('ETH/USDT', type='contract', contractSize=0.1)
codeflash_output = xt_instance.parse_trade(trade, market); result = codeflash_output # 64.9μs -> 58.2μs (11.6% faster)
def test_parse_trade_unusual_types_and_values(xt_instance):
# Unusual types: int, float, None, empty string
trade = {
"symbol": "btc_usdt",
"tradeId": 123456789,
"price": 100.0,
"quantity": "",
"fee": None,
"feeCurrency": None,
"time": None
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 49.7μs -> 45.2μs (10.1% faster)
def test_parse_trade_side_case_insensitive(xt_instance):
# Side is case-insensitive
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"orderSide": "SELL",
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 65.1μs -> 58.1μs (12.1% faster)
trade['orderSide'] = "BUY"
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 32.3μs -> 28.4μs (13.5% faster)
def test_parse_trade_taker_maker_case_insensitive(xt_instance):
# takerMaker is case-insensitive
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"takerMaker": "MAKER",
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 64.1μs -> 57.7μs (11.1% faster)
trade['takerMaker'] = "TAKER"
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 32.2μs -> 28.6μs (12.6% faster)
def test_parse_trade_missing_market_id(xt_instance):
# No symbol/s field
trade = {
"tradeId": "123456789",
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 56.0μs -> 49.9μs (12.4% faster)
def test_parse_trade_fee_as_string_true_false(xt_instance):
# Fee as string 'true'/'false'
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"fee": "true",
"feeCurrency": "usdt",
"price": "100",
"quantity": "1",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 66.8μs -> 59.2μs (12.9% faster)
def test_parse_trade_zero_fee_and_amount(xt_instance):
# Zero fee and amount
trade = {
"symbol": "btc_usdt",
"tradeId": "123456789",
"fee": "0",
"feeCurrency": "usdt",
"price": "100",
"quantity": "0",
"time": 1679032290215
}
codeflash_output = xt_instance.parse_trade(trade); result = codeflash_output # 68.8μs -> 61.5μs (11.8% faster)
3. Large Scale Test Cases
def test_parse_trade_many_trades_spot(xt_instance):
# Parse 1000 spot trades
trades = []
for i in range(1000):
trades.append({
"i": str(i),
"t": 1678227505815 + i,
"p": str(10000 + i),
"q": str(0.001 * i),
"b": bool(i % 2)
})
results = [xt_instance.parse_trade(trade) for trade in trades]
def test_parse_trade_many_trades_contract(xt_instance):
# Parse 500 contract trades with contractSize=0.01
market = DummyMarket('BTC/USDT', type='contract', contractSize=0.01)
trades = []
for i in range(500):
trades.append({
"t": 1678227683897 + i,
"s": "btc_usdt",
"p": str(20000 + i),
"a": str(i),
"m": "BID" if i % 2 == 0 else "ASK"
})
results = [xt_instance.parse_trade(trade, market) for trade in trades]
def test_parse_trade_performance_large_scale(xt_instance):
# Performance: parse 1000 trades quickly
import time
trades = []
for i in range(1000):
trades.append({
"i": str(i),
"t": 1678227505815 + i,
"p": str(10000 + i),
"q": str(0.001 * i),
"b": bool(i % 2)
})
start = time.time()
results = [xt_instance.parse_trade(trade) for trade in trades]
duration = time.time() - start
codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
To edit these changes
git checkout codeflash/optimize-xt.parse_trade-mhv4xwxqand push.