Skip to content

bug: Arithmetic operators fail with LHS numeric literals - all backends #11742

@discreteds

Description

@discreteds

What happened?

The problem

Arithmetic operations fail when a numeric or string literal is on the LEFT side and a Deferred column reference (ibis._['column']) is on the RIGHT side:
It works for Boolean literals, as demonstrated below.

Illustrative example

# Just an illustrative example - not working, just easy to understand!
lit = ibis.literal(5)
col = ibis._['value']

# This works ✅
col + lit

# This fails ❌
lit + col
# InputTypeError: Unable to infer datatype of value _['value']
# with type <class 'ibis.common.deferred.Deferred'>

Working Reproduction of the bug

import ibis
import pandas as pd


# Setup backends
polars_backend = ibis.polars.connect()
duckdb_backend = ibis.duckdb.connect()
sqlite_backend = ibis.sqlite.connect()

# Create test data
pl_df = pd.DataFrame({'value': [10]})

# Create tables
df_polars = polars_backend.create_table('test', pl_df, overwrite=True)
df_duckdb = duckdb_backend.create_table('test', pl_df, overwrite=True)
df_sqlite = sqlite_backend.create_table('test', pl_df, overwrite=True)

# Define expressions using Deferred syntax
lit = ibis.literal(5)
col = ibis._['value']  # Deferred column reference

# Test each backend
results = []

for backend_name, table in [
    ('polars', df_polars),
    ('duckdb', df_duckdb),
    ('sqlite', df_sqlite),
]:
    result = {'backend': backend_name}

    # Test: col + lit (should work)
    try:
        expr = col + lit
        res = table.select(expr.name('result')).execute()
        result['col+lit'] = res.result[0]
    except Exception as e:
        result['col+lit'] = f'{type(e).__name__}'

    # Test: lit + col (fails with InputTypeError)
    try:
        expr = lit + col
        res = table.select(expr.name('result')).execute()
        result['lit+col'] = res.result[0]
    except Exception as e:
        result['lit+col'] = f'{type(e).__name__}'

    results.append(result)

# Display results
df_results = pd.DataFrame(results)
print(df_results.to_markdown(index=False))

Output

backend col+lit lit+col
polars 15 InputTypeError
duckdb 15 InputTypeError
sqlite 15 InputTypeError

Affected Scalar Types and Operations

IntegerScalar & FloatingScalar (BROKEN):

  • Addition: literal(5) + ibis._['x'] → ❌ InputTypeError
  • Subtraction: literal(100) - ibis._['x'] → ❌ InputTypeError
  • Multiplication: literal(2) * ibis._['x'] → ❌ InputTypeError
  • Division: literal(100) / ibis._['x'] → ❌ InputTypeError
  • Modulo: literal(100) % ibis._['x'] → ❌ InputTypeError
  • Power: literal(2) ** ibis._['x'] → ❌ InputTypeError
  • Floor division: literal(100) // ibis._['x'] → ❌ InputTypeError

BooleanScalar (ALREADY WORKS):

  • AND: literal(True) & ibis._['x'] → ✅ Works correctly
  • OR: literal(False) | ibis._['x'] → ✅ Works correctly

StringScalar (BROKEN - different error):

  • Concatenation: literal('x') + ibis._['y'] → ❌ SignatureValidationError (not InputTypeError)

Impact

This breaks expression building frameworks that generate expressions programmatically, where operand order may not be predictable. It also violates Python's numeric type behavior where a + b and b + a should both work.

Bug location and suggested fixes

except (ValidationError, NotImplementedError):

The error occurs for float and integer literals on the LHS of arithmetic operators.
A similar issue occurs for string literals, but requires a different fix.
Boolean literals work as they return a SignatureValidationError which is a subclass of ValidationError.

Fix for Integer and String Literals

This fix would align with the principles stated in the comments for _binop:
"returns NotImplemented if we aren't sure:"

File: ibis/expr/types/core.py
Function: _binop
Change: Add InputTypeError to the exception tuple (one line)

Current code:

except (ValidationError, NotImplementedError):
    return NotImplemented

Suggested fix:

except (ValidationError, NotImplementedError, InputTypeError):
    return NotImplemented

Why StringScalar still fails (different reason):**

StringValue uses a completely different code path that bypasses _binop:

# NumericValue.__add__ (goes through _binop)
def __add__(self, other):
    return _binop(ops.Add, self, other)  # ✅ Will be fixed

# BooleanValue.__and__ (goes through _binop)
def __and__(self, other):
    return _binop(ops.And, self, other)  # ✅ Already works

# StringValue.__add__ (BYPASSES _binop!)
def __add__(self, other):
    return self.concat(other)  # ❌ Won't be fixed by this change

def concat(self, other, *args):
    return ops.StringConcat((self, other, *args)).to_expr()  # No _binop!

Scripts

See attached scripts (written by Claude) identifying:

  • cases where the errors occur
  • code paths taken by each error
    These highlight why integer and float arithmetic ops will be fixed by this change, but string ops will require a different fix ( that I haven't yet identified! )

ibis_reverse_operator_comprehensive_repro.py
ibis_reverse_operator_code_path_analysis.py

What version of ibis are you using?

11.0.0

What backend(s) are you using, if any?

Error is consistent on Polars, SQLite and Duckdb.

Relevant log output

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions