DOC: Typo within `Series.mask()` docs - alignment is done between self and cond, not cond and other

[nickodell]

Pandas version checks

• I have checked that the issue still exists on the latest versions of the docs on main here

Location of the documentation

https://pandas.pydata.org/docs/reference/api/pandas.Series.mask.html#pandas.Series.mask

Similar issues exist for DataFrame.mask, Series.where, and DataFrame.where; they appear to use the same docstring with replacements.

Documentation problem

In this passage:

The mask method is an application of the if-then idiom. For each element in the calling DataFrame, if cond is False the element is used; otherwise the corresponding element from the DataFrame other is used. If the axis of other does not align with axis of cond Series/DataFrame, the misaligned index positions will be filled with True.

The bolded sentence is not correct. Here is an example where the other value is not aligned to cond, because the d value in cond has no match in other. However, cond is still not filled with True.

import pandas as pd

a = pd.Series(['apple', 'banana', 'cherry', 'dango'], index=['a', 'b', 'c', 'd'])
b = pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])

other = pd.Series(['asparagus', 'broccoli', 'carrot', 'dill'], index=['a', 'b', 'c', 'D'])
cond = b.lt(3)

print("Cond matches other?", cond.index == other.index)
print("Cond matches self?", cond.index == a.index)

a.mask(cond, other)

Output:

Cond matches other? [ True  True  True False]
Cond matches self? [ True  True  True  True]

a    asparagus
b     broccoli
c       cherry
d        dango

In this example, you can see that even though cond's d index has no corresponding aligned element in other, it still does not make a replacement for item d - not even to replace it with an NA value.

Rather, the alignment is done between self and cond, not other and cond.

Suggested fix for documentation

Proposed fix:

The mask method is an application of the if-then idiom. For each element in the calling DataFrame, if cond is False the element is used; otherwise the corresponding element from the DataFrame other is used. If the axis of self does not align with axis of cond Series/DataFrame, the misaligned index positions will be filled with True.

Here is an example which shows that this is correct. In the following code, cond and self are not aligned. The unaligned value in cond is treated as True.

import pandas as pd

a = pd.Series(['apple', 'banana', 'cherry', 'dango'], index=['a', 'b', 'c', 'd'])
b = pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'D'])

other = pd.Series(['asparagus', 'broccoli', 'carrot', 'dill'], index=['a', 'b', 'c', 'd'])
cond = b.lt(3)

print("Cond matches other?", cond.index == other.index)
print("Cond matches self?", cond.index == a.index)

a.mask(cond, other)

Output:

Cond matches other? [ True  True  True False]
Cond matches self? [ True  True  True False]

a    asparagus
b     broccoli
c       cherry
d         dill
dtype: object

[samukweku]

Thanks @nickodell I believe a PR suffices for this. Thoughts @rhshadrach @jbrockmendel @mroeschke ? If accepted, maybe the PR could be extended to case_when?

[sanggon6107]

Hi @nickodell ,

I think self tries to align with other in _where as below.

pandas/pandas/core/generic.py

Lines 9760 to 9771 in a2315af

	# try to align with other
	if isinstance(other, NDFrame):
	# align with me
	if other.ndim <= self.ndim:
	# CoW: Make sure reference is not kept alive
	other = self.align(
	other,
	join="left",
	axis=axis,
	level=level,
	fill_value=None,
	)[1]

The reason why you see "dango" from your first example is because cond is False for d, and then _where let "dango" propagate.

Interstingly, when I changed cond to b.ge(2), I can see the value is filled with NaN:

import pandas as pd
a = pd.Series(['apple', 'banana', 'cherry', 'dango'], index=['a', 'b', 'c', 'd'])
b = pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])

other = pd.Series(['asparagus', 'broccoli', 'carrot', 'dill'], index=['a', 'b', 'c', 'D'])
cond = b.gt(2) # [False,  False,  True,  True]

a.mask(cond, other)
# a     apple
# b    banana
# c    carrot
# d       NaN
# dtype: object

So I think we might be able to consider changing the sentence like :

If the axis of other does not align with axis of cond Series/DataFrame and cond is True, the misaligned index positions will be filled with NaN.

Furthermore, I think the behavior of mask in your second example could be inconsistent depending on inplace.

import pandas as pd

a = pd.Series(['apple', 'banana', 'cherry', 'dango'], index=['a', 'b', 'c', 'd'])
b = pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'D'])

other = pd.Series(['asparagus', 'broccoli', 'carrot', 'dill'], index=['a', 'b', 'c', 'd'])
cond = b.lt(3)

a.mask(cond, other)

# a    asparagus
# b     broccoli
# c       cherry
# d         dill    # filled with True
# dtype: object

a.mask(cond, other, inplace=True)
a
# a    asparagus
# b     broccoli
# c       cherry
# d        dango    # filled with False
# dtype: object

This is because cond would be filled with inplace in _where . (fillna comes before align. FYI, see #52955 (comment))

pandas/pandas/core/generic.py

Lines 9731 to 9734 in a2315af

	# make sure we are boolean
	fill_value = bool(inplace)
	cond = cond.fillna(fill_value)
	cond = cond.infer_objects()

Please note that I'm currently working on a PR regarding this. (#60772)