fillna(inplace=True) does not work with columns selected by loc

[WenjieDu]

not only multiple columns, but also one column.
df.loc[df.id==123, 'num'].fillna(0, inplace=True) doesn't work ,
but df.loc[df.id==123, 'num'] = 123 works

why not edit the fillna function to adapt it in the future.
It seems like a bug.

Originally posted by @shuiyuejihua in #14858 (comment)

Hi, I met with the same problem. Now that df.loc[df.id==123, 'num'] = 123 works, which means the operation takes effect on original df, why fillna(inplace=True) doesn't work? This is really wired , and seems haven't been fixed. My pandas version is 0.25.3.

Any comment or explaination are welcome, thanks!

[MarcoGorelli]

Thanks @WenjayDu

Can confirm this is still present:

>>> import numpy as np
>>> import pandas as pd
>>> df = pd.DataFrame({'id': [123], 'num': [np.nan]})
>>> df.loc[df.id==123, 'num'].fillna(0)
0    0.0
Name: num, dtype: float64

>>> df.loc[df.id==123, 'num'].fillna(0, inplace=True)
>>> df
    id  num
0  123  NaN

however, this works:

>>> df.fillna(0, inplace=True) 
>>> df                                                                                                                                                            
    id  num
0  123  0.0

Having said that, inplace=True is now generally not recommended, better to do

>>> df.loc[df.id==123, 'num'] = df.loc[df.id==123, 'num'].fillna(0)
>>> df
    id  num
0  123  0.0

[WenjieDu]

Thanks for your reply @MarcoGorelli

Yes, df.fillna(0, inplace=True) does work, but when selection criteria becomes complex, like df.loc[df['facility_name']=='Cisco', ['feature1','feature2']], unfortunately, without inplace=True, filling will become quite verbose like

df.loc[df['facility_name']=='Cisco', ['feature1','feature2']] = \ 
df.loc[df['facility_name']=='Cisco', ['feature1','feature2']].fillna(0)

BTW, what do you mean when you say "inplace=True is now generally not recommended"? Only for fillna() function, or for other functions like reset_index() having inplace parameter as well?

[MarcoGorelli]

In general - see #16529 (or https://youtu.be/hK6o_TDXXN8)

[jreback]

this is not a bug, rather a limitation of the language itself

you are likely getting a SettingWithCioy warning that what you are doing is unsafe

https://pandas.pydata.org/docs/user_guide/indexing.html?highlight=settingwithcopy#indexing-view-versus-copy

[MarcoGorelli]

@jreback no warning is currently raised for me

[WenjieDu]

@jreback I noticed this official doc. In my opration, df.loc[df.id==123]['num']=1 will trigger SettingWithCopy warning.

But df.loc[df.id==123, 'num'].fillna(0, inplace=True) didn't throw any SettingWithCopy warning. Agree with @MarcoGorelli . Like I said, this is wierd.

[rafsid]

This problem still persists. Took help of stackoverflow for an alternative method

[MrAnimaniac]

It still persists and really is confusing.

[MicheleBellomo]

The issue still persists, and I think it is unacceptable for a library like Pandas to have such a bug.