-
-
Notifications
You must be signed in to change notification settings - Fork 19.3k
BUG: Fix DataFrame.from_dict empty row drop #62893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
BUG: Fix DataFrame.from_dict empty row drop #62893
Conversation
|
Thought the fix would be simple but I ran into several edge cases during testing, and so I tried this approach
I appreciate any feedback! |
6fce7ea to
64e0ae0
Compare
64e0ae0 to
52046f4
Compare
rhshadrach
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! Some questions.
|
|
||
| def _from_nested_dict( | ||
| data: Mapping[HashableT, Mapping[HashableT2, T]], | ||
| ) -> collections.defaultdict[HashableT2, dict[HashableT, T]]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is T -> Any necessary?
| if isinstance(s, (dict, ABCSeries)): | ||
| for col in all_cols_list: | ||
| new_data[col][index] = s.get(col, None) | ||
| elif s is None or is_scalar(s): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For what input is this case hit?
| new_data[col][index] = v | ||
| if isinstance(s, (dict, ABCSeries)): | ||
| for col in all_cols_list: | ||
| new_data[col][index] = s.get(col, None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For a Series, I believe this will be much slower than the previous implementation. Can you profile this case.
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.