-
Notifications
You must be signed in to change notification settings - Fork 15.4k
[libc++] Support using Newlib as libc with locale support. #167962
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+247
−0
Merged
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
0cebb7a
[libc++] Support using Newlib as libc with locale support.
wecing 0c9ee29
clang-format fix
wecing 8acbd88
Merge branch 'llvm:main' into newlib_locale_review
wecing c7b9492
Use cmake variables defined in commit a6643f2.
wecing 2bc8ee2
Reduce divergence between newlib.h and linux.h
wecing 8d0a102
Update libcxx/include/__locale_dir/support/newlib.h
ldionne File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,269 @@ | ||
| //===-----------------------------------------------------------------------===// | ||
| // | ||
| // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| // See https://llvm.org/LICENSE.txt for license information. | ||
| // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| #ifndef _LIBCPP___LOCALE_DIR_SUPPORT_NEWLIB_H | ||
| #define _LIBCPP___LOCALE_DIR_SUPPORT_NEWLIB_H | ||
|
|
||
| #include <__config> | ||
| #include <clocale> | ||
| #include <cstdio> | ||
| #include <cstdlib> | ||
| #include <ctype.h> | ||
| #include <stdarg.h> | ||
| #include <string.h> | ||
| #include <time.h> | ||
| #if _LIBCPP_HAS_WIDE_CHARACTERS | ||
| # include <cwchar> | ||
| # include <wctype.h> | ||
| #endif | ||
|
|
||
| #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) | ||
| # pragma GCC system_header | ||
| #endif | ||
|
|
||
| _LIBCPP_BEGIN_NAMESPACE_STD | ||
| namespace __locale { | ||
|
|
||
| struct __locale_guard { | ||
| _LIBCPP_HIDE_FROM_ABI __locale_guard(locale_t& __loc) : __old_loc_(::uselocale(__loc)) {} | ||
|
|
||
| _LIBCPP_HIDE_FROM_ABI ~__locale_guard() { | ||
| if (__old_loc_) | ||
| ::uselocale(__old_loc_); | ||
| } | ||
|
|
||
| locale_t __old_loc_; | ||
|
|
||
| __locale_guard(__locale_guard const&) = delete; | ||
| __locale_guard& operator=(__locale_guard const&) = delete; | ||
| }; | ||
|
|
||
| // | ||
| // Locale management | ||
| // | ||
|
|
||
| using __locale_t _LIBCPP_NODEBUG = locale_t; | ||
| using __lconv_t _LIBCPP_NODEBUG = lconv; | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI __locale_t __newlocale(int __category_mask, const char* __locale, __locale_t __base) { | ||
| return ::newlocale(__category_mask, __locale, __base); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI void __freelocale(__locale_t __loc) { ::freelocale(__loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI char* __setlocale(int __category, char const* __locale) { | ||
| return ::setlocale(__category, __locale); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI __lconv_t* __localeconv(__locale_t& __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::localeconv(); | ||
| } | ||
|
|
||
| #define _LIBCPP_COLLATE_MASK LC_COLLATE_MASK | ||
| #define _LIBCPP_CTYPE_MASK LC_CTYPE_MASK | ||
| #define _LIBCPP_MONETARY_MASK LC_MONETARY_MASK | ||
| #define _LIBCPP_NUMERIC_MASK LC_NUMERIC_MASK | ||
| #define _LIBCPP_TIME_MASK LC_TIME_MASK | ||
| #define _LIBCPP_MESSAGES_MASK LC_MESSAGES_MASK | ||
| #define _LIBCPP_ALL_MASK LC_ALL_MASK | ||
| #define _LIBCPP_LC_ALL LC_ALL | ||
|
|
||
| // | ||
| // Strtonum functions | ||
| // | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI float __strtof(const char* __nptr, char** __endptr, __locale_t __loc) { | ||
| return ::strtof_l(__nptr, __endptr, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI double __strtod(const char* __nptr, char** __endptr, __locale_t __loc) { | ||
| return ::strtod_l(__nptr, __endptr, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI long double __strtold(const char* __nptr, char** __endptr, __locale_t __loc) { | ||
| return ::strtold_l(__nptr, __endptr, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI long long __strtoll(const char* __nptr, char** __endptr, int __base, __locale_t __loc) { | ||
| return ::strtoll_l(__nptr, __endptr, __base, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI unsigned long long | ||
| __strtoull(const char* __nptr, char** __endptr, int __base, __locale_t __loc) { | ||
| return ::strtoull_l(__nptr, __endptr, __base, __loc); | ||
| } | ||
|
|
||
| // | ||
| // Character manipulation functions | ||
| // | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __isdigit(int __c, __locale_t __loc) { return isdigit_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __isxdigit(int __c, __locale_t __loc) { return isxdigit_l(__c, __loc); } | ||
|
|
||
| #if defined(_LIBCPP_BUILDING_LIBRARY) | ||
| inline _LIBCPP_HIDE_FROM_ABI int __toupper(int __c, __locale_t __loc) { return toupper_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __tolower(int __c, __locale_t __loc) { return tolower_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __strcoll(const char* __s1, const char* __s2, __locale_t __loc) { | ||
| return strcoll_l(__s1, __s2, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t __strxfrm(char* __dest, const char* __src, size_t __n, __locale_t __loc) { | ||
| return strxfrm_l(__dest, __src, __n, __loc); | ||
| } | ||
|
|
||
| # if _LIBCPP_HAS_WIDE_CHARACTERS | ||
| inline _LIBCPP_HIDE_FROM_ABI int __iswctype(wint_t __c, wctype_t __type, __locale_t __loc) { | ||
| return iswctype_l(__c, __type, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswspace(wint_t __c, __locale_t __loc) { return iswspace_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswprint(wint_t __c, __locale_t __loc) { return iswprint_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswcntrl(wint_t __c, __locale_t __loc) { return iswcntrl_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswupper(wint_t __c, __locale_t __loc) { return iswupper_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswlower(wint_t __c, __locale_t __loc) { return iswlower_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswalpha(wint_t __c, __locale_t __loc) { return iswalpha_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswblank(wint_t __c, __locale_t __loc) { return iswblank_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswdigit(wint_t __c, __locale_t __loc) { return iswdigit_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswpunct(wint_t __c, __locale_t __loc) { return iswpunct_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __iswxdigit(wint_t __c, __locale_t __loc) { return iswxdigit_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI wint_t __towupper(wint_t __c, __locale_t __loc) { return towupper_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI wint_t __towlower(wint_t __c, __locale_t __loc) { return towlower_l(__c, __loc); } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __wcscoll(const wchar_t* __ws1, const wchar_t* __ws2, __locale_t __loc) { | ||
| return wcscoll_l(__ws1, __ws2, __loc); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t __wcsxfrm(wchar_t* __dest, const wchar_t* __src, size_t __n, __locale_t __loc) { | ||
| return wcsxfrm_l(__dest, __src, __n, __loc); | ||
| } | ||
| # endif // _LIBCPP_HAS_WIDE_CHARACTERS | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI | ||
| size_t __strftime(char* __s, size_t __max, const char* __format, const struct tm* __tm, __locale_t __loc) { | ||
| return strftime_l(__s, __max, __format, __tm, __loc); | ||
| } | ||
|
|
||
| // | ||
| // Other functions | ||
| // | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI decltype(MB_CUR_MAX) __mb_len_max(__locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return MB_CUR_MAX; | ||
| } | ||
|
|
||
| # if _LIBCPP_HAS_WIDE_CHARACTERS | ||
| inline _LIBCPP_HIDE_FROM_ABI wint_t __btowc(int __c, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::btowc(__c); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __wctob(wint_t __c, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::wctob(__c); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t | ||
| __wcsnrtombs(char* __dest, const wchar_t** __src, size_t __nwc, size_t __len, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return ::wcsnrtombs(__dest, __src, __nwc, __len, __ps); // non-standard | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t __wcrtomb(char* __s, wchar_t __wc, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::wcrtomb(__s, __wc, __ps); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t | ||
| __mbsnrtowcs(wchar_t* __dest, const char** __src, size_t __nms, size_t __len, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return ::mbsnrtowcs(__dest, __src, __nms, __len, __ps); // non-standard | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t | ||
| __mbrtowc(wchar_t* __pwc, const char* __s, size_t __n, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::mbrtowc(__pwc, __s, __n, __ps); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI int __mbtowc(wchar_t* __pwc, const char* __pmb, size_t __max, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::mbtowc(__pwc, __pmb, __max); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t __mbrlen(const char* __s, size_t __n, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::mbrlen(__s, __n, __ps); | ||
| } | ||
|
|
||
| inline _LIBCPP_HIDE_FROM_ABI size_t | ||
| __mbsrtowcs(wchar_t* __dest, const char** __src, size_t __len, mbstate_t* __ps, __locale_t __loc) { | ||
| __locale_guard __current(__loc); | ||
| return std::mbsrtowcs(__dest, __src, __len, __ps); | ||
| } | ||
| # endif // _LIBCPP_HAS_WIDE_CHARACTERS | ||
| #endif // _LIBCPP_BUILDING_LIBRARY | ||
|
|
||
| #ifndef _LIBCPP_COMPILER_GCC // GCC complains that this can't be always_inline due to C-style varargs | ||
| _LIBCPP_HIDE_FROM_ABI | ||
| #endif | ||
| inline _LIBCPP_ATTRIBUTE_FORMAT(__printf__, 4, 5) int __snprintf( | ||
| char* __s, size_t __n, __locale_t __loc, const char* __format, ...) { | ||
| va_list __va; | ||
| va_start(__va, __format); | ||
| __locale_guard __current(__loc); | ||
| int __res = std::vsnprintf(__s, __n, __format, __va); | ||
| va_end(__va); | ||
| return __res; | ||
| } | ||
|
|
||
| #ifndef _LIBCPP_COMPILER_GCC // GCC complains that this can't be always_inline due to C-style varargs | ||
| _LIBCPP_HIDE_FROM_ABI | ||
| #endif | ||
| inline _LIBCPP_ATTRIBUTE_FORMAT(__printf__, 3, 4) int __asprintf( | ||
| char** __s, __locale_t __loc, const char* __format, ...) { | ||
| va_list __va; | ||
| va_start(__va, __format); | ||
| __locale_guard __current(__loc); | ||
| int __res = ::vasprintf(__s, __format, __va); // non-standard | ||
| va_end(__va); | ||
| return __res; | ||
| } | ||
|
|
||
| #ifndef _LIBCPP_COMPILER_GCC // GCC complains that this can't be always_inline due to C-style varargs | ||
| _LIBCPP_HIDE_FROM_ABI | ||
| #endif | ||
| inline _LIBCPP_ATTRIBUTE_FORMAT(__scanf__, 3, 4) int __sscanf( | ||
| const char* __s, __locale_t __loc, const char* __format, ...) { | ||
| va_list __va; | ||
| va_start(__va, __format); | ||
| __locale_guard __current(__loc); | ||
| int __res = std::vsscanf(__s, __format, __va); | ||
| va_end(__va); | ||
| return __res; | ||
| } | ||
|
|
||
| } // namespace __locale | ||
| _LIBCPP_END_NAMESPACE_STD | ||
|
|
||
| #endif // _LIBCPP___LOCALE_DIR_SUPPORT_NEWLIB_H |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is Newlib ever used on a platform where
__linux__is defined? It seems to me like we might be checking the wrong thing by checkingelif defined(__linux__)above, perhaps what we want iselif defined(_GLIBC)?Also CC @reactive-firewall
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wecing
__linux__flag just declares the presence of the linux headers, which may or may not be relevant to the libc (which by standard (e.g. ISO 9989: ) has no requirement on the linux headers, e.g., musl and BSD libc can compile without the linux headers) so baremetal builds should not expect it to describe libc.EDIT: @ldionne
elif defined(__linux__)above, perhaps what we want iselif defined(_GLIBC)?__linux__being defined when bootstraping a stage0 toolset, (possibly even for an eventual linux system). AFAIK, newlib does seem to provide thestrto*_lfunctions unconditionally (albeit not sure where in their codebase 🤷)_GNU_SOURCEor_BSD_SOURCE_GLIBCwould be more intuitive and granular for identifying the particular standard c library (e.g. glibc, stdlibc, libc, libsystem_c etc.)In summation:
elif defined(_GLIBC)seems more granular and maintainable regarding near-baremetal (e.g. just minimal libc) use-cases.🙇 Hopefully this helps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the review and insights, @ldionne and @reactive-firewall !
For the record, they are defined in the non-
_l.cfiles, e.g. newlib/libc/stdlib/strtol.c forstrtol_l.Per my discussion with @philnik777 above,
linux.handnewlib.hare essentially the same. So it sounds like we need something like this:And rename
linux.htominimal_libc.h?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's going to sound weird, but for the moment I think I'd rather keep the changes minimal even if it incurs some code duplication. So for this patch, I prefer the current approach with
The reason is that I already want to formalize the way we define the C library we use, so there's some refactoring I'll need to do in that area. I also want to further refactor some of the localization base API stuff to better support LLVM-libc. So for the time being, please just follow the status quo closely and I'll think about how to refactor this in the future to reduce duplication a bit.