clang-19: error: implicit instantiation of undefined template 'std::char_traits<unsigned short>'

[nunotexbsd]

Describe the bug

At FreeBSD main with base clang 19.1.2 build error:

usr/include/c++/v1/string:2861:5: error: implicit instantiation of undefined template 'std::char_traits<unsigned short>'
 2861 |     traits_type::assign(std::__to_address(__p) + __sz, __n, __c);
      |     ^
/usr/include/c++/v1/string:3302:5: note: in instantiation of member function 'std::basic_string<unsigned short>::append' requested here
 3302 |     append(__n - __sz, __c);
      |     ^
/usr/include/c++/v1/string:1303:84: note: in instantiation of member function 'std::basic_string<unsigned short>::resize' requested here
 1303 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX20 void resize(size_type __n) { resize(__n, value_type()); }

Full log:
dosbox-x-2024.10.01.log

For what I investigate, a similar issue is found at rnpgp/sexpp#51 and fixed in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=282347

Any hints?

Steps to reproduce the behaviour

FreeBSD ports framework:
cd /usr/ports/emulators/dosbox-x/ && make install clean

https://www.freshports.org/emulators/dosbox-x/

Expected behavior

No response

What operating system(s) this bug have occurred on?

FreeBSD 15 CURRENT amd64 1500026

What version(s) of DOSBox-X have this bug?

2024.10.01

Used configuration

No response

Output log

No response

Additional information

No response

Have you checked that no similar bug report(s) exist?

• I have searched and didn't find any similar bug report.

Code of Conduct & Contributing Guidelines

• I agree to follow the code of conduct and the contributing guidelines.

[maron2000]

It builds fine with clang-18 on ArchLinux.

According to the log, it seems that we need to patch src/dos/dos_programs.cpp as below in order to build on clang-19 but I don't know if there are any regressions.
If there is more errors, you may add patches with the same way as below.

IMO, if a compiler changes warnings to errors, there should be a switch to revert to old setting in order to maintain compatibility with old codes.

@@ -79,11 +79,11 @@ host_cnv_char_t *CodePageGuestToHost(const char *s);
 #else
 #include <libgen.h>
 #endif
 #ifdef C_ICONV
 #include "iconvpp.hpp"
-typedef uint16_t test_char_t;
+typedef int16_t test_char_t;
 typedef std::basic_string<test_char_t> test_string;
 typedef std::basic_string<char> test_char;
 #endif
 int freesizecap = 1;
 int result_errorcode = 0;
@@ -7811,11 +7811,11 @@ void UTF16::Run()
         if (!m || ch==0x1A || ch==0xA || (c>1 && text[c-2] == 0xD)) {
             if (ch!=0xA && c>1 && text[c-2] == 0xD) {text[c-1] = 0xA;}
             wch=new test_char_t[c+1];
             for (unsigned int i=0; i<c; i++) wch[i]=(test_char_t)text[i];
             wch[c]=0;
-            if (CodePageHostToGuestUTF16(temp,wch)) {
+            if (CodePageHostToGuestUTF16(temp,(uint16_t *)wch)) {
                 WriteOut_NoParsing(temp, true);
             } else {
                 x->set_src(wch);
                 if ((customcp && dos.loaded_codepage==customcp) || (altcp && dos.loaded_codepage==altcp) || x->string_convert_dest(dst) < 0 || (c && !dst.size())) {
                     WriteOut("An error occurred during text conversion.\n");

[maron2000]

A simple test shows that Japanese UTF16LE text converts to CP932 well even after the above-mentioned patch. (Tested on VS x64 build on Windows 10)

[DimitryAndric]

The issue is that FreeBSD uses libc++ as its C++ library, and libc++ has recently removed support for instantiating std::char_traits<> with any other types than char, wchar_t, char8_t, char16_t, char32_t. These instantiations are non-standard, and have been deprecated for 2 libc++ releases now. See also https://releases.llvm.org/19.1.0/projects/libcxx/docs/ReleaseNotes.html#deprecations-and-removals.

In any case, most of the time this kind of thing can be fixed by replacing uint16_t with char16_t, which is now the intended type for UTF-16. You typically only need a few casts here and there to make it work.

I have a patch set for the FreeBSD port, which I will submit in the FreeBSD bug tracker. If you like, I can also submit a pull request here.

[nunotexbsd]

@DimitryAndric thank you so much for PR with fix ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=282386 ).

Please submit a pull request here.

Cheers

[nunotexbsd]

2 patches fixing this PR by @DimitryAndric applied to FreeBSD ports:
https://cgit.freebsd.org/ports/commit/?id=573cdb2e7e34d6e97828e0b42456b17fc7c13f4b

--- src/dos/dos_programs.cpp.orig       2024-10-02 07:16:36.000000000 +0100
+++ src/dos/dos_programs.cpp    2024-10-30 08:44:44.828400000 +0000
@@ -81,7 +81,7 @@
 #endif
 #ifdef C_ICONV
 #include "iconvpp.hpp"
-typedef uint16_t test_char_t;
+typedef char16_t test_char_t;
 typedef std::basic_string<test_char_t> test_string;
 typedef std::basic_string<char> test_char;
 #endif
@@ -102,6 +102,9 @@
 bool qmount = false;
 bool nowarn = false;
 bool CodePageHostToGuestUTF8(char *d/*CROSS_LEN*/,const char *s/*CROSS_LEN*/), CodePageHostToGuestUTF16(char *d/*CROSS_LEN*/,const uint16_t *s/*CROSS_LEN*/);
+inline bool CodePageHostToGuestUTF16(char *d/*CROSS_LEN*/,const char16_t *s/*CROSS_LEN*/) {
+    return CodePageHostToGuestUTF16(d, reinterpret_cast<const uint16_t *>(s));
+}
 extern bool systemmessagebox(char const * aTitle, char const * aMessage, char const * aDialogType, char const * aIconType, int aDefaultButton);
 extern bool addovl, addipx, addne2k, prepared, inshell, usecon, uao, loadlang, morelen, mountfro[26], mountiro[26], resetcolor, staycolors, printfont, notrycp, internal_program;
 extern bool clear_screen(), OpenGL_using(void), DOS_SetAnsiAttr(uint8_t attr), isDBCSCP();

and

--- src/dos/drive_iso.cpp.orig  2024-10-02 07:16:36.000000000 +0100
+++ src/dos/drive_iso.cpp       2024-10-30 08:44:52.915362000 +0000
@@ -40,6 +40,10 @@
 extern bool gbk, isDBCSCP(), isKanji1_gbk(uint8_t chr), shiftjis_lead_byte(int c);
 extern bool filename_not_8x3(const char *n), filename_not_strict_8x3(const char *n);
 extern bool CodePageHostToGuestUTF16(char *d/*CROSS_LEN*/,const uint16_t *s/*CROSS_LEN*/);
+inline bool CodePageHostToGuestUTF16(uint8_t *d/*CROSS_LEN*/,const uint8_t *s/*CROSS_LEN*/) {
+    std::u16string u16s(reinterpret_cast<const char16_t *>(s));
+    return CodePageHostToGuestUTF16(reinterpret_cast<char *>(d), reinterpret_cast<const uint16_t *>(u16s.c_str()));
+}

 using namespace std;

@@ -1762,7 +1766,7 @@
                                // The string is big Endian UCS-16, convert to host Endian UCS-16
                                for (size_t i=0;((const uint16_t*)de->ident)[i] != 0;i++) ((uint16_t*)de->ident)[i] = be16toh(((uint16_t*)de->ident)[i]);
                                // finally, convert from UCS-16 to local code page, using C++ string construction to make a copy first
-                               CodePageHostToGuestUTF16((char*)de->ident,std::basic_string<uint16_t>((const uint16_t*)de->ident).c_str());
+                               CodePageHostToGuestUTF16(de->ident, de->ident);
                        }
                }
        } else {
@@ -1784,7 +1788,7 @@
                        // The string is big Endian UCS-16, convert to host Endian UCS-16
                        for (size_t i=0;((const uint16_t*)de->ident)[i] != 0;i++) ((uint16_t*)de->ident)[i] = be16toh(((uint16_t*)de->ident)[i]);
                        // finally, convert from UCS-16 to local code page, using C++ string construction to make a copy first
-                       CodePageHostToGuestUTF16((char*)de->ident,std::basic_string<uint16_t>((const uint16_t*)de->ident).c_str());
+                       CodePageHostToGuestUTF16(de->ident, de->ident);
                }
                else {
                        // remove any file version identifiers as there are some cdroms that don't have them

Could a pull request be made?

Thanks

[maron2000]

On behalf of @DimitryAndric I put PR #5264.

[DimitryAndric]

Thanks!