Read FILE* input with std::fgetc under MemorySanitizer (#5874)

With glibc and optimization, <stdio.h> turns getc_unlocked into an extern
inline function that reads the stdio buffer (fp->_IO_read_ptr) directly.
glibc, which is not instrumented, fills that buffer, so MemorySanitizer
treats the bytes as uninitialized and reports a false positive when the
parsers read from a FILE* (e.g., the fuzz harness FILE* leg, OSS-Fuzz
575915982). std::fgetc is an opaque call whose result MemorySanitizer treats
as initialized.

Select JSON_UNLOCKED_FILE_IO 0 when compiled with MemorySanitizer. This is a
regression from #5828 and not part of any release.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

docs/mkdocs/docs/api/macros/json_disable_unlocked_file_io.md

@@ -10,6 +10,9 @@ earlier, instead of locking the file once and reading it with the unlocked stdio
 It does not change any result: values, exceptions, and the position of the file after parsing are the same either way.
 Only the speed and the locking of the file differ (see the notes on locking below).
 
+Builds with MemorySanitizer use `std::fgetc` automatically, because the inline `getc_unlocked` of glibc reads the stdio
+buffer directly, which glibc (not instrumented) filled, and MemorySanitizer would report it as uninitialized.
+
 ## Default definition
 
 The default value is `0`: the file is locked once and read with the unlocked stdio functions where they are available.

include/nlohmann/detail/input/input_adapters.hpp

@@ -38,8 +38,16 @@
     // 2 = Microsoft CRT _lock_file/_fgetc_nolock, 0 = none (std::fgetc, which
     // locks the file for every character). Only C libraries known to declare
     // the functions are listed; the macros are tested after <cstdio>, which
-    // defines __GLIBC__.
+    // defines __GLIBC__. MemorySanitizer builds always use std::fgetc: glibc's
-    #if JSON_DISABLE_UNLOCKED_FILE_IO
+    // inline getc_unlocked reads the stdio buffer directly, which glibc (not
+    // instrumented) filled, so MemorySanitizer would report a false positive.
+    #if defined(__has_feature)
+        #if __has_feature(memory_sanitizer)
+            #define JSON_UNLOCKED_FILE_IO_MSAN 1
+        #endif
+    #endif
+
+    #if JSON_DISABLE_UNLOCKED_FILE_IO || defined(JSON_UNLOCKED_FILE_IO_MSAN)
         #define JSON_UNLOCKED_FILE_IO 0
     #elif defined(_MSC_VER)
         #define JSON_UNLOCKED_FILE_IO 2
@@ -48,6 +56,7 @@
     #else
         #define JSON_UNLOCKED_FILE_IO 0
     #endif
+    #undef JSON_UNLOCKED_FILE_IO_MSAN
 #endif  // JSON_NO_IO
 
 NLOHMANN_JSON_NAMESPACE_BEGIN

single_include/nlohmann/json.hpp

@@ -8441,8 +8441,16 @@ NLOHMANN_JSON_NAMESPACE_END
     // 2 = Microsoft CRT _lock_file/_fgetc_nolock, 0 = none (std::fgetc, which
     // locks the file for every character). Only C libraries known to declare
     // the functions are listed; the macros are tested after <cstdio>, which
-    // defines __GLIBC__.
+    // defines __GLIBC__. MemorySanitizer builds always use std::fgetc: glibc's
-    #if JSON_DISABLE_UNLOCKED_FILE_IO
+    // inline getc_unlocked reads the stdio buffer directly, which glibc (not
+    // instrumented) filled, so MemorySanitizer would report a false positive.
+    #if defined(__has_feature)
+        #if __has_feature(memory_sanitizer)
+            #define JSON_UNLOCKED_FILE_IO_MSAN 1
+        #endif
+    #endif
+
+    #if JSON_DISABLE_UNLOCKED_FILE_IO || defined(JSON_UNLOCKED_FILE_IO_MSAN)
         #define JSON_UNLOCKED_FILE_IO 0
     #elif defined(_MSC_VER)
         #define JSON_UNLOCKED_FILE_IO 2
@@ -8451,6 +8459,7 @@ NLOHMANN_JSON_NAMESPACE_END
     #else
         #define JSON_UNLOCKED_FILE_IO 0
     #endif
+    #undef JSON_UNLOCKED_FILE_IO_MSAN
 #endif  // JSON_NO_IO
 
 NLOHMANN_JSON_NAMESPACE_BEGIN