Niels Lohmann
f650d5c6ee
Rerun on develop 7fc8312c64 with the open pull requests #6029, #6031,
#6033 and #6039 merged, measured on November 7: the suite (median of 12
runs, now 3.3x), the timeline (358 commits, #6031 shown as a pending
step on the candidate, #6028 labeled, a tick for November), the compile
times, the stream gap, the allocator check and the comparison on the
Mac and on x86-64, including the held-out files.
The comparison now leaves out the "copy untimed" variants, as the post
describes, and the held-out chart no longer counts json::accept() as
another library. Appendix D, the Intel column of Appendix B and the
GCC 16/Clang 23 check on the Mac are still from November 3.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
9.7 KiB
9.7 KiB
| 1 | benchmark | unit | 3.12.0 | 3.13.0 | speedup |
|---|---|---|---|---|---|
| 2 | Accept/canada | ns | 1.27789e+07 | 1.34304e+06 | 9.515 |
| 3 | Accept/citm_catalog | ns | 6.47666e+06 | 474064 | 13.662 |
| 4 | Accept/floats | ns | 1.14227e+08 | 1.72173e+07 | 6.634 |
| 5 | Accept/jeopardy | ns | 2.18081e+08 | 2.79002e+07 | 7.816 |
| 6 | Accept/signed_ints | ns | 1.15981e+08 | 2.33778e+07 | 4.961 |
| 7 | Accept/small_signed_ints | ns | 5.64996e+07 | 1.31818e+07 | 4.286 |
| 8 | Accept/twitter | ns | 2.48725e+06 | 234277 | 10.617 |
| 9 | Accept/unsigned_ints | ns | 1.14729e+08 | 2.28147e+07 | 5.029 |
| 10 | BinaryToCbor/1024 | ns | 315.368 | 108.041 | 2.919 |
| 11 | BinaryToCbor/128 | ns | 266.074 | 114.105 | 2.332 |
| 12 | BinaryToCbor/16 | ns | 246.212 | 49.912 | 4.933 |
| 13 | BinaryToCbor/16384 | ns | 950.054 | 418.045 | 2.273 |
| 14 | BinaryToCbor/2048 | ns | 340.213 | 126.796 | 2.683 |
| 15 | BinaryToCbor/256 | ns | 279.79 | 91.7512 | 3.049 |
| 16 | BinaryToCbor/32 | ns | 262.471 | 50.3433 | 5.214 |
| 17 | BinaryToCbor/32768 | ns | 1578.4 | 837.8 | 1.884 |
| 18 | BinaryToCbor/4096 | ns | 431.963 | 174.276 | 2.479 |
| 19 | BinaryToCbor/512 | ns | 296.768 | 102.073 | 2.907 |
| 20 | BinaryToCbor/64 | ns | 259.159 | 84.2641 | 3.076 |
| 21 | BinaryToCbor/8 | ns | 250.574 | 49.9122 | 5.020 |
| 22 | BinaryToCbor/8192 | ns | 527.418 | 253.616 | 2.080 |
| 23 | Copy/canada | ns | 3.68841e+06 | 2.9017e+06 | 1.271 |
| 24 | Copy/citm_catalog | ns | 1.64706e+06 | 1.48507e+06 | 1.109 |
| 25 | Copy/twitter | ns | 810558 | 728698 | 1.112 |
| 26 | CopyAssign/canada | ns | 3.73912e+06 | 2.94393e+06 | 1.270 |
| 27 | CopyAssign/citm_catalog | ns | 1.6545e+06 | 1.53713e+06 | 1.076 |
| 28 | CopyAssign/twitter | ns | 817997 | 752415 | 1.087 |
| 29 | CopyOnly/canada | ns | 1.53602e+06 | 1.52075e+06 | 1.010 |
| 30 | CopyOnly/citm_catalog | ns | 754817 | 794741 | 0.950 |
| 31 | CopyOnly/twitter | ns | 406966 | 410330 | 0.992 |
| 32 | CopyRing/canada | ns | 3.74734e+06 | 2.92592e+06 | 1.281 |
| 33 | CopyRing/citm_catalog | ns | 1.7477e+06 | 1.54757e+06 | 1.129 |
| 34 | CopyRing/twitter | ns | 876191 | 784261 | 1.117 |
| 35 | DestroyOnly/canada | ns | 2.16216e+06 | 1.38217e+06 | 1.564 |
| 36 | DestroyOnly/citm_catalog | ns | 895535 | 681877 | 1.313 |
| 37 | DestroyOnly/twitter | ns | 406450 | 326620 | 1.244 |
| 38 | Diff/canada | ns | 3.91782e+07 | 3.75209e+06 | 10.442 |
| 39 | Diff/citm_catalog | ns | 6.91552e+06 | 1.09211e+06 | 6.332 |
| 40 | Diff/twitter | ns | 4.92965e+06 | 639734 | 7.706 |
| 41 | Dump/canada / - | ns | 6.70617e+06 | 2.31399e+06 | 2.898 |
| 42 | Dump/canada / 4 | ns | 8.81664e+06 | 3.03083e+06 | 2.909 |
| 43 | Dump/citm_catalog / - | ns | 1.39132e+06 | 295525 | 4.708 |
| 44 | Dump/citm_catalog / 4 | ns | 1.94729e+06 | 421536 | 4.620 |
| 45 | Dump/floats / - | ns | 6.35321e+07 | 2.37757e+07 | 2.672 |
| 46 | Dump/floats / 4 | ns | 7.19879e+07 | 2.4663e+07 | 2.919 |
| 47 | Dump/jeopardy / - | ns | 2.0176e+08 | 3.0624e+07 | 6.588 |
| 48 | Dump/jeopardy / 4 | ns | 2.20493e+08 | 3.43504e+07 | 6.419 |
| 49 | Dump/signed_ints / - | ns | 2.36853e+07 | 1.07405e+07 | 2.205 |
| 50 | Dump/signed_ints / 4 | ns | 3.2029e+07 | 1.13731e+07 | 2.816 |
| 51 | Dump/small_signed_ints / - | ns | 1.78623e+07 | 7.93339e+06 | 2.252 |
| 52 | Dump/small_signed_ints / 4 | ns | 2.50508e+07 | 8.80939e+06 | 2.844 |
| 53 | Dump/twitter / - | ns | 1.40813e+06 | 213513 | 6.595 |
| 54 | Dump/twitter / 4 | ns | 1.59095e+06 | 249353 | 6.380 |
| 55 | Dump/unsigned_ints / - | ns | 2.35653e+07 | 8.07378e+06 | 2.919 |
| 56 | Dump/unsigned_ints / 4 | ns | 3.17269e+07 | 8.50708e+06 | 3.729 |
| 57 | Equal/canada | ns | 463162 | 336269 | 1.377 |
| 58 | Equal/citm_catalog | ns | 264124 | 260967 | 1.012 |
| 59 | Equal/twitter | ns | 138843 | 128906 | 1.077 |
| 60 | Flatten/canada | ns | 2.86941e+07 | 1.1214e+07 | 2.559 |
| 61 | Flatten/citm_catalog | ns | 6.4406e+06 | 3.00538e+06 | 2.143 |
| 62 | Flatten/twitter | ns | 3.53051e+06 | 2.05449e+06 | 1.718 |
| 63 | FlattenOrdered/citm_catalog | ns | 2.56371e+08 | 1.40896e+06 | 181.958 |
| 64 | FlattenOrdered/twitter | ns | 5.00313e+07 | 863594 | 57.934 |
| 65 | FromBinaryBuffer/bjdata / canada | ns | 8.15141e+06 | 4.17154e+06 | 1.954 |
| 66 | FromBinaryBuffer/bjdata / twitter | ns | 3.18297e+06 | 893076 | 3.564 |
| 67 | FromBinaryBuffer/bjdata_optimized / canada | ns | 8.73882e+06 | 2.42167e+06 | 3.609 |
| 68 | FromBinaryBuffer/bjdata_optimized / twitter | ns | 3.15905e+06 | 884856 | 3.570 |
| 69 | FromBinaryBuffer/bson / canada | ns | 9.33283e+06 | 4.27358e+06 | 2.184 |
| 70 | FromBinaryBuffer/bson / citm_catalog | ns | 3.91854e+06 | 1.61073e+06 | 2.433 |
| 71 | FromBinaryBuffer/bson / twitter | ns | 3.176e+06 | 910290 | 3.489 |
| 72 | FromBinaryBuffer/cbor / canada | ns | 8.39002e+06 | 2.08256e+06 | 4.029 |
| 73 | FromBinaryBuffer/cbor / citm_catalog | ns | 3.71495e+06 | 1.31999e+06 | 2.814 |
| 74 | FromBinaryBuffer/cbor / floats | ns | 2.91606e+07 | 5.48446e+06 | 5.317 |
| 75 | FromBinaryBuffer/cbor / jeopardy | ns | 3.52964e+08 | 1.00739e+08 | 3.504 |
| 76 | FromBinaryBuffer/cbor / signed_ints | ns | 3.59874e+07 | 9.77712e+06 | 3.681 |
| 77 | FromBinaryBuffer/cbor / twitter | ns | 3.08997e+06 | 878827 | 3.516 |
| 78 | FromBinaryBuffer/msgpack / canada | ns | 8.25869e+06 | 2.09779e+06 | 3.937 |
| 79 | FromBinaryBuffer/msgpack / citm_catalog | ns | 3.71825e+06 | 1.28675e+06 | 2.890 |
| 80 | FromBinaryBuffer/msgpack / jeopardy | ns | 3.56624e+08 | 9.69017e+07 | 3.680 |
| 81 | FromBinaryBuffer/msgpack / twitter | ns | 3.06392e+06 | 850067 | 3.604 |
| 82 | FromBinaryBuffer/ubjson / canada | ns | 8.12649e+06 | 4.14411e+06 | 1.961 |
| 83 | FromBinaryBuffer/ubjson / citm_catalog | ns | 3.82184e+06 | 1.65659e+06 | 2.307 |
| 84 | FromBinaryBuffer/ubjson / jeopardy | ns | 3.65507e+08 | 1.01303e+08 | 3.608 |
| 85 | FromBinaryBuffer/ubjson / twitter | ns | 3.15221e+06 | 895746 | 3.519 |
| 86 | FromBinaryBuffer/ubjson_optimized / canada | ns | 8.5126e+06 | 2.38858e+06 | 3.564 |
| 87 | FromBinaryBuffer/ubjson_optimized / twitter | ns | 3.12199e+06 | 882166 | 3.539 |
| 88 | FromBinaryFile/bjdata / twitter | ns | 1.04222e+07 | 1.45214e+06 | 7.177 |
| 89 | FromBinaryFile/bson / twitter | ns | 1.02874e+07 | 1.23514e+06 | 8.329 |
| 90 | FromBinaryFile/cbor / canada | ns | 1.43434e+07 | 2.4071e+06 | 5.959 |
| 91 | FromBinaryFile/cbor / twitter | ns | 9.92359e+06 | 1.075e+06 | 9.231 |
| 92 | FromBinaryFile/ubjson / canada | ns | 1.35516e+07 | 6.2336e+06 | 2.174 |
| 93 | FromBinaryFile/ubjson / twitter | ns | 1.05159e+07 | 1.46077e+06 | 7.199 |
| 94 | FromBinaryShape/containers / bjdata | ns | 1.43175e+07 | 7.24204e+06 | 1.977 |
| 95 | FromBinaryShape/containers / bson | ns | 1.74215e+07 | 7.97531e+06 | 2.184 |
| 96 | FromBinaryShape/containers / cbor | ns | 1.3898e+07 | 3.93021e+06 | 3.536 |
| 97 | FromBinaryShape/containers / msgpack | ns | 1.378e+07 | 3.88449e+06 | 3.547 |
| 98 | FromBinaryShape/containers / ubjson | ns | 1.44223e+07 | 7.2737e+06 | 1.983 |
| 99 | FromBinaryShape/containers / ubjson_optimized | ns | 1.49736e+07 | 4.48674e+06 | 3.337 |
| 100 | FromBinaryShape/nested / bjdata | ns | 96363.9 | 38197.9 | 2.523 |
| 101 | FromBinaryShape/nested / bson | ns | 102515 | 42496.1 | 2.412 |
| 102 | FromBinaryShape/nested / cbor | ns | 93842.1 | 35385.5 | 2.652 |
| 103 | FromBinaryShape/nested / msgpack | ns | 92033 | 34888.5 | 2.638 |
| 104 | FromBinaryShape/nested / ubjson | ns | 96455.9 | 37922.6 | 2.543 |
| 105 | FromBinaryShape/scalars / bjdata | ns | 2.48048e+07 | 1.1476e+07 | 2.161 |
| 106 | FromBinaryShape/scalars / cbor | ns | 2.56641e+07 | 5.68711e+06 | 4.513 |
| 107 | FromBinaryShape/scalars / msgpack | ns | 2.44254e+07 | 5.56716e+06 | 4.387 |
| 108 | FromBinaryShape/scalars / ubjson | ns | 2.49328e+07 | 1.15997e+07 | 2.149 |
| 109 | FromCborChunkedString/10000 chunks | ns | 90902.3 | 61512.6 | 1.478 |
| 110 | FromMsgpack/canada | ns | 1.43596e+07 | 2.40425e+06 | 5.973 |
| 111 | FromMsgpack/citm_catalog | ns | 9.03526e+06 | 1.66705e+06 | 5.420 |
| 112 | FromMsgpack/floats | ns | 5.97879e+07 | 2.82314e+07 | 2.118 |
| 113 | FromMsgpack/jeopardy | ns | 1.14023e+09 | 1.1207e+08 | 10.174 |
| 114 | FromMsgpack/signed_ints | ns | 6.30943e+07 | 3.27435e+07 | 1.927 |
| 115 | FromMsgpack/small_signed_ints | ns | 6.34225e+07 | 3.52538e+07 | 1.799 |
| 116 | FromMsgpack/twitter | ns | 9.99383e+06 | 1.01241e+06 | 9.871 |
| 117 | FromMsgpack/unsigned_ints | ns | 5.61151e+07 | 2.82024e+07 | 1.990 |
| 118 | GetArray<std::array<int, 1000>> | ns | 2242.43 | 641.094 | 3.498 |
| 119 | GetArray<std::deque<int>> | ns | 8157.44 | 1172.81 | 6.955 |
| 120 | GetArray<std::set<int>> | ns | 58833.4 | 41553 | 1.416 |
| 121 | GetArray<std::vector<int>> | ns | 6725.67 | 670.816 | 10.026 |
| 122 | GetMap/citm_catalog | ns | 3.44193e+06 | 2.43771e+06 | 1.412 |
| 123 | GetVectorDouble/floats | ns | 6.7017e+06 | 650766 | 10.298 |
| 124 | GetVectorInt/signed_ints | ns | 8.06579e+06 | 3.40643e+06 | 2.368 |
| 125 | Iterate/canada | ns | 873618 | 445143 | 1.963 |
| 126 | Iterate/citm_catalog | ns | 301801 | 157823 | 1.912 |
| 127 | Iterate/twitter | ns | 87345 | 58936.4 | 1.482 |
| 128 | JsonPointerAt/citm_catalog | ns | 1.85369e+06 | 1.1599e+06 | 1.598 |
| 129 | JsonPointerAt/twitter | ns | 791487 | 599585 | 1.320 |
| 130 | JsonPointerContains/citm_catalog | ns | 2.73824e+06 | 1.25465e+06 | 2.182 |
| 131 | JsonPointerContains/twitter | ns | 1.32403e+06 | 620312 | 2.134 |
| 132 | JsonPointerParse/citm_catalog | ns | 5.72778e+06 | 2.21453e+06 | 2.586 |
| 133 | JsonPointerParse/twitter | ns | 2.4672e+06 | 1.00864e+06 | 2.446 |
| 134 | JsonPointerToString/citm_catalog | ns | 6.65484e+06 | 1.89788e+06 | 3.506 |
| 135 | JsonPointerToString/twitter | ns | 2.13903e+06 | 811916 | 2.635 |
| 136 | ParseFILE/canada | ns | 5.29148e+07 | 6.01689e+06 | 8.794 |
| 137 | ParseFILE/citm_catalog | ns | 3.49593e+07 | 2.34075e+06 | 14.935 |
| 138 | ParseFILE/twitter | ns | 1.38263e+07 | 1.34963e+06 | 10.245 |
| 139 | ParseFile/canada | ns | 1.76922e+07 | 6.22534e+06 | 2.842 |
| 140 | ParseFile/citm_catalog | ns | 7.85962e+06 | 2.47244e+06 | 3.179 |
| 141 | ParseFile/floats | ns | 1.29413e+08 | 4.83839e+07 | 2.675 |
| 142 | ParseFile/jeopardy | ns | 3.30328e+08 | 1.56161e+08 | 2.115 |
| 143 | ParseFile/signed_ints | ns | 1.27675e+08 | 4.65862e+07 | 2.741 |
| 144 | ParseFile/small_signed_ints | ns | 6.44651e+07 | 3.33577e+07 | 1.933 |
| 145 | ParseFile/twitter | ns | 3.39729e+06 | 1.37932e+06 | 2.463 |
| 146 | ParseFile/unsigned_ints | ns | 1.23326e+08 | 4.73146e+07 | 2.607 |
| 147 | ParseIndented/canada / 4 | ns | 3.41603e+07 | 4.60976e+06 | 7.410 |
| 148 | ParseIndented/citm_catalog / 4 | ns | 7.45491e+06 | 1.45798e+06 | 5.113 |
| 149 | ParseIndented/jeopardy / 4 | ns | 3.38801e+08 | 1.14349e+08 | 2.963 |
| 150 | ParseIndented/twitter / 4 | ns | 3.61322e+06 | 975391 | 3.704 |
| 151 | ParseStreamValues/ndjson | ns | 7.07974e+06 | 3.7154e+06 | 1.906 |
| 152 | ParseStreamValues/numbers | ns | 1.92884e+06 | 388042 | 4.971 |
| 153 | ParseString/canada | ns | 1.52389e+07 | 3.73427e+06 | 4.081 |
| 154 | ParseString/citm_catalog | ns | 7.40128e+06 | 1.44821e+06 | 5.111 |
| 155 | ParseString/floats | ns | 1.19856e+08 | 2.49654e+07 | 4.801 |
| 156 | ParseString/jeopardy | ns | 3.03449e+08 | 1.13053e+08 | 2.684 |
| 157 | ParseString/signed_ints | ns | 1.21175e+08 | 2.86805e+07 | 4.225 |
| 158 | ParseString/small_signed_ints | ns | 6.07636e+07 | 1.77037e+07 | 3.432 |
| 159 | ParseString/twitter | ns | 3.21074e+06 | 920551 | 3.488 |
| 160 | ParseString/unsigned_ints | ns | 1.20011e+08 | 2.85056e+07 | 4.210 |
| 161 | ParseStringOrdered/canada | ns | 1.52544e+07 | 3.77964e+06 | 4.036 |
| 162 | ParseStringOrdered/citm_catalog | ns | 1.09927e+07 | 1.34339e+06 | 8.183 |
| 163 | ParseStringOrdered/jeopardy | ns | 3.60435e+08 | 9.81296e+07 | 3.673 |
| 164 | ParseStringOrdered/twitter | ns | 4.41151e+06 | 803859 | 5.488 |
| 165 | Patch/canada | ns | 1.01249e+07 | 3.41425e+06 | 2.965 |
| 166 | Patch/citm_catalog | ns | 2.54415e+06 | 1.65413e+06 | 1.538 |
| 167 | Patch/twitter | ns | 1.4932e+06 | 870217 | 1.716 |
| 168 | SaxParse/canada | ns | 1.28611e+07 | 2.49889e+06 | 5.147 |
| 169 | SaxParse/citm_catalog | ns | 6.43798e+06 | 655372 | 9.823 |
| 170 | SaxParse/twitter | ns | 2.48859e+06 | 334038 | 7.450 |
| 171 | ToCbor/canada | ns | 1.39101e+06 | 554952 | 2.507 |
| 172 | ToCbor/citm_catalog | ns | 1.0334e+06 | 260397 | 3.969 |
| 173 | ToCbor/floats | ns | 8.93763e+06 | 1.84589e+06 | 4.842 |
| 174 | ToCbor/jeopardy | ns | 7.14785e+07 | 2.10551e+07 | 3.395 |
| 175 | ToCbor/signed_ints | ns | 1.13863e+07 | 2.74211e+06 | 4.152 |
| 176 | ToCbor/small_signed_ints | ns | 1.02822e+07 | 2.37622e+06 | 4.327 |
| 177 | ToCbor/twitter | ns | 636979 | 144369 | 4.412 |
| 178 | ToCbor/unsigned_ints | ns | 8.04576e+06 | 2.74066e+06 | 2.936 |
| 179 | ToJsonStrings/copy | ns | 5.85988e+06 | 5.60056e+06 | 1.046 |
| 180 | ToJsonStrings/move | ns | 5.92335e+06 | 3.04643e+06 | 1.944 |
| 181 | Unflatten/canada | ns | 4.17811e+07 | 1.08132e+07 | 3.864 |
| 182 | Unflatten/citm_catalog | ns | 9.80315e+06 | 3.53672e+06 | 2.772 |
| 183 | Unflatten/twitter | ns | 4.56968e+06 | 2.16566e+06 | 2.110 |