Back to home page

Project CMSSW displayed by LXR

 
 

    


File indexing completed on 2021-02-14 12:51:30

0001 /*****************************************************************************/
0002 /**

0003  * \file portable_oarchive.hpp

0004  * \brief Provides an archive to create portable binary files.

0005  * \author christian.pfligersdorffer@gmx.at

0006  * \version 5.1

0007  *

0008  * This pair of archives brings the advantages of binary streams to the cross

0009  * platform boost::serialization user. While being almost as fast as the native

0010  * binary archive it allows its files to be exchanged between cpu architectures

0011  * using different byte order (endianness). Speaking of speed: in serializing

0012  * numbers the (portable) binary approach is approximately ten times faster than

0013  * the ascii implementation (that is inherently portable)!

0014  *

0015  * Based on the portable archive example by Robert Ramey this implementation

0016  * uses Beman Dawes endian library and fp_utilities from Johan Rade, both being

0017  * in boost since 1.36. Prior to that you need to add them both (header only)

0018  * to your boost directory before you're able to use the archives provided. 

0019  * Our archives have been tested successfully for boost versions 1.33 to 1.49!

0020  *

0021  * \note Correct behaviour has so far been confirmed using PowerPC-32, x86-32

0022  *       and x86-64 platforms featuring different byte order. So there is a good

0023  *       chance it will instantly work for your specific setup. If you encounter

0024  *       problems or have suggestions please contact the author.

0025  *

0026  * \note Version 5.1 is now compatible with boost up to version 1.59. Thanks to

0027  *       ecotax for pointing to the issue with shared_ptr_helper.

0028  *

0029  * \note Version 5.0 is now compatible with boost up to version 1.49 and enables

0030  *       serialization of std::wstring by converting it to/from utf8 (thanks to

0031  *       Arash Abghari for this suggestion). With that all unit tests from the

0032  *       serialization library pass again with the notable exception of user

0033  *       defined primitive types. Those are not supported and as a result any

0034  *       user defined type to be used with the portable archives are required 

0035  *       to be at least object_serializable.

0036  *

0037  * \note Oliver Putz pointed out that -0.0 was not serialized correctly, so

0038  *       version 4.3 provides a fix for that. Thanks Ollie!

0039  *

0040  * \note Version 4.2 maintains compatibility with the latest boost 1.45 and adds

0041  *       serialization of special floating point values inf and NaN as proposed

0042  *       by Francois Mauger.

0043  *

0044  * \note Version 4.1 makes the archives work together with boost 1.40 and 1.41.

0045  *       Thanks to Francois Mauger for his suggestions.

0046  *

0047  * \note Version 4 removes one level of the inheritance hierarchy and directly

0048  *       builds upon binary primitive and basic binary archive, thereby fixing

0049  *       the last open issue regarding array serialization. Thanks to Robert

0050  *       Ramey for the hint.

0051  *

0052  * \note A few fixes introduced in version 3.1 let the archives pass all of the

0053  *       serialization tests. Thanks to Sergey Morozov for running the tests.

0054  *       Wouter Bijlsma pointed out where to find the fp_utilities and endian

0055  *       libraries headers inside the boost distribution. I would never have

0056  *       found them so thank him it works out of the box since boost 1.36.

0057  *

0058  * \note With Version 3.0 the archives have been made portable across different

0059  *       boost versions. For that purpose a header is added to the data that

0060  *       supplies the underlying serialization library version. Backwards

0061  *       compatibility is maintained by assuming library version boost 1.33 if

0062  *       the iarchive is created using the no_header flag. Whether a header is

0063  *       present or not can be guessed by peeking into the stream: the header's

0064  *       first byte is the magic number 127 coinciding with 'e'|'o'|'s' :-)

0065  *

0066  * \note Version 2.1 removes several compiler warnings and enhances floating

0067  *       point diagnostics to inform the user if some preconditions are violated

0068  *       on his platform. We do not strive for the universally portable solution

0069  *       in binary floating point serialization as desired by some boost users.

0070  *       Instead we support only the most widely used IEEE 754 format and try to

0071  *       detect when requirements are not met and hence our approach must fail.

0072  *       Contributions we made by Johan Rade and Ákos Maróy.

0073  *

0074  * \note Version 2.0 fixes a serious bug that effectively transformed most

0075  *       of negative integral values into positive values! For example the two

0076  *       numbers -12 and 234 were stored in the same 8-bit pattern and later

0077  *       always restored to 234. This was fixed in this version in a way that

0078  *       does not change the interpretation of existing archives that did work

0079  *       because there were no negative numbers. The other way round archives

0080  *       created by version 2.0 and containing negative numbers will raise an

0081  *       integer type size exception when reading it with version 1.0. Thanks

0082  *       to Markus Frohnmaier for testing the archives and finding the bug.

0083  *

0084  * \copyright The boost software license applies.

0085  */
0086 /*****************************************************************************/
0087 
0088 #ifndef CondFormats_Serialization_portable_oarchive_hpp
0089 #define CondFormats_Serialization_portable_oarchive_hpp
0090 
0091 #include <ostream>
0092 
0093 // basic headers

0094 #include <boost/version.hpp>
0095 #include <boost/utility/enable_if.hpp>
0096 #include <boost/archive/basic_binary_oprimitive.hpp>
0097 #include <boost/archive/basic_binary_oarchive.hpp>
0098 
0099 #if BOOST_VERSION >= 103500 && BOOST_VERSION < 105600
0100 #include <boost/archive/shared_ptr_helper.hpp>
0101 #endif
0102 
0103 #if BOOST_VERSION >= 104500
0104 #include <boost/program_options/config.hpp>
0105 #include <boost/program_options/detail/convert.hpp>
0106 #endif
0107 
0108 // funny polymorphics

0109 #if BOOST_VERSION < 103500
0110 #include <boost/archive/detail/polymorphic_oarchive_impl.hpp>
0111 #define POLYMORPHIC(T) boost::archive::detail::polymorphic_oarchive_impl<T>
0112 
0113 #elif BOOST_VERSION < 103600
0114 #include <boost/archive/detail/polymorphic_oarchive_dispatch.hpp>
0115 #define POLYMORPHIC(T) boost::archive::detail::polymorphic_oarchive_dispatch<T>
0116 
0117 #else
0118 #include <boost/archive/detail/polymorphic_oarchive_route.hpp>
0119 #define POLYMORPHIC(T) boost::archive::detail::polymorphic_oarchive_route<T>
0120 #endif
0121 
0122 // endian and fpclassify

0123 #if BOOST_VERSION < 103600
0124 #include <boost/integer/endian.hpp>
0125 #include <boost/math/fpclassify.hpp>
0126 #elif BOOST_VERSION < 104800
0127 #include <boost/spirit/home/support/detail/integer/endian.hpp>
0128 #include <boost/spirit/home/support/detail/math/fpclassify.hpp>
0129 #elif BOOST_VERSION >= 106900
0130 #include <boost/math/special_functions/fpclassify.hpp>
0131 #include <boost/endian/conversion.hpp>
0132 #else
0133 #include <boost/spirit/home/support/detail/endian/endian.hpp>
0134 #include <boost/spirit/home/support/detail/math/fpclassify.hpp>
0135 #endif
0136 
0137 // namespace alias fp_classify

0138 #if BOOST_VERSION < 103800
0139 namespace fp = boost::math;
0140 #elif BOOST_VERSION >= 106900
0141 namespace fp = boost::math;
0142 #else
0143 namespace fp = boost::spirit::math;
0144 #endif
0145 
0146 // namespace alias endian

0147 #if BOOST_VERSION < 104800
0148 namespace endian = boost::detail;
0149 #elif BOOST_VERSION >= 106900
0150 namespace endian = boost::endian;
0151 #else
0152 namespace endian = boost::spirit::detail;
0153 #endif
0154 
0155 #if BOOST_VERSION >= 104500 && !defined BOOST_NO_STD_WSTRING
0156 // used for wstring to utf8 conversion

0157 #include <boost/program_options/config.hpp>
0158 #include <boost/program_options/detail/convert.hpp>
0159 #endif
0160 
0161 // generic type traits for numeric types

0162 #include <boost/type_traits/is_integral.hpp>
0163 #include <boost/type_traits/is_signed.hpp>
0164 #include <boost/type_traits/is_arithmetic.hpp>
0165 #include <boost/type_traits/is_floating_point.hpp>
0166 
0167 #include "portable_archive_exception.hpp"
0168 
0169 // hint from Johan Rade: on VMS there is still support for

0170 // the VAX floating point format and this macro detects it

0171 #if defined(__vms) && defined(__DECCXX) && !__IEEE_FLOAT
0172 #error "VAX floating point format is not supported!"
0173 #endif
0174 
0175 namespace eos {
0176 
0177   // forward declaration

0178   class portable_oarchive;
0179 
0180   typedef boost::archive::basic_binary_oprimitive<portable_oarchive
0181 #if BOOST_VERSION < 103400
0182                                                   ,
0183                                                   std::ostream
0184 #else
0185                                                   ,
0186                                                   std::ostream::char_type,
0187                                                   std::ostream::traits_type
0188 #endif
0189                                                   >
0190       portable_oprimitive;
0191 
0192   /**

0193      * \brief Portable binary output archive using little endian format.

0194      *

0195      * This archive addresses integer size, endianness and floating point types so

0196      * that data can be transferred across different systems. The archive consists

0197      * primarily of three different save implementations for integral types,

0198      * floating point types and string types. Those functions are templates and use

0199      * enable_if to be correctly selected for overloading.

0200      *

0201      * \note The class is based on the portable binary example by Robert Ramey and

0202      *       uses Beman Dawes endian library plus fp_utilities by Johan Rade.

0203      */
0204   class portable_oarchive : public portable_oprimitive
0205 
0206       // the example derives from common_oarchive but that lacks the

0207       // save_override functions so we chose to stay one level higher

0208       ,
0209                             public boost::archive::basic_binary_oarchive<portable_oarchive>
0210 #if BOOST_VERSION >= 103500 && BOOST_VERSION < 105600
0211       // mix-in helper class for serializing shared_ptr

0212       ,
0213                             public boost::archive::detail::shared_ptr_helper
0214 #endif
0215   {
0216     // workaround for gcc: use a dummy struct

0217     // as additional argument type for overloading

0218     template <int>
0219     struct dummy {
0220       dummy(int) {}
0221     };
0222 
0223     // stores a signed char directly to stream

0224     inline void save_signed_char(const signed char& c) { portable_oprimitive::save(c); }
0225 
0226     // archive initialization

0227     void init(unsigned flags) {
0228       // it is vital to have version information if the archive is

0229       // to be parsed with a newer version of boost::serialization

0230       // therefor we create a header, no header means boost 1.33

0231       if (flags & boost::archive::no_header)
0232         BOOST_ASSERT(archive_version == 3);
0233       else {
0234         // write our minimalistic header (magic byte plus version)

0235         // the boost archives write a string instead - by calling

0236         // boost::archive::basic_binary_oarchive<derived_t>::init()

0237         save_signed_char(magic_byte);
0238 
0239         // write current version

0240         //              save<unsigned>(archive_version);

0241         operator<<(archive_version);
0242       }
0243     }
0244 
0245   public:
0246     /**

0247          * \brief Constructor on a stream using ios::binary mode!

0248          *

0249          * We cannot call basic_binary_oprimitive::init which stores type

0250          * sizes to the archive in order to detect transfers to non-compatible

0251          * platforms.

0252          *

0253          * We could have called basic_binary_oarchive::init which would create

0254          * the boost::serialization standard archive header containing also the

0255          * library version. Due to efficiency we stick with our own.

0256          */
0257     portable_oarchive(std::ostream& os, unsigned flags = 0)
0258 #if BOOST_VERSION < 103400
0259         : portable_oprimitive(os, flags & boost::archive::no_codecvt)
0260 #else
0261         : portable_oprimitive(*os.rdbuf(), flags & boost::archive::no_codecvt)
0262 #endif
0263           ,
0264           boost::archive::basic_binary_oarchive<portable_oarchive>(flags) {
0265       init(flags);
0266     }
0267 
0268 #if BOOST_VERSION >= 103400
0269     portable_oarchive(std::streambuf& sb, unsigned flags = 0)
0270         : portable_oprimitive(sb, flags & boost::archive::no_codecvt),
0271           boost::archive::basic_binary_oarchive<portable_oarchive>(flags) {
0272       init(flags);
0273     }
0274 #endif
0275 
0276     //! Save narrow strings.

0277     void save(const std::string& s) { portable_oprimitive::save(s); }
0278 
0279 #ifndef BOOST_NO_STD_WSTRING
0280     /**

0281          * \brief Save wide strings.

0282          *

0283          * This is rather tricky to get right for true portability as there

0284          * are so many different character encodings around. However, wide

0285          * strings that are encoded in one of the Unicode schemes only need

0286          * to be _transcoded_ which is a lot easier actually.

0287          *

0288          * We expect the input string to be encoded in the system's native

0289          * format, ie. UTF-16 on Windows and UTF-32 on Linux machines. Don't

0290          * know about Mac here so I can't really say about that.

0291          */
0292     void save(const std::wstring& s) { save(boost::to_utf8(s)); }
0293 #endif
0294 
0295     /**

0296          * \brief Saving bool type.

0297          *

0298          * Saving bool directly, not by const reference

0299          * because of tracking_type's operator (bool).

0300          *

0301          * \note If you cannot compile your application and it says something

0302          * about save(bool) cannot convert your type const A& into bool then

0303          * you should check your BOOST_CLASS_IMPLEMENTATION setting for A, as

0304          * portable_archive is not able to handle custom primitive types in

0305          * a general manner.

0306          */
0307     void save(const bool& b) {
0308       save_signed_char(b);
0309       if (b)
0310         save_signed_char('T');
0311     }
0312 
0313     /**

0314          * \brief Save integer types.

0315          *

0316          * First we save the size information ie. the number of bytes that hold the

0317          * actual data. We subsequently transform the data using store_little_endian

0318          * and store non-zero bytes to the stream.

0319          */
0320     template <typename T>
0321     typename boost::enable_if<boost::is_integral<T> >::type save(const T& t, dummy<2> = 0) {
0322       if (T temp = t) {
0323         // examine the number of bytes

0324         // needed to represent the number

0325         signed char size = 0;
0326         if (sizeof(T) == 1) {
0327           size = 1;
0328         } else {
0329           do {
0330             temp >>= CHAR_BIT;
0331             ++size;
0332           } while (temp != 0 && temp != (T)-1);
0333         }
0334 
0335         // encode the sign bit into the size

0336         save_signed_char(t > 0 ? size : -size);
0337         BOOST_ASSERT(t > 0 || boost::is_signed<T>::value);
0338 
0339 // we choose to use little endian because this way we just

0340 // save the first size bytes to the stream and skip the rest

0341 #if BOOST_VERSION >= 106900
0342         temp = endian::native_to_little(t);
0343 #else
0344         endian::store_little_endian<T, sizeof(T)>(&temp, t);
0345 #endif
0346         save_binary(&temp, size);
0347       }
0348       // zero optimization

0349       else
0350         save_signed_char(0);
0351     }
0352 
0353     /**

0354          * \brief Save floating point types.

0355          * 

0356          * We simply rely on fp_traits to extract the bit pattern into an (unsigned)

0357          * integral type and store that into the stream. Francois Mauger provided

0358          * standardized behaviour for special values like inf and NaN, that need to

0359          * be serialized in his application.

0360          *

0361          * \note by Johan Rade (author of the floating point utilities library):

0362          * Be warned that the math::detail::fp_traits<T>::type::get_bits() function 

0363          * is *not* guaranteed to give you all bits of the floating point number. It

0364          * will give you all bits if and only if there is an integer type that has

0365          * the same size as the floating point you are copying from. It will not

0366          * give you all bits for double if there is no uint64_t. It will not give

0367          * you all bits for long double if sizeof(long double) > 8 or there is no

0368          * uint64_t. 

0369          * 

0370          * The member fp_traits<T>::type::coverage will tell you whether all bits

0371          * are copied. This is a typedef for either math::detail::all_bits or

0372          * math::detail::not_all_bits. 

0373          * 

0374          * If the function does not copy all bits, then it will copy the most

0375          * significant bits. So if you serialize and deserialize the way you

0376          * describe, and fp_traits<T>::type::coverage is math::detail::not_all_bits,

0377          * then your floating point numbers will be truncated. This will introduce

0378          * small rounding off errors. 

0379          */
0380     template <typename T>
0381     typename boost::enable_if<boost::is_floating_point<T> >::type save(const T& t, dummy<3> = 0) {
0382       typedef typename fp::detail::fp_traits<T>::type traits;
0383 
0384       // if the no_infnan flag is set we must throw here

0385       if (get_flags() & no_infnan && !fp::isfinite(t))
0386         throw portable_archive_exception(t);
0387 
0388       // if you end here there are three possibilities:

0389       // 1. you're serializing a long double which is not portable

0390       // 2. you're serializing a double but have no 64 bit integer

0391       // 3. your machine is using an unknown floating point format

0392       // after reading the note above you still might decide to

0393       // deactivate this static assert and try if it works out.

0394       typename traits::bits bits;
0395       static_assert(sizeof(bits) == sizeof(T));
0396       static_assert(std::numeric_limits<T>::is_iec559);
0397 
0398       // examine value closely

0399       switch (fp::fpclassify(t)) {
0400 //case FP_ZERO: bits = 0; break;

0401 #if BOOST_VERSION >= 106900
0402         case FP_NAN:
0403           bits = traits::exponent | traits::significand;
0404           break;
0405 #else
0406         case FP_NAN:
0407           bits = traits::exponent | traits::mantissa;
0408           break;
0409 #endif
0410         case FP_INFINITE:
0411           bits = traits::exponent | (t < 0) * traits::sign;
0412           break;
0413         case FP_SUBNORMAL:
0414           assert(std::numeric_limits<T>::has_denorm);  // pass

0415         case FP_ZERO:                                  // note that floats can be ±0.0

0416         case FP_NORMAL:
0417           traits::get_bits(t, bits);
0418           break;
0419         default:
0420           throw portable_archive_exception(t);
0421       }
0422 
0423       save(bits);
0424     }
0425 
0426     // in boost 1.44 version_type was splitted into library_version_type and

0427     // item_version_type, plus a whole bunch of additional strong typedefs.

0428     template <typename T>
0429     typename boost::disable_if<boost::is_arithmetic<T> >::type save(const T& t, dummy<4> = 0) {
0430       // we provide a generic save routine for all types that feature

0431       // conversion operators into an unsigned integer value like those

0432       // created through BOOST_STRONG_TYPEDEF(X, some unsigned int) like

0433       // library_version_type, collection_size_type, item_version_type,

0434       // class_id_type, object_id_type, version_type and tracking_type

0435       save((typename boost::uint_t<sizeof(T) * CHAR_BIT>::least)(t));
0436     }
0437   };
0438 
0439   // polymorphic portable binary oarchive typedef

0440   typedef POLYMORPHIC(portable_oarchive) polymorphic_portable_oarchive;
0441 #undef POLYMORPHIC
0442 
0443 }  // namespace eos

0444 
0445 // required by export

0446 #if BOOST_VERSION < 103500
0447 #define BOOST_ARCHIVE_CUSTOM_OARCHIVE_TYPES eos::portable_oarchive
0448 #else
0449 BOOST_SERIALIZATION_REGISTER_ARCHIVE(eos::portable_oarchive)
0450 BOOST_SERIALIZATION_REGISTER_ARCHIVE(eos::polymorphic_portable_oarchive)
0451 #endif
0452 
0453 // if you include this header multiple times and your compiler is picky

0454 // about multiple template instantiations (eg. gcc is) then you need to

0455 // define NO_EXPLICIT_TEMPLATE_INSTANTIATION before every include but one

0456 // or you move the instantiation section into an implementation file

0457 // #ifndef NO_EXPLICIT_TEMPLATE_INSTANTIATION

0458 //

0459 // #include <boost/archive/impl/basic_binary_oarchive.ipp>

0460 // #include <boost/archive/impl/basic_binary_oprimitive.ipp>

0461 //

0462 // #if BOOST_VERSION < 104000

0463 // #include <boost/archive/impl/archive_pointer_oserializer.ipp>

0464 // #elif !defined BOOST_ARCHIVE_SERIALIZER_INCLUDED

0465 // #include <boost/archive/impl/archive_serializer_map.ipp>

0466 // #define BOOST_ARCHIVE_SERIALIZER_INCLUDED

0467 // #endif

0468 //

0469 // namespace boost {

0470 //   namespace archive {

0471 //

0472 //     // explicitly instantiate for this type of binary stream

0473 //     template class basic_binary_oarchive<eos::portable_oarchive>;

0474 //

0475 //     template class basic_binary_oprimitive<eos::portable_oarchive

0476 // #if BOOST_VERSION < 103400

0477 //                                            ,

0478 //                                            std::ostream

0479 // #else

0480 //                                            ,

0481 //                                            std::ostream::char_type,

0482 //                                            std::ostream::traits_type

0483 // #endif

0484 //                                            >;

0485 //

0486 // #if BOOST_VERSION < 104000

0487 //     template class detail::archive_pointer_oserializer<eos::portable_oarchive>;

0488 // #else

0489 //     template class detail::archive_serializer_map<eos::portable_oarchive>;

0490 //     //template class detail::archive_serializer_map<eos::polymorphic_portable_oarchive>;

0491 // #endif

0492 //

0493 //   }  // namespace archive

0494 // }  // namespace boost

0495 //

0496 // #endif

0497 
0498 #endif