c++11 string转ing,将std :: string编码/解码为UTF-16

I have to handle a file format (both read from and write to it) in which strings are encoded in UTF-16 (2 bytes per character). Since characters out of the ASCII table are rarely used in the application domain, all of the strings in my C++ model classes are stored in instances of std::string (UTF-8 encoded).

I'm looking for a library (searched in STL and Boost with no luck) or a set of C/C++ functions to handle this std::string UTF-16 conversion when loading from or saving to file format (actually modeled as a bytestream) including the generation/recognition of surrogate pairs and all that Unicode stuffs (I'm admittedly no expert with)...

Any suggestions? Thanks!

EDIT: forgot to mention it should be cross-platform (Win / Mac) and cannot use C++11.

解决方案

C++11 has this functionality:

std::string s = u8"Hello, World!";

// #include

std::wstring_convert<:codecvt>,char16_t> convert;

std::u16string u16 = convert.from_bytes(s);

std::string u8 = convert.to_bytes(u16);

However to my knowledge the only implementation that has this so far is libc++. C++11 also has std::codecvt_utf8_utf16 which some other implementations have. Specifically, codecvt_utf8_utf16 works in VS 2010 and above, and since wchar_t is used by Windows to represent UTF-16 you can use this to convert between UTF-8 and Windows' native encoding.

The specialization codecvt converts between the UTF-16 and UTF-8 encoding

schemes, and the specialization codecvt converts between the UTF-32 and

UTF-8 encoding schemes.

— [locale.codecvt] 22.4.1.4/3

Oh, and std::codecvt specializations have protected destructors, and wstring_convert requires access to the destructor so you really need an adapter:

template

class usable_facet : public Facet {

public:

using Facet::Facet; // inherit constructors

~usable_facet() {}

// workaround for compilers without inheriting constructors:

// template usable_facet(Args&& ...args) : Facet(std::forward(args)...) {}

};

template

using codecvt = usable_facet<:codecvt externt statet>>;

std::wstring_convert> convert;