iconv

iconv Command[@more@]
[ Bottom of Page | Previous Page | Next Page | Contents | Index | Library Home | Legal | Search ]

National Language Support Guide and Reference

Using the iconv Command

Any converter installed in the system can be used through the iconv command, which uses the iconv library. The iconv command acts as a filter for converting from one code set to another. For example, the following command filters data from PC Code (IBM-850) to ISO8859-1:

 
 

The iconv command converts the encoding of characters read from either standard input or the specified file and then writes the results to standard output.

Understanding libiconv

The iconv application programming interface (API) consists of the following subroutines that accomplish conversion:

iconv_open
Performs the initialization required to convert characters from the code set specified by the FromCode parameter to the code set specified by the ToCode parameter. The strings specified are dependent on the converters installed in the system. If initialization is successful, the converter descriptor, iconv_t, is returned in its initial state.
iconv
Invokes the converter function using the descriptor obtained from the iconv_open subroutine. The inbuf parameter points to the first character in the input buffer, and the inbytesleft parameter indicates the number of bytes to the end of the buffer being converted. The outbuf parameter points to the first available byte in the output buffer, and the outbytesleft parameter indicates the number of available bytes to the end of the buffer.

For state-dependent encoding, the subroutine is placed in its initial state by a call for which the inbuf value is a null pointer. Subsequent calls with the inbuf parameter as something other than a null pointer cause the internal state of the function to be altered as necessary.

iconv_close
Closes the conversion descriptor specified by the cd variable and makes it usable again.

In a network environment, the following factors determine how data should be converted:

Code sets of the sender and the receiver Communication protocol (8-bit or 7-bit data)

The following table outlines the conversion methods and recommends how to convert data in different situations. See the Interchange Converters--7-bit and the Interchange Converters--8-bit for more information.

Outline of Methods and Recommended Choices
Communication with system using the same code setCommunication with system using different code set (or receiver's code set is unknown)
ProtocolProtocol
Method to choose7-bit only8-bit7-bit only8-bit
as isNot validBest choiceNot validNot valid if remote code set is unknown
fold7OKOKBest choiceOK
fold8Not validOKNot validBest choice
uucodeBest choiceOKNot validNot valid

If the sender uses the same code set as the receiver, the following possibilities exist:

When protocol allows 8-bit data, the data can be sent without conversions. When protocol allows only 7-bit data, the 8-bit code points must be mapped to 7-bit values. Use the iconv interface and one of the following methods:
uucodeProvides the same mapping as the uuencode and uudecode commands. This is the recommended method. For more information, see Interchange Converters--uucode.
7-bitConverts internal code sets using 7-bit data. This method passes ASCII without any change. For more information, see Interchange Converters--7-bit.

If the sender uses a code set different from the receiver, there are two possibilities:

When protocol allows only 7-bit data, use the fold7 method. When protocol allows 8-bit data and you know the receiver's code set, use the iconv interface to convert the data. If you do not know the receiver's code set, use the following method:
8-bitConverts internal code sets to standard interchange formats. The 8-bit data is transmitted and the information is preserved so that the receiver can reconstruct the data in its code set. For more information, see Interchange Converters--8-bit.

Using the iconv_open Subroutine

The following examples illustrate how to use the iconv_open subroutine in different situations:

When the sender and receiver use the same code sets, and if the protocol allows 8-bit data, you can send data without converting it. If the protocol allows only 7-bit data, do the following:
Sender:
cd = iconv_open("uucode", nl_langinfo(CODESET));


Receiver:
cd = iconv_open(nl_langinfo(CODESET), "uucode");
Whne the sender and receiver use different code sets, and if the protocol allows 8-bit data and the receiver's code set is unknown, do the following:
Sender:
cd = iconv_open("fold8", nl_langinfo(CODESET));


Receiver:
cd = iconv_open(nl_langinfo(CODESET),"fold8" );

If the protocol allows only 7-bit data, do the following:

Sender:
cd = iconv_open("fold7", nl_langinfo(CODESET));


Receiver:
cd = iconv_open(nl_langinfo(CODESET), "fold7" );

The iconv_open subroutine uses the LOCPATH environment variable to search for a converter whose name is in the following form:

The FromCodeSet string represents the sender's code set, and the ToCodeSet string represents the receiver's code set. The underscore character separates the two strings.

Note:
All setuid and setgid programs ignore the LOCPATH environment variable.

Because the iconv converter is a loadable object module, a different object is required when running in the 64-bit environment. In the 64-bit environment, the iconv_open routine uses the LOCPATH environment variable to search for a converter whose name is in the following form:

 
 

The iconv library automatically chooses whether to load the standard converter object or the 64-bit converter object. If the iconv_open subroutine does not find the converter, it uses the from,to pair to search for a file that defines a table-driven conversion. The file contains a conversion table created by the genxlt command.

The iconvTable converter uses the LOCPATH environment variable to search for a file whose name is in the following form:

If the converter is found, it performs a load operation and is initialized. The converter descriptor, iconv_t, is returned in its initial state.

Converter Programs versus Tables

Converter programs are executable functions that convert data according to a set of rules. Converter tables are single-byte conversion tables that perform stateless conversions. Programs and tables are in separate directories, as follows:

/usr/lib/nls/loc/iconvConverter programs
/usr/lib/nls/loc/iconvTableConverter tables

After a converter program is compiled and linked with the libiconv.a library, the program is placed in the /usr/lib/nls/loc/iconv directory.

To build a table converter, build a source converter table file. Use the genxlt command to compile translation tables into a format understood by the table converter. The output file is then placed in the /usr/lib/nls/loc/iconvTable directory.

Unicode and Universal Converters

Unicode (or UCS-2) conversion tables are found in:

 
 

The $LOCPATH/uconv/UCSTBL converter program is used to perform the conversion to and from UCS-2 using the iconv utilities.

A Universal converter program is provided that can be used to convert between any two code sets whose conversions to and from UCS-2 is defined. Given the following uconv tables:

 
 

a universal conversion can be defined that maps the following:

 
 

by use of the $LOCPATH/iconv/Universal_UCS_Conv.

Universal UCS Converter

UCS-2 is a universal 16-bit encoding that can be used as an interchange medium to provide conversion capability between virtually any code sets. The conversion can be accomplished using the Universal UCS Converter, which converts between any two code sets XXX and YYY as follows:

 
 

The XXX and YYY conversions must be included in the supported List of UCS-2 Interchange Converters, and must be installed on the system.

The universal converter is installed as the file /usr/lib/nls/loc/iconv/Universal_UCS_Conv.

The conversion between multibyte and wide character code depends on the current locale setting. Do not exchange wide character codes between two processes, unless you have knowledge that each locale that might be used handles wide character codes in a consistent fashion. Most locales for this operating system use the Unicode character value as a wide character code, except locales based on IBM-eucTW codesets.

Using Converters

The iconv interface is a set of the following subroutines used to open, perform, and close conversions:

iconv_open iconv iconv_close

Code Set Conversion Filter Example

The following example shows how you can use these subroutines to create a code set conversion filter that accepts the ToCode and FromCode parameters as input arguments:

 
 
 
 
 
 

Naming Converters

Code set names are in the form CodesetRegistry-CodesetEncoding where:

CodesetRegistryIdentifies the registration authority for the encoding. The CodesetRegistry must be made of characters from the portable code set (usually A-Z and 0-9).
CodesetEncodingIdentifies the coded character set defined by the registered authority.

The from,to variable used by the iconv command and iconv_open subroutine identifies a file whose name should be in the form /usr/lib/nls/loc/iconv/%f_%t or /usr/lib/nls/loc/iconvTable/%f_%t, where:

%fRepresents the FromCode set name
%tRepresents the ToCode set name

List of Converters

Converters change data from one code set to another. The sets of converters supported with the iconv library are listed in the following sections. All converters shipped with the BOS Runtime Environment are located in the /usr/lib/nls/loc/iconv/* or /usr/lib/nls/loc/iconvTable/* directory.

These directories also contain private converters; that is, they are used by other converters. However, users and programs should only depend on the converters in the following lists.

Any converter shipped with the BOS Runtime Environment and not listed here should be considered private and subject to change or deletion. Converters supplied by other products can be placed in the /usr/lib/nls/loc/iconv/* or /usr/lib/nls/loc/iconvTable/* directory.

Programmers are encouraged to use registered code set names or code set names associated with an application. The X Consortium maintains a registry of code set names for reference. See Code Sets for National Language Support for more information about code sets.

PC, ISO, and EBCDIC Code Set Converters

These converters provide conversion between PC, ISO, and EBCDIC single-byte stateless code sets. The following types of conversions are supported: PC to/from ISO, PC to/from EBCDIC, and ISO to/from EBCDIC.

Conversion is provided between compatible code sets such as Latin-1 to Latin-1 and Greek to Greek. However, conversion between different EBCDIC national code sets is not supported. For information about converting between incompatible character sets, refer to the Interchange Converters--7-bit and the Interchange Converters--8-bit.

Conversion tables in the iconvTable directory are created by the genxlt command.

Compatible Code Set Names

The following table lists code set names that are compatible. Each line defines to/from strings that may be used when requesting a converter.

Note:
The PC and ISO code sets are ASCII-based.
Code Set Compatibility
Character SetLanguagesPCISOEBCDIC
Latin-1U.S. English, Portuguese, Canadian FrenchN/AISO8859-1IBM-037
Latin-1Danish, NorwegianN/AISO8859-1IBM-277
Latin-1Finnish, SwedishN/AISO8859-1IBM-278
Latin-1ItalianN/AISO8859-1IBM-280
Latin-1JapaneseN/AISO8859-1IBM-281
Latin-1SpanishN/AISO8859-1IBM-284
Latin-1U.K. EnglishN/AISO8859-1IBM-285
Latin-1GermanN/AISO8859-1IBM-273
Latin-1FrenchN/AISO8859-1IBM-297
Latin-1Belgian, Swiss GermanN/AISO8859-1IBM-500
Latin-2Croatian, Czechoslovakian, Hungarian, Polish, Romanian, Serbian Latin, Slovak, SloveneIBM-852ISO88859-2IBM-870
CyrillicBulgarian, Macedonian, Serbian Cyrillic, RussianIBM-855ISO8859-5IBM-880 IBM-1025
CyrillicRussianIBM-866ISO8859-5IBM-1025
HebrewHebrewIBM-856 IBM-862ISO8859-8IBM-424 IBM-803
TurkishTurkishIBM-857ISO8859-9IBM-1026
ArabicArabicIBM-864 IBM-1046ISO8859-6IBM-420
GreekGreekIBM-869ISO8859-7IBM-875
GreekGreekIBM-869ISO8859-7IBM-875
BalticLithuanian, Latvian, EstonianIBM-921 IBM-922IBM-1112 IBM-1122
Note:
A character that exists in the source code set but does not exist in the target code set is converted to a converter-defined substitute character.
Files

The following table describes the inconvTable converters found in the /usr/lib/nls/loc/iconvTable directory:

iconvTable Converters
Converter TableDescriptionLanguage
IBM-037_IBM-850IBM-037 to IBM-850U.S. English, Portuguese, Canadian-French
IBM-273_IBM-850IBM-273 to IBM-850German
IBM-277_IBM-850IBM-277 to IBM-850Danish, Norwegian
IBM-278_IBM-850IBM-278 to IBM-850Finnish, Swedish
IBM-280_IBM-850IBM-280 to IBM-850Italian
IBM-281_IBM-850IBM-281 to IBM-850Japanese-Latin
IBM-284_IBM-850IBM-284 to IBM-850Spanish
IBM-285_IBM-850IBM-285 to IBM-850U.K. English
IBM-297_IBM-850IBM-297 to IBM-850French
IBM-420_IBM_1046IBM-420 to IBM-1046Arabic
IBM-424_IBM-856IBM-424 to IBM-856Hebrew
IBM-424_IBM-862IBM-424 to IBM-862Hebrew
IBM-500_IBM-850IBM-500 to IBM-850Belgian, Swiss German
IBM-803_IBM-856IBM-803 to IBM-856Hebrew
IBM-803_IBM-862IBM-803 to IBM-862Hebrew
IBM-850_IBM-037IBM-850 to IBM-037U.S. English, Portuguese, Canadian-French
IBM-850_IBM-273IBM-850 to IBM-273German
IBM-850_IBM-277IBM-850 to IBM-277Danish, Norwegian
IBM-850_IBM-278IBM-850 to IBM-278Finnish, Swedish
IBM-850_IBM-280IBM-850 to IBM-280Italian
IBM-850_IBM-281IBM-850 to IBM-281Japanese-Latin
IBM-850_IBM-284IBM-850 to IBM-284Spanish
IBM-850_IBM-285IBM-850 to IBM-285U.K. English
IBM-850_IBM-297IBM-850 to IBM-297French
IBM-850_IBM-500IBM-850 to IBM-500Belgian, Swiss German
IBM-856_IBM-424IBM-856 to IBM-424Hebrew
IBM-856_IBM-803IBM-856 to IBM-803Hebrew
IBM-856_IBM-862IBM-856 to IBM-862Hebrew
IBM-862_IBM-424IBM-862 to IBM-424Hebrew
IBM-862_IBM-803IBM-862 to IBM-803Hebrew
IBM-862_IBM-856IBM-862 to IBM-856Hebrew
IBM-864_IBM-1046IBM-864 to IBM-1046Arabic
IBM-921_IBM-1112IBM-921 to IBM-1112Lithuanian, Latvian
IBM-922_IBM-1122IBM-922 to IBM-1122Estonian
IBM-1112_IBM-921IBM-1121 to IBM-921Lithuanian, Latvian
IBM-1122_IBM-922IBM-1122 to IBM-922Estonian
IBM-1046_IBM-420IBM-1046 to IBM-420Arabic
IBM-1046_IBM-864IBM-1046 to IBM-864Arabic
IBM-037_ISO8859-1IBM-037 to ISO8859-1U.S. English, Portuguese, Canadian French
IBM-273_ISO8859-1IBM-273 to ISO8859-1German
IBM-277_ISO8859-1IBM-277 to ISO8859-1Danish, Norwegian
IBM-278_ISO8859-1IBM-278 to ISO8859-1Finnish, Swedish
IBM-280_ISO8859-1IBM-280 to ISO8859-1Italian
IBM-281_ISO8859-1IBM-281 to ISO8859-1Japanese-Latin
IBM-284_ISO8859-1IBM-284 to ISO8859-1Spanish
IBM-285_ISO8859-1IBM-285 to ISO8859-1U.K. English
IBM-297_ISO8859-1IBM-297 to ISO8859-1French
IBM-420_ISO8859-6IBM-420 to ISO8859-6Arabic
IBM-424_ISO8859-8IBM-424 to ISO8859-8Hebrew
IBM-500_ISO8859-1IBM-500 to ISO8859-1Belgian, Swiss German
IBM-803_ISO8859-8IBM-803 to ISO8859-8Hebrew
IBM-852_ISO8859-2IBM-852 to ISO8859-2Croatian, Czechoslovakian, Hungarian, Polish, Romanian, Serbian Latin, Slovak, Slovene
IBM-855_ISO8859-5IBM-855 to ISO8859-5Bulgarian, Macedonian, Serbian Cyrillic, Russian
IBM-866_ISO8859-5IBM-866 to ISO8859-5Russian
IBM-869_ISO8859-7IBM-869 to ISO8859-7Greek
IBM-875_ISO8859-7IBM-875 to ISO8859-7Greek
IBM-870_ISO8859-2IBM-870 to ISO8859-2Croatian, Czechoslovakian, Hungarian, Polish, Romanian, Serbian, Slovak, Slovene
IBM-880_ISO8859-5IBM-880 to ISO8859-5Bulgarian, Macedonian, Serbian Cyrillic, Russian
IBM-1025_ISO8859-5IBM-1025 to ISO8859-5Bulgarian, Macedonian, Serbian Cyrillic, Russian
IBM-857_ISO8859-9IBM-857 to ISO8859-9Turkish
IBM-1026_ISO8859-9IBM-1026 to ISO8859-9Turkish
IBM-850_ISO8859-1IBM-850 to ISO8859-1Latin
IBM-856_ISO8859-8IBM-856 to ISO8859-8Hebrew
IBM-862_ISO8859-8IBM-862 to ISO8859-8Hebrew
IBM-864_ISO8859-6IBM-864 to ISO8859-6Arabic
IBM-1046_ISO8859-6IBM-1046 to ISO8859-6Arabic
ISO8859-1_IBM-850ISO8859-1 to IBM-850Latin
ISO8859-6_IBM-864ISO8859-6 to IBM-864Arabic
ISO8859-6_IBM-1046ISO8859-6 to IBM-1046Arabic
ISO8859-8_IBM-856ISO8859-8 to IBM-856Hebrew
ISO8859-8_IBM-862ISO8859-8 to IBM-862Hebrew
ISO8859-1_IBM-037ISO8859-1 to IBM-037U.S. English, Portuguese, Canadian French
ISO8859-1_IBM-273ISO8859-1 to IBM-273German
ISO8859-1_IBM-277ISO8859-1 to IBM-277Danish, Norwegian
ISO8859-1_IBM-278ISO8859-1 to IBM-278Finnish, Swedish
ISO8859-1_IBM-280ISO8859-1 to IBM-280Italian
ISO8859-1_IBM-281ISO8859-1 to IBM-281Japanese-Latin
ISO8859-1_IBM-284ISO8859-1 to IBM-284Spanish
ISO8859-1_IBM-285ISO8859-1 to IBM-285U.K. English
ISO8859-1_IBM-297ISO8859-1 to IBM-297French
ISO8859-1_IBM-500ISO8859-1 to IBM-500Belgian, Swiss German
ISO8859-2_IBM-852ISO8859-2 to IBM-852Croatian, Czechoslovakian, Hungarian, Polish, Romanian, Serbian Latin, Slovak, Slovene
ISO8859-2_IBM-870ISO8859-2 to IBM-870Croatian, Czechoslovakian, Hungarian, Polish, Romanian, Serbian Latin, Slovak, Slovene
ISO8859-5_IBM-855ISO8859-5 to IBM-855Bulgarian, Macedonian, Serbian Cyrillic, Russian
ISO8859-5_IBM-880ISO8859-5 to IBM-880Bulgarian, Macedonian, Serbian Cyrillic, Russian
ISO8859-5_IBM-1025ISO8859-5 to IBM-1025Bulgarian, Macedonian, Serbian Cyrillic, Russian
ISO8859-6_IBM-420ISO8859-6 to IBM-420Arabic
ISO8859-5_IBM-866ISO8859-5 to IBM-866Russian
ISO8859-7_IBM-869ISO8859-7 to IBM-869Greek
ISO8859-7_IBM-875ISO8859-7 to IBM-875Greek
ISO8859-8_IBM-424ISO8859-8 to IBM-424Hebrew
ISO8859-8_IBM-803ISO8859-8 to IBM-803Hebrew
ISO8859-9_IBM-857ISO8859-9 to IBM-857Turkish
ISO8859-9_IBM-1026ISO8859-9 to IBM-1026Turkish

Multibyte Code Set Converters

Multibyte code-set converters convert characters among the following code sets:

PC multibyte code sets EUC multibyte code sets (ISO-based) EBCDIC multibyte code sets

The following table lists code set names that are compatible. Each line defines to/from strings that may be used when requesting a converter.

Code Set Compatibility
LanguagePCISOEBCDIC
JapaneseIBM-932IBM-eucJPIBM-930, IBM-939
Japanese
(MS compatible)
IBM-943IBM-eucJPIBM-930, IBM-939
KoreanIBM-934IBM-eucKRIBM-933
Traditional ChineseIBM-938, big-5IBM-eucTWIBM-937
Simplified ChineseIBM-1381IBM-eucCNIBM-935
Conversions between Simplified and Traditional Chinese are provided (IBM-eucTW UTF-8 is an additional code set. See UTF-8 Interchange Converters for more information.
Files

The following list describes the Multibyte Code Set converters that are found in the /usr/lib/nls/loc/iconv directory.

ConverterDescription
IBM-eucJP_IBM-932IBM-eucJP to IBM-932
IBM-eucJP_IBM-943IBM-eucJP to IBM-943
IBM-eucJP_IBM-930IBM-eucJP to IBM-930
IBM-eucCN_IBM-936(PC5550)IBM-eucCN to IBM-936(PC5550)
IBM-eucCN_IBM-935IBM-eucCN to IBM-935
IBM-eucJP_IBM-939IBM-eucJP to IBM-939
IBM-eucCN_IBM-1381IBM-eucCN to IBM-1381
IBM-943_IBM-932IBM-943 to IBM-932
IBM-932_IBM-943IBM-932 to IBM-943
IBM-930_IBM-932IBM-930 to IBM-932
IBM-930_IBM-943IBM-930 to IBM-943
IBM-930_IBM-eucJPIBM-930 to IBM-eucJP
IBM-932_IBM-eucJPIBM-932 to IBM-eucJP
IBM-932_IBM-930IBM-932 to IBM-930
IBM-943_IBM-eucJPIBM-943 to IBM-eucJP
IBM-943_IBM-930IBM-943 to IBM-930
IBM-936(PC5550)_IBM-935IBM-936(PC5550) to IBM-935
IBM-936_IBM-935IBM-936 to IBM-935
IBM-932_IBM-939IBM-932 to IBM-939
IBM-939_IBM-932IBM-939 to IBM-932
IBM-943_IBM-939IBM-943 to IBM-939
IBM-939_IBM-943IBM-939 to IBM-943
IBM-935_IBM-936(PC5550)IBM-935 to IBM-936(PC5550)
IBM-935_IBM-936IBM-935 to IBM-936
IBM-1381_IBM-935IBM-1381 to IBM-935
IBM-935_IBM-1381IBM-935 to IBM-1381
IBM-935_IBM-eucCNIBM-935 to IBM-eucCN
IBM-936(PC5550)_IBM-eucCNIBM-936(PC5550) to IBM-eucCN
IBM-eucTW_IBM-eucCNIBM-eucTW to IBM-eucCN
big5_IBM-eucCNbig5 to IBM-eucCN
IBM-1381_IBM-eucCNIBM-1381 to IBM-eucCN
IBM-939_IBM-eucJPIBM-939 to IBM-eucJP
IBM-eucKR_IBM-934IBM-eucKR to IBM-934
IBM-934_IBM-eucKRIBM-934 to IBM-eucKR
IBM-eucKR_IBM-933IBM-eucKR to IBM-933
IBM-933_IBM-eucKRIBM-933 to IBM-eucKR
IBM-eucTW_IBM-937IBM-eucTW to IBM-937
IBM-938_IBM-937IBM-938 to IBM-937
big-5_IBM-937big-5 to IBM-937
IBM-eucCN_IBM-eucTWIBM-eucCN to IBM-eucTW
IBM-937_IBM-eucTWIBM-937 to IBM-eucTW
IBM-937_IBM-938IBM-937 to IBM-938
IBM-eucTW_IBM-938IBM_eucTW to IBM_938
IBM-eucCN_big5IBM-eucCN to big5

来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/7343861/viewspace-888299/,如需转载,请注明出处,否则将追究法律责任。

转载于:http://blog.itpub.net/7343861/viewspace-888299/

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值