java 中文url转码_对 url 中含有的中文进行转码操作

对 url 中含有的中文进行转码操作

一般情况下，将带有中文的 url 拷贝到开发工具，开发工具都会有相应的转码(自动转码)，

现在大部分的浏览器也可以对含有中文的 url 进行转码(自动转码)

情景说明

android

安卓系统，能够识别中文路径：

https://spdbimg.idoupiao.com/film/my/美国队长3

IOS

苹果系统，不能够识别中文路径：

https://spdbimg.idoupiao.com/film/my/美国队长3

必须将以上路径进行转码为下来形式才可以访问：

https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

这和他们各自的战略定位有关系。。。

编码实现

1、先寻找有没有已经实现类似功能的成熟工具。

2、找到 JDK 提供的 URLEncoder 类的 encode 方法基本满足需求。

3、测试是否满足当下需求。

经过测试发现，JDK 提供的 URLEncoder 类的 encode 方法会将 ':' 和 '/' 等字符也进行编码。

4、查找 JDK 提供的 URLEncoder 类的 encode 方法 相关源码。

5、发现可以进行部分改造满足当前需求，重写 URLEncoder 类，并将其设置为私有静态类，多人合作时，防止他人误用。

6、改造完成进行验证型测试。

添加以下两条语句：

// 排除这两个特殊字符的编码

dontNeedEncoding.set(':');

dontNeedEncoding.set('/');

重写后的 URLEncoder 类：

/**

* Utility class for HTML form encoding. This class contains static methods

* for converting a String to the application/x-www-form-urlencoded MIME

* format. For more information about HTML form encoding, consult the HTML

* specification.

*

*

* When encoding a String, the following rules apply:

*

*

*

The alphanumeric characters "{@code a}" through

* "{@code z}", "{@code A}" through

* "{@code Z}" and "{@code 0}"

* through "{@code 9}" remain the same.

*

The special characters "{@code .}",

* "{@code -}", "{@code *}", and

* "{@code _}" remain the same.

*

The space character "   " is

* converted into a plus sign "{@code +}".

*

All other characters are unsafe and are first converted into

* one or more bytes using some encoding scheme. Then each byte is

* represented by the 3-character string

* "{@code %xy}", where xy is the

* two-digit hexadecimal representation of the byte.

* The recommended encoding scheme to use is UTF-8. However,

* for compatibility reasons, if an encoding is not specified,

* then the default encoding of the platform is used.

*

*

*

* For example using UTF-8 as the encoding scheme the string "The

* string ü@foo-bar" would get converted to

* "The+string+%C3%BC%40foo-bar" because in UTF-8 the character

* ü is encoded as two bytes C3 (hex) and BC (hex), and the

* character @ is encoded as one byte 40 (hex).

*

* @author Herb Jellinek

* @since JDK1.0

*/

private static class URLEncoder {

static BitSet dontNeedEncoding;

static final int caseDiff = ('a' - 'A');

static String dfltEncName = null;

static {

/* The list of characters that are not encoded has been

* determined as follows:

*

* RFC 2396 states:

* -----

* Data characters that are allowed in a URI but do not have a

* reserved purpose are called unreserved. These include upper

* and lower case letters, decimal digits, and a limited set of

* punctuation marks and symbols.

*

* unreserved = alphanum | mark

*

* mark = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"

*

* Unreserved characters can be escaped without changing the

* semantics of the URI, but this should not be done unless the

* URI is being used in a context that does not allow the

* unescaped character to appear.

* -----

*

* It appears that both Netscape and Internet Explorer escape

* all special characters from this list with the exception

* of "-", "_", ".", "*". While it is not clear why they are

* escaping the other characters, perhaps it is safest to

* assume that there might be contexts in which the others

* are unsafe if not escaped. Therefore, we will use the same

* list. It is also noteworthy that this is consistent with

* O'Reilly's "HTML: The Definitive Guide" (page 164).

*

* As a last note, Intenet Explorer does not encode the "@"

* character which is clearly not unreserved according to the

* RFC. We are being consistent with the RFC in this matter,

* as is Netscape.

*

*/

dontNeedEncoding = new BitSet(256);

int i;

for (i = 'a'; i <= 'z'; i++) {

dontNeedEncoding.set(i);

}

for (i = 'A'; i <= 'Z'; i++) {

dontNeedEncoding.set(i);

}

for (i = '0'; i <= '9'; i++) {

dontNeedEncoding.set(i);

}

dontNeedEncoding.set(' '); /* encoding a space to a + is done

* in the encode() method */

dontNeedEncoding.set('-');

dontNeedEncoding.set('_');

dontNeedEncoding.set('.');

dontNeedEncoding.set('*');

// 排除这两个特殊字符的编码

dontNeedEncoding.set(':');

dontNeedEncoding.set('/');

dfltEncName = AccessController.doPrivileged(

new GetPropertyAction("file.encoding")

);

}

/**

* You can't call the constructor.

*/

private URLEncoder() { }

/**

* Translates a string into {@code x-www-form-urlencoded}

* format. This method uses the platform's default encoding

* as the encoding scheme to obtain the bytes for unsafe characters.

*

* @param s {@code String} to be translated.

* @deprecated The resulting string may vary depending on the platform's

* default encoding. Instead, use the encode(String,String)

* method to specify the encoding.

* @return the translated {@code String}.

*/

@Deprecated

public static String encode(String s) {

String str = null;

try {

str = encode(s, dfltEncName);

} catch (UnsupportedEncodingException e) {

// The system should always have the platform default

}

return str;

}

/**

* Translates a string into {@code application/x-www-form-urlencoded}

* format using a specific encoding scheme. This method uses the

* supplied encoding scheme to obtain the bytes for unsafe

* characters.

*

* Note: The

* "http://www.w3.org/TR/html40/appendix/notes.html#non-ascii-chars">

* World Wide Web Consortium Recommendation

states that

* UTF-8 should be used. Not doing so may introduce

* incompatibilities.

*

* @param s {@code String} to be translated.

* @param enc The name of a supported

* character

* encoding.

* @return the translated {@code String}.

* @exception UnsupportedEncodingException

* If the named encoding is not supported

* @see URLDecoder#decode(java.lang.String, java.lang.String)

* @since 1.4

*/

public static String encode(String s, String enc)

throws UnsupportedEncodingException {

boolean needToChange = false;

StringBuffer out = new StringBuffer(s.length());

Charset charset;

CharArrayWriter charArrayWriter = new CharArrayWriter();

if (enc == null)

throw new NullPointerException("charsetName");

try {

charset = Charset.forName(enc);

} catch (IllegalCharsetNameException e) {

throw new UnsupportedEncodingException(enc);

} catch (UnsupportedCharsetException e) {

throw new UnsupportedEncodingException(enc);

}

for (int i = 0; i < s.length();) {

int c = (int) s.charAt(i);

//System.out.println("Examining character: " + c);

if (dontNeedEncoding.get(c)) {

if (c == ' ') {

c = '+';

needToChange = true;

}

//System.out.println("Storing: " + c);

out.append((char)c);

i++;

} else {

// convert to external encoding before hex conversion

do {

charArrayWriter.write(c);

/*

* If this character represents the start of a Unicode

* surrogate pair, then pass in two characters. It's not

* clear what should be done if a bytes reserved in the

* surrogate pairs range occurs outside of a legal

* surrogate pair. For now, just treat it as if it were

* any other character.

*/

if (c >= 0xD800 && c <= 0xDBFF) {

/*

System.out.println(Integer.toHexString(c)

+ " is high surrogate");

*/

if ( (i+1) < s.length()) {

int d = (int) s.charAt(i+1);

/*

System.out.println("\tExamining "

+ Integer.toHexString(d));

*/

if (d >= 0xDC00 && d <= 0xDFFF) {

/*

System.out.println("\t"

+ Integer.toHexString(d)

+ " is low surrogate");

*/

charArrayWriter.write(d);

i++;

}

}

}

i++;

} while (i < s.length() && !dontNeedEncoding.get((c = (int) s.charAt(i))));

charArrayWriter.flush();

String str = new String(charArrayWriter.toCharArray());

byte[] ba = str.getBytes(charset);

for (int j = 0; j < ba.length; j++) {

out.append('%');

char ch = Character.forDigit((ba[j] >> 4) & 0xF, 16);

// converting to use uppercase letter as part of

// the hex value if ch is a letter.

if (Character.isLetter(ch)) {

ch -= caseDiff;

}

out.append(ch);

ch = Character.forDigit(ba[j] & 0xF, 16);

if (Character.isLetter(ch)) {

ch -= caseDiff;

}

out.append(ch);

}

charArrayWriter.reset();

needToChange = true;

}

}

return (needToChange? out.toString() : s);

}

}

测试代码：

@Test

public void testReplace() throws UnsupportedEncodingException {

// https://ojjqzz83h.qnssl.com/film/my/美国队长3?imageView2/1/w/140/h/90/interlace/0/q/100

// https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

String oldImgUrl = "https://spdbimg.idoupiao.com/film/my/美国队长3";

oldImgUrl = URLEncoder.encode(oldImgUrl, "UTF-8");

// oldImgUrl = new String(Base64.getEncoder().encode(oldImgUrl.getBytes()));

String oldDomain = "spdbimg.idoupiao.com";

String newDomain = "ojjqzz83h.qnssl.com";

String suffix = "?imageView2/1/w/140/h/90/interlace/0/q/100";

String newImgUrl = StringUtils.replace(oldImgUrl, oldDomain, newDomain).concat(suffix);

System.out.println("newImgUrl === " + newImgUrl);

}

完整代码如下：

package com.github.ljmatlight.util;

import org.junit.Test;

import sun.security.action.GetPropertyAction;

import java.io.CharArrayWriter;

import java.io.UnsupportedEncodingException;

import java.net.URLDecoder;

import java.nio.charset.Charset;

import java.nio.charset.IllegalCharsetNameException;

import java.nio.charset.UnsupportedCharsetException;

import java.security.AccessController;

import java.util.BitSet;

/**

* Description：对 url 中含有的中文进行转码操作

*
Author：ljmatlight

*/

public class StringTest {

@Test

public void testReplace() throws UnsupportedEncodingException {

// https://ojjqzz83h.qnssl.com/film/my/美国队长3?imageView2/1/w/140/h/90/interlace/0/q/100

// https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

String oldImgUrl = "https://spdbimg.idoupiao.com/film/my/美国队长3";

oldImgUrl = URLEncoder.encode(oldImgUrl, "UTF-8");

// oldImgUrl = new String(Base64.getEncoder().encode(oldImgUrl.getBytes()));

String oldDomain = "spdbimg.idoupiao.com";

String newDomain = "ojjqzz83h.qnssl.com";

String suffix = "?imageView2/1/w/140/h/90/interlace/0/q/100";

String newImgUrl = StringUtils.replace(oldImgUrl, oldDomain, newDomain).concat(suffix);

System.out.println("newImgUrl === " + newImgUrl);

}

/**

* Utility class for HTML form encoding. This class contains static methods

* for converting a String to the application/x-www-form-urlencoded MIME

* format. For more information about HTML form encoding, consult the HTML

* specification.

*

*

* When encoding a String, the following rules apply:

*

*

*

The alphanumeric characters "{@code a}" through

* "{@code z}", "{@code A}" through

* "{@code Z}" and "{@code 0}"

* through "{@code 9}" remain the same.

*

The special characters "{@code .}",

* "{@code -}", "{@code *}", and

* "{@code _}" remain the same.

*

The space character "   " is

* converted into a plus sign "{@code +}".

*

All other characters are unsafe and are first converted into

* one or more bytes using some encoding scheme. Then each byte is

* represented by the 3-character string

* "{@code %xy}", where xy is the

* two-digit hexadecimal representation of the byte.

* The recommended encoding scheme to use is UTF-8. However,

* for compatibility reasons, if an encoding is not specified,

* then the default encoding of the platform is used.

*

*

*

* For example using UTF-8 as the encoding scheme the string "The

* string ü@foo-bar" would get converted to

* "The+string+%C3%BC%40foo-bar" because in UTF-8 the character

* ü is encoded as two bytes C3 (hex) and BC (hex), and the

* character @ is encoded as one byte 40 (hex).

*

* @author Herb Jellinek

* @since JDK1.0

*/

private static class URLEncoder {

static BitSet dontNeedEncoding;

static final int caseDiff = ('a' - 'A');

static String dfltEncName = null;

static {

/* The list of characters that are not encoded has been

* determined as follows:

*

* RFC 2396 states:

* -----

* Data characters that are allowed in a URI but do not have a

* reserved purpose are called unreserved. These include upper

* and lower case letters, decimal digits, and a limited set of

* punctuation marks and symbols.

*

* unreserved = alphanum | mark

*

* mark = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"

*

* Unreserved characters can be escaped without changing the

* semantics of the URI, but this should not be done unless the

* URI is being used in a context that does not allow the

* unescaped character to appear.

* -----

*

* It appears that both Netscape and Internet Explorer escape

* all special characters from this list with the exception

* of "-", "_", ".", "*". While it is not clear why they are

* escaping the other characters, perhaps it is safest to

* assume that there might be contexts in which the others

* are unsafe if not escaped. Therefore, we will use the same

* list. It is also noteworthy that this is consistent with

* O'Reilly's "HTML: The Definitive Guide" (page 164).

*

* As a last note, Intenet Explorer does not encode the "@"

* character which is clearly not unreserved according to the

* RFC. We are being consistent with the RFC in this matter,

* as is Netscape.

*

*/

dontNeedEncoding = new BitSet(256);

int i;

for (i = 'a'; i <= 'z'; i++) {

dontNeedEncoding.set(i);

}

for (i = 'A'; i <= 'Z'; i++) {

dontNeedEncoding.set(i);

}

for (i = '0'; i <= '9'; i++) {

dontNeedEncoding.set(i);

}

dontNeedEncoding.set(' '); /* encoding a space to a + is done

* in the encode() method */

dontNeedEncoding.set('-');

dontNeedEncoding.set('_');

dontNeedEncoding.set('.');

dontNeedEncoding.set('*');

// 排除这两个特殊字符的编码

dontNeedEncoding.set(':');

dontNeedEncoding.set('/');

dfltEncName = AccessController.doPrivileged(

new GetPropertyAction("file.encoding")

);

}

/**

* You can't call the constructor.

*/

private URLEncoder() { }

/**

* Translates a string into {@code x-www-form-urlencoded}

* format. This method uses the platform's default encoding

* as the encoding scheme to obtain the bytes for unsafe characters.

*

* @param s {@code String} to be translated.

* @deprecated The resulting string may vary depending on the platform's

* default encoding. Instead, use the encode(String,String)

* method to specify the encoding.

* @return the translated {@code String}.

*/

@Deprecated

public static String encode(String s) {

String str = null;

try {

str = encode(s, dfltEncName);

} catch (UnsupportedEncodingException e) {

// The system should always have the platform default

}

return str;

}

/**

* Translates a string into {@code application/x-www-form-urlencoded}

* format using a specific encoding scheme. This method uses the

* supplied encoding scheme to obtain the bytes for unsafe

* characters.

*

* Note: The

* "http://www.w3.org/TR/html40/appendix/notes.html#non-ascii-chars">

* World Wide Web Consortium Recommendation

states that

* UTF-8 should be used. Not doing so may introduce

* incompatibilities.

*

* @param s {@code String} to be translated.

* @param enc The name of a supported

* character

* encoding.

* @return the translated {@code String}.

* @exception UnsupportedEncodingException

* If the named encoding is not supported

* @see URLDecoder#decode(java.lang.String, java.lang.String)

* @since 1.4

*/

public static String encode(String s, String enc)

throws UnsupportedEncodingException {

boolean needToChange = false;

StringBuffer out = new StringBuffer(s.length());

Charset charset;

CharArrayWriter charArrayWriter = new CharArrayWriter();

if (enc == null)

throw new NullPointerException("charsetName");

try {

charset = Charset.forName(enc);

} catch (IllegalCharsetNameException e) {

throw new UnsupportedEncodingException(enc);

} catch (UnsupportedCharsetException e) {

throw new UnsupportedEncodingException(enc);

}

for (int i = 0; i < s.length();) {

int c = (int) s.charAt(i);

//System.out.println("Examining character: " + c);

if (dontNeedEncoding.get(c)) {

if (c == ' ') {

c = '+';

needToChange = true;

}

//System.out.println("Storing: " + c);

out.append((char)c);

i++;

} else {

// convert to external encoding before hex conversion

do {

charArrayWriter.write(c);

/*

* If this character represents the start of a Unicode

* surrogate pair, then pass in two characters. It's not

* clear what should be done if a bytes reserved in the

* surrogate pairs range occurs outside of a legal

* surrogate pair. For now, just treat it as if it were

* any other character.

*/

if (c >= 0xD800 && c <= 0xDBFF) {

/*

System.out.println(Integer.toHexString(c)

+ " is high surrogate");

*/

if ( (i+1) < s.length()) {

int d = (int) s.charAt(i+1);

/*

System.out.println("\tExamining "

+ Integer.toHexString(d));

*/

if (d >= 0xDC00 && d <= 0xDFFF) {

/*

System.out.println("\t"

+ Integer.toHexString(d)

+ " is low surrogate");

*/

charArrayWriter.write(d);

i++;

}

}

}

i++;

} while (i < s.length() && !dontNeedEncoding.get((c = (int) s.charAt(i))));

charArrayWriter.flush();

String str = new String(charArrayWriter.toCharArray());

byte[] ba = str.getBytes(charset);

for (int j = 0; j < ba.length; j++) {

out.append('%');

char ch = Character.forDigit((ba[j] >> 4) & 0xF, 16);

// converting to use uppercase letter as part of

// the hex value if ch is a letter.

if (Character.isLetter(ch)) {

ch -= caseDiff;

}

out.append(ch);

ch = Character.forDigit(ba[j] & 0xF, 16);

if (Character.isLetter(ch)) {

ch -= caseDiff;

}

out.append(ch);

}

charArrayWriter.reset();

needToChange = true;

}

}

return (needToChange? out.toString() : s);

}

}

}

作者：随风浮云

出处：http://www.cnblogs.com/ljmatlight

本文版权归作者所有，欢迎转载，但未经作者同意必须保留此段声明。

文中有不妥或者错误的地方，欢迎勘误，如果你有更好的建议，可以给我留言讨论，共同进步。

互联网技术时效性较强，引用请慎重。