抓数据+代理

网络爬虫技术学习，更多知识请访问https://www.itkc8.com

测试代码如下
public static void proxy() throws IOException {
URL url = new URL("http://www.test.com/");
InetSocketAddress address = new InetSocketAddress("x.x.x.x", 8888);
Proxy proxy = new Proxy(Proxy.Type.HTTP, address);
Authenticator.setDefault(new MyAuthenticator(new String("root".getBytes("UTF-8")), new String("password".getBytes("UTF-8"))));
HttpURLConnection connection = (HttpURLConnection) url.openConnection(proxy);
InputStreamReader in = null;
BufferedReader reader = null;
try {
in = new InputStreamReader(connection.getInputStream());
reader = new BufferedReader(in);
StringBuffer buffer = new StringBuffer();
String line;
while ((line = reader.readLine()) != null) {
buffer.append(line);
}
System.out.println(buffer);
} finally {
if (in != null) {
in.close();
}
if (reader != null) {
reader.close();
}
}
}
如果不使用代理，一切正常