java 模拟登录解析html值,使用Java登录后解析HTML源

I've been trying to access a website to parse data for an Android application I am developing, but I am having no luck when it comes to logging in.

And below is a stripped out version of the form from that page (HTML):

Login

Can anyone please suggest how I can login to this website using Java then parse the redirected page?

Up to now, I've tried processes on the lines of:

public static void main(Context context) {

try {

// Construct data

String data = URLEncoder.encode("nickname", "UTF-8") + "=" + URLEncoder.encode("testingA", "UTF-8");

data += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode("testing", "UTF-8");

// Send data

URL url = new URL("https://giffgaff.com/mobile/login");

URLConnection conn = url.openConnection();

conn.setDoOutput(true);

OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());

wr.write(data);

wr.flush();

// Get the response

BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));

String str = "";

String line;

while ((line = rd.readLine()) != null) {

str += line;

}

AlertDialog alertDialog = new AlertDialog.Builder(context).create();

alertDialog.setTitle("Output");

alertDialog.setMessage(str);

alertDialog.setButton("Okay", new DialogInterface.OnClickListener() {

public void onClick(DialogInterface dialog, int which) {

}

});

alertDialog.show();

wr.close();

rd.close();

} catch (Exception e) {

AlertDialog alertDialog = new AlertDialog.Builder(context).create();

alertDialog.setTitle("ERROR");

alertDialog.setMessage(e.toString());

alertDialog.setButton("Okay", new DialogInterface.OnClickListener() {

public void onClick(DialogInterface dialog, int which) {

}

});

alertDialog.show();

}

}

But my attempts return the page as if the login information was incorrect.

If you would like to see for yourself how the login page behaves, here's some test login details:

Nickname (username): testingA

Password: testing

The site also seems to depend on a Cookie called "napaSessionId"

解决方案

First a word of caution, if you don't have direct permission to do this, beware, the site in question may preclude this in their terms of service.

To answer the question, there are many, many reasons a site would reject a login. To do this successfully you need to get as close as possible to how a browser would handle the transaction. To do that you need to see what a real browser is doing.

https is more tricky as many http sniffers can't deal with it but httpwatch claims it can. Check out the HTTP transactions and then try to replicate them.

Your url.openConnection() call will actually return an instance of HTTPURLConnction, cast to that & then you'll be able to easily set various http headers such as the User-Agent.

A final note, you say a cookie may be required. Your code isn't going to deal with cookies. To do that you'll need to use a cookie manager, e.g.: http://download.oracle.com/javase/tutorial/networking/cookies/index.html