python中beautifulsoup怎么输出文本内容,在Python中使用BeautifulSoup从脚本标签中提取文本...

Could you please help me with this lil thing. I am looking to extract email, phone and name value from the below code in SCRIPT tag(not in Body) using Beautiful soup(Python). I am new to Python and blog are recommending to use Beautiful soup for extracting.

I tried getting page using the following code -

fileDetails = BeautifulSoup(urllib2.urlopen('http://www.example.com').read())

results = fileDetails.find(email:")

This Ajax request code is not repeating in the page again. Can we also write try and catch so that if it doesn't found it in the page, it won't throw any error.

$(document).ready( function (){

$('#message').click(function(){

alert();

});

$('#addmessage').click(function(){

$.ajax({

type: "POST",

url: 'http://www.example.com',

data: {

email: 'abc@g.com',

phone: '9999999999',

name: 'XYZ'

}

});

});

});

Once I get this, I also want to store in an excel file.

Thanks in anticipation.

解决方案

Alternatively to the regex-based approach, you can parse the javascript code using slimit module, that builds an Abstract Syntax Tree and gives you a way of getting all assignments and putting them into the dictionary:

from bs4 import BeautifulSoup

from slimit import ast

from slimit.parser import Parser

from slimit.visitors import nodevisitor

data = """

My Sample Page

$.ajax({

type: "POST",

url: 'http://www.example.com',

data: {

email: 'abc@g.com',

phone: '9999999999',

name: 'XYZ'

}

});

What a wonderful world

"""

# get the script tag contents from the html

soup = BeautifulSoup(data)

script = soup.find('script')

# parse js

parser = Parser()

tree = parser.parse(script.text)

fields = {getattr(node.left, 'value', ''): getattr(node.right, 'value', '')

for node in nodevisitor.visit(tree)

if isinstance(node, ast.Assign)}

print fields

Prints:

{u'name': u"'XYZ'", u'url': u"'http://www.example.com'", u'type': u'"POST"', u'phone': u"'9999999999'", u'data': '', u'email': u"'abc@g.com'"}

Among other fields, there are email, name and phone that you are interested in.

Hope that helps.