本博文实现了教务系统自动化,可以自动爬取成绩,其他功能也是一样的道理
首先传入登录界面的URL,http://bkjw.guet.edu.cn/student/public/login.asp?
driver.get("http://bkjw.guet.edu.cn/student/public/login.asp")
打开开发者工具找到用户名和口令还有提交按钮元素,
username = WebDriverWait(driver, 10, 0.5).until(EC.presence_of_element_located((By.NAME , "username")))
username.send_keys("1500130238")
password = driver.find_element_by_name("passwd")
password.send_keys("xxxx")
然后模拟点击按钮
button = driver.find_element_by_name("login")
button.click()
这样我们已经登录成功了
这里可以看到有三个frame,分别是左边栏,中间栏,还有底部栏,如果直接找成绩查询这个按钮时就会提示找不到该元素,这里就可以切换frame来操作元素,切换到左边栏然后,通过XPATH找到成绩查询这个按钮并且点击
driver.switch_to.frame("contents")
driver.find_element(By.XPATH, "//*[@id='menubg']/nobr/li[14]/a/div").click()
切换到中间栏然后选择查询的学期,然后点击
driver.switch_to.parent_frame()
driver.switch_to.frame("main")
chaxun = driver.find_element(By.XPATH, "/html/body/form/div/input[2]").click()
最后可以将页面截图保存下来,
driver.get_screenshot_as_file("C:\\Users\\Administrator\\Downloads\\chengji.jpg")
最终版代码是这样的
# -*- coding:utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
driver = webdriver.Chrome()
# driver.implicitly_wait(10)
driver.get("http://bkjw.guet.edu.cn/student/public/login.asp")
username = WebDriverWait(driver, 10, 0.5).until(EC.presence_of_element_located((By.NAME , "username")))
username.send_keys("1500130238")
password = driver.find_element_by_name("passwd")
password.send_keys("xxxx")
button = driver.find_element_by_name("login")
button.click()
driver.switch_to.frame("contents")
driver.find_element(By.XPATH, "//*[@id='menubg']/nobr/li[14]/a/div").click()
driver.switch_to.parent_frame()
driver.switch_to.frame("main")
chaxun = driver.find_element(By.XPATH, "/html/body/form/div/input[2]").click()
driver.get_screenshot_as_file("C:\\Users\\Administrator\\Downloads\\aa.jpg")
driver.quit()
最后将页面做一下处理,利用beautifulsoap将有用的数据提取出来,并且打印
for child in soup.tr.children:
print child.string
for child in soup.tr.next_sibling.next_sibling:
print child.string
x = 1
while x < 24:
soup.tr.next_sibling.next_sibling = soup.tr.next_sibling.next_sibling.next_sibling.next_sibling
for child in soup.tr.next_sibling.next_sibling:
print child.string
x += 1