我正在尝试基于该网页的网页报废来构建数据框
html firstable我说selenium点击我选择的页面然后我把xpath和标签元素用于构建标题和正文但我没有我想要的格式是NaN或重复的格式.
按照我的脚本:
def get_browser(url_selector):
"""Get the browser (a "driver")."""
#option = webdriver.ChromeOptions()
#option.add_argument(' — incognito')
path_to_chromedriver = r"C:/Users/xxxxx/Downloads/chromedriver_win32/chromedriver.exe"
browser = webdriver.Chrome(executable_path= path_to_chromedriver)
browser.get(url_selector)
""" Try with Italie"""
browser.find_element_by_xpath(italie_buton_xpath).click()
""" Raise exception : down browser if loading take more than 45sec : timer is the logo website as a flag"""
# Wait 45 seconds for page to load
timeout = 45
try:
WebDriverWait(browser, timeout).until(EC.visibility_of_element_located((By.XPATH, '//*[@id="s5_logo_wrap"]/img')))
except TimeoutException:
print("Timed out waiting for page to load")
browser.quit()
return browser
browser = get_browser(url_selector)
headers = browser.find_element_by_xpath('//*[@id="s5_component_wrap_inner"]/main/div[2]/div[2]/div[3]/table/thead').find_elements_by_tag_name('tr')
headings = [i.text.strip() for i in headers]
bs_obj = BeautifulSoup(browser.page_source, 'html.parser')
rows = bs_obj.find_all('table')[0].find('tbody').find_all('tr')[1:]
table = []
for row in rows :
line = next(td.get_text() for td in row.find_all("td"))
print(line)
table.append(line)
browser.quit()
pd.DataFrame(line, columns = headings)
它返回
一列数据框如:
School Holiday Region Start date End date Week
0 Easter holidays 2018
1 REMARK: Small differences by region are possi...
2 Summer holiday 2018
3 REMARK: First region through to last region.
4 Christmas holiday 2018
有三个问题,我不希望REMARK行和学校假期开始日期和结束日期被视为单独的单词,整个数据框是未分割的.
如果我分裂我的标题并排列两个不匹配的形状
由于REMARKS行,我的列表中有9个元素而不是3个,由于分隔的单词,我在标题中得到8个元素而不是5个元素.