As a newbie in Python (2.7) I`m looking for next suggestion:
I have a csv file with stored http links in one column comma delimited.
http://example.com/file.pdf,
http://example.com/file.xls,
http://example.com/file.xlsx,
http://example.com/file.doc,
The main aim is to loop through all these links and download files by them in original extention and name.
So my search results and help here gave me next script:
import urllib2
import pandas as pd
links = pd.read_csv('links.csv', sep=',', header =(0))
url = links # I know this part wrong by don`n know how to do right
user_agent = 'Mozilla 5.0 (Windows 7; Win64; x64)'
file_name = "tessst" # here the files name by how to get their original names
u = urllib2.Request(url, headers = {'User-Agent' : user_agent})
req = urllib2.urlopen(u)
f = open(file_name, 'wb')
f.write(req.read())
f.close()
please any help
P S not sure about pandas - maybe csv better?
解决方案
If I can assume your CSV file to be one column only, containing links then this would work .
import csv, sys
import requests
import urllib2
import os
filename = 'test.csv'
with open(filename, 'rb') as f:
reader = csv.reader(f)
try:
for row in reader:
if 'http' in row[0]:
#print row
rev = row[0][::-1]
i = rev.index('/')
tmp = rev[0:i]
#print tmp[::-1]
rq = urllib2.Request(row[0])
res = urllib2.urlopen(rq)
if not os.path.exists("./"+tmp[::-1]):
pdf = open("./" + tmp[::-1], 'wb')
pdf.write(res.read())
pdf.close()
else:
print "file: ", tmp[::-1], "already exist"
except csv.Error as e:
sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e))