How do you read multiple lines containing say 3 URLs from a .txt file, "curl" them to extract the HTML, and output them all to their own individual files in python?
I've tried-
import commands
import os
import json
# RAW DATA
input = open('uri.txt', 'r')
t_1 = open('command', 'w')
counter_1 = 0
for line in input:
counter_1 += 1
if counter_1 < 3:
filename = str(counter_1)
print str(line)
filename= str(count)
command ='curl ' + '"' + str(line).rstrip('\n') + '"'+ '> ./rawData/' + filename
output_1 = commands.getoutput(command)
input.close()
When I run my code in PyCharm, it creates a single blank file in my directory called "filename.txt"
解决方案
You should use requests instead of cURL from within python, as this will make handling the responses much easier. Also, the canonical way to work with files is to use the context manager: with open(filename) as f.