Python之fileinput

fileinput module
fileinput module可以对一个或多个文件中的内容进行迭代、遍历等操作。该模块的input()函数有点类似文件readlines()方法。
区别在于前者是一个迭代对象(iterable object)，需要用for循环迭代，后者是一次性读取所有行。

用fileinput对文件进行循环遍历，格式化输出，查找、替换等操作，非常方便。

1. 典型用法

import fileinput
for line in fileinput.input('D:\\HeadFirstPython\\chapter1\\data.txt'):
    print(line, end = "")

似乎比 with open(filename) as f: for i in f: print(i) 这种要好一些

2. FUNCTIONS

close()
    Close the sequence.
filelineno()
    Return the line number in the current file. Before the first line
    has been read, returns 0. After the last line of the last file has
    been read, returns the line number of that line within the file.
返回当前读取的文件内容的行数
filename()
    Return the name of the file currently being read.
    Before the first line has been read, returns None.
input(files=None, inplace=False, backup='', bufsize=0, mode='r', openhook=None)
    Return an instance of the FileInput class, which can be iterated.    
    The parameters are passed to the constructor of the FileInput class.
    The returned instance, in addition to being an iterator,
    keeps global state for the functions of this module,.
files:                  #文件的路径列表，默认是stdin方式，多文件['1.txt','2.txt',...]
inplace:                #是否将标准输出的结果写回文件，默认不取代，即不写入文件
backup:                 #备份文件的扩展名，只指定扩展名，如.bak。如果该文件的备份文件已存在，则会自动覆盖。
bufsize:                #缓冲区大小，默认为0，如果文件很大，可以修改此参数，一般默认即可
mode:                   #读写模式，默认为只读
openhook:               #该钩子用于控制打开的所有文件，比如说编码方式等;
isfirstline()
    Returns true the line just read is the first line of its file,
    otherwise returns false.
注：主要是当一次性读取多个文件时，区分不同的文件

isstdin()
    Returns true if the last line was read from sys.stdin,
    otherwise returns false.
判断最后一行是否从stdin中读取,stdin指标准输入, 默认是从键盘输入
lineno()
    Return the cumulative line number of the line that has just been read.
    Before the first line has been read, returns 0. After the last line
    of the last file has been read, returns the line number of that line.
返回当前已经读取的行的数量(或者序号), 是累计的行数
nextfile()
    Close the current file so that the next iteration will read the first
    line from the next file (if any); lines not read from the file will
    not count towards the cumulative line count. The filename is not
    changed until after the first line of the next file has been read.
    Before the first line has been read, this function has no effect;
    it cannot be used to skip the first file. After the last line of the
    last file has been read, this function has no effect.

3. 常见例子
3.1 利用fileinput读取一个文件所有行

import fileinput
with fileinput.input('D:\\HeadFirstPython\\chapter1\\data.txt') as f:
    for line in f:
        print(fileinput.filename(),'|','Line Number:',fileinput.filelineno(),'|: ',line, end = "")

output:

D:\HeadFirstPython\chapter1\data.txt | Line Number: 1 |:  Perl
D:\HeadFirstPython\chapter1\data.txt | Line Number: 2 |:  Java
D:\HeadFirstPython\chapter1\data.txt | Line Number: 3 |:  C/C++
D:\HeadFirstPython\chapter1\data.txt | Line Number: 4 |:  Shell

命令行方式：

#test.py
import fileinput
for line in fileinput.input():
    print(fileinput.filename(),'|','Line Number:',fileinput.lineno(),'|: ',line, end = "")

output:

D:\HeadFirstPython\chapter1>python.exe test.py data.txt
data.txt | Line Number: 1 |: Python
data.txt | Line Number: 2 |: Java
data.txt | Line Number: 3 |: C/C++
data.txt | Line Number: 4 |: Shell

3.2 利用fileinput对多文件操作，并原地修改内容
3.2.1 文件内容：

#test02.py
import fileinput
def process(line):
    return(line.rstrip() + " line")

file_names = ['D:\\HeadFirstPython\\chapter1\\1.txt', 'D:\\HeadFirstPython\\chapter1\\2.txt']
for line in fileinput.input(file_names, inplace = 1):
    print(process(line))
直接F5, 或者进入命令行
D:\HeadFirstPython\chapter1>python.exe test02.py

得到结果是1.txt/2.txt中变成如下：

1.txt
first line
second line

2.txt
third line 
fourth line 
注：如果inpalce = False，则不会向1.txt/2.txt中插入，而会直接打印到屏幕

3.2.2 命令行方式：

#test03.py
import fileinput
def process(line):
    return(line.rstrip() + ' line') 
for line in fileinput.input(inplace = True):
    print(process(line))
#执行命令：
D:\HeadFirstPython\chapter1>python.exe test03.py 1.txt 2.txt

3.3 利用fileinput实现文件内容替换，并将原文件作备份

#test04.py
import fileinput
for line in fileinput.input("D:\\HeadFirstPython\\chapter1\\data.txt", backup = ".bak", inplace = 1):
    print(line.rstrip().replace("Python", "Perl"))

output:

Perl
Java
C/C++
Shell

3.4 利用fileinput对文件简单处理

#FileName: test.py
import sys
import fileinput

for line in fileinput.input('150827.txt'):
    sys.stdout.write('=> ')
    sys.stdout.write(line)
#注：print是对sys.stout.write的友好封装, 并且print可以指定sep，但是sys.stout.write也有它自己的优势

3.5 利用fileinput批处理文件

import glob
import fileinput
a = glob.iglob('15*.txt')
for line in fileinput.input(a):
    if fileinput.isfirstline():
        print('-'*20, 'Reading %s...' % fileinput.filename(), '-'*20)
    print(str(fileinput.lineno()) + ': ' + line.upper())
#备注：glob中就两个知识点：glob和iglob，其中iglob把结果变成生成器iterator

3.6 利用fileinput及re做日志分析: 提取所有含日期的行

import re
import fileinput
import sys
 
pattern = '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'
#pattern = '1970-01-01 13:45:30'
for line in fileinput.input('error.log', backup = '.bak', inplace = False):
    if re.findall(pattern,line):
        sys.stdout.write('=>')
        sys.stdout.write(line)

测试结果：

=>1970-01-01 13:45:30  Error: **** Due to System1 Disk spacke not enough...
=>1970-01-02 13:45:30  Error: **** Due to System2 Disk spacke not enough...
=>1970-01-03 10:20:30  Error: **** Due to System3 Out of Memory...
=>1970-01-04 10:20:30  Error: **** Due to System4 Out of Memory...

3.7 利用fileinput及re做分析: 提取符合条件的电话号码

#---样本文件: phone.txt---
010-110-12345
800-333-1234
010-99999999
05718888888
021-88888888

#---测试脚本: test.py---
import re
import fileinput
 
pattern = '[010|021]-\d{8}'  #提取区号为010或021电话号码，格式:010-12345678
#pattern = '010-110-12345'
for line in fileinput.input('phone.txt'):
    if re.findall(pattern,line):
        print('=' * 50)
        print('Filename:'+ fileinput.filename()
            + ' | Line Number:'+str(fileinput.lineno())
            + ' | '+ line, end = '')

output:

==================================================
Filename:phone.txt | Line Number:3 | 010-99999999
==================================================
Filename:phone.txt | Line Number:5 | 021-88888888
#注：print中, '+' 和 ',' 都可以, 但是','打印出来一个空格, 而'+'则没有

3.8 csv和fileinput比较：

import csv,fileinput

with open("app_dim_pop_phy_vender.csv") as csvfile:
    reader = csv.reader(csvfile)
    #print(reader)
    for row in reader:
        vender_id,vender_name,cx_dept_id,company_id,company_name = row
        #print(vender_id + '\t' + vender_name + '\t' + cx_dept_id + '\t' + company_id + '\t' + company_name)

for row in fileinput.input("app_dim_pop_phy_vender.csv"):
    vender_id,vender_name,cx_dept_id,company_id,company_name = row.split(',')
    print(vender_id + '\t' + vender_name + '\t' + cx_dept_id + '\t' + company_id + '\t' + company_name)