提取指定的PDF表格保存到Excel

最新推荐文章于 2024-07-12 13:04:58 发布
a useful man
最新推荐文章于 2024-07-12 13:04:58 发布
阅读量473
点赞数
分类专栏： python机器学习
本文链接：https://blog.csdn.net/sinat_23971513/article/details/109328059
版权
本文介绍了一个程序，用于从PDF中提取特定表格并保存到Excel。程序根据指定的规则，利用PDF解析和Excel读写模块，通过多进程加速处理。
摘要由CSDN通过智能技术生成
摘要：本文介绍一个提取PDF中的表格内容的程序。首先，程序给出使用示例，最后给出代码开发思路及细节。
0.需求说明

PDF中存在大量表格，需要从表格中提取出指定类型的表格，这些表格主要通过表头和表中的关键字来确定。
1.PDF示例

样例PDF下载地址：样本一、样本二、样本三
2.提取规则

提取规则通过Excel指定，如下示例：
在这里插入图片描述
3.提取结果示例

提取的结果保存在Excel中，结果如下：
在这里插入图片描述
4.使用方法

首先准备好Demo.xlsx文件（下载），同时下载PDFparser.exe 程序（下载），将二者放在同一个目录下，然后将PDF文件准备好放在任意文件夹xxx中，将xxx文件夹和以上两个文件放在同一目录下，双击运行程序即可。
5.代码说明

程序使用pdfplumber模块进行PDF解析以获取表格和文本
程序使用xlwt模块和xlrd 模块进行Excel的读写
程序使用多进程+多线程模式加快速度
程序使用re模块来使用Python正则表达式
6.代码细节

PDF解析

# 该类用来实现PDF表格和文字内容的提取
class Extractor(object):
    def __init__(self, file_path, rules):
        '''
        :param file_path:PDF file path
        :param rules: extract rules
        '''
        self.file_path = file_path
        self.rules = rules
<span class="token comment"># 加载PDF文件</span>
<span class="token keyword">def</span> <span class="token function">parse_pages</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">try</span><span class="token punctuation">:</span>
        pages <span class="token operator">&#61;</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
        pdf <span class="token operator">&#61;</span> pdfplumber<span class="token punctuation">.</span><span class="token builtin">open</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>file_path<span class="token punctuation">)</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">&#39;parse file:{}   page num:{}&#39;</span><span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>basename<span class="token punctuation">(</span>self<span class="token punctuation">.</span>file_path<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pdf<span class="token punctuation">.</span>pages<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
        <span class="token keyword">for</span> index<span class="token punctuation">,</span> page <span class="token keyword">in</span> <span class="token builtin">enumerate</span><span class="token punctuation">(</span>pdf<span class="token punctuation">.</span>pages<span class="token punctuation">)</span><span class="token punctuation">:</span>
            tables <span class="token operator">&#61;</span> page<span class="token punctuation">.</span>extract_tables<span class="token punctuation">(</span><span class="token punctuation">)</span>
            <span class="token keyword">if</span> <span class="token builtin">len</span><span class="token punctuation">(</span>tables<span class="token punctuation">)</span> <span class="token operator">&lt;</span> <span class="token number">1</span><span class="token punctuation">:</span>
                <span class="token keyword">continue</span>
            pages<span class="token punctuation">.</span>append<span class="token punctuation">(</span><span class="token punctuation">{<!-- --></span><span class="token string">&#39;text&#39;</span><span class="token punctuation">:</span> page<span class="token punctuation">.</span>extract_text<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&#39;tables&#39;</span><span class="token punctuation">:</span> tables<span class="token punctuation">,</span> <span class="token string">&#39;page&#39;</span><span class="token punctuation">:</span> index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">}</span><span class="token punctuation">)</span>
        <span class="token keyword">return</span> pages
    <span class="token keyword">except</span> Exception <span class="token keyword">as</span> e<span class="token punctuation">:</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span>
    <span class="token keyword">return</span> <span class="token boolean">None</span>

<span class="token comment"># 提取特定类型表头的表格&#xff0c;规则有rules参数指定</span>
<span class="token keyword">def</span> <span class="token function">extract_table_with_specific_header</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> pages<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">if</span> pages <span class="token keyword">is</span> <span class="token boolean">None</span> <span class="token operator">or</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pages<span class="token punctuation">)</span> <span class="token operator">&lt;</span> <span class="token number">1</span><span class="token punctuation">:</span>
        <span class="token comment"># print(&#39;no-page...&#39;)</span>
        <span class="token keyword">return</span>
    target_tables <span class="token operator">&#61;</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>
    <span class="token comment"># 遍历所有页面</span>
    <span class="token keyword">for</span> index<span class="token punctuation">,</span> page <span class="token keyword">in</span> <span class="token builtin">enumerate</span><span class="token punctuation">(</span>pages<span class="token punctuation">)</span><span class="token punctuation">:</span>
        text <span class="token operator">&#61;</span> page<span class="token punctuation">[</span><span class="token string">&#39;text&#39;</span><span class="token punctuation">]</span>
        tables <span class="token operator">&#61;</span> page<span class="token punctuation">[</span><span class="token string">&#39;tables&#39;</span><span class="token punctuation">]</span>
        page_id <span class="token operator">&#61;</span> page<span class="token punctuation">[</span><span class="token string">&#39;page&#39;</span><span class="token punctuation">]</span> <span class="token operator">-</span> <span class="token number">1</span>
        lines <span class="token operator">&#61;</span> re<span class="token punctuation">.</span>split<span class="token punctuation">(</span>r<span class="token string">&#39;\n&#43;&#39;</span><span class="token punctuation">,</span> text<span class="token punctuation">)</span>
        <span class="token comment"># 遍历当前页面的所有行</span>
        <span class="token keyword">for</span> ind<span class="token punctuation">,</span> line <span class="token keyword">in</span> <span class="token builtin">enumerate</span><span class="token punctuation">(</span>lines<span class="token punctuation">)</span><span class="token punctuation">:</span>
            <span class="token comment"># 判定表头符合规则的表格</span>
            <span class="token keyword">if</span> <span class="token builtin">sum</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token number">1</span> <span class="token keyword">for</span> rule <span class="token keyword">in</span> self<span class="token punctuation">.</span>rules<span class="token punctuation">[</span><span class="token string">&#39;in-header&#39;</span><span class="token punctuation">]</span> <span class="token keyword">if</span> re<span class="token punctuation">.</span>search<span class="token punctuation">(</span>rule<span class="token punctuation">,</span> line<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">&gt;</span> <span class="token number">0</span> <span class="token operator">and</span> \
                    <span class="token builtin">sum</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token number">1</span> <span class="token keyword">for</span> rule <span class="token keyword">in</span> self<span class="token punctuation">.</span>rules<span class="token punctuation">[</span><span class="token string">&#39;not-in-header&#39;</span><span class="token punctuation">]</span> <span class="token keyword">if</span> re<span class="token punctuation">.</span>search<span class="token punctuation">(</span>rule<span class="token punctuation">,</span> line<span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">&#61;&#61;</span> <span class="token number">0</span><span class="token punctuation">:</span>
                <span class="token keyword">if</span> ind <span class="token operator">&gt;&#61;</span> <span class="token builtin">len</span><span class="token punctuation">(</span>lines<span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">:</span>
                    <span class="token keyword">break</span>
                cnt <span class="token operator">&#61;</span> ind <span class="token operator">&#43;</span> <span class="token number">1</span>
                <span class="token builtin">next</span> <span class="token operator">&#61;</span> lines<span class="token punctuation">[</span>ind <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span>
                <span class="token keyword">if</span> ind <span class="token operator">&lt;</span> <span class="token builtin">len</span><span class="token punctuation">(</span>lines<span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">2</span> <span class="token operator">and</span> re<span class="token punctuation">.</span>search<span class="token punctuation">(</span>r<span class="token string">&#39;单位[&#xff1a;:]&#39;</span><span class="token punctuation">,</span> <span class="token builtin">next</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                    <span class="token builtin">next</span> <span class="token operator">&#61;</span> lines<span class="token punctuation">[</span>ind <span class="token operator">&#43;</span> <span class="token number">2</span><span class="token punctuation">]</span>
                    cnt <span class="token operator">&#43;&#61;</span> <span class="token number">1</span>
                <span class="token keyword">for</span> ti<span class="token punctuation">,</span> table <span class="token keyword">in</span> <span class="token builtin">enumerate</span><span class="token punctuation">(</span>tables<span class="token punctuation">)</span><span class="token punctuation">:</span>
                    <span class="token keyword">if</span> <span class="token operator">not</span> table<span class="token punctuation">:</span>
                        <span class="token keyword">continue</span>
                    first <span class="token operator">&#61;</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">.</span>join<span class="token punctuation">(</span><span class="token punctuation">[</span>word <span class="token keyword">for</span> word <span class="token keyword">in</span> table<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token keyword">if</span> word <span class="token keyword">is</span> <span class="token operator">not</span> <span class="token boolean">None</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
                    <span class="token comment"># 表格是完整的情况</span>
                    <span class="token keyword">if</span> first <span class="token operator">&#61;&#61;</span> re<span class="token punctuation">.</span>sub<span class="token punctuation">(</span>r<span class="token string">&#39;\s&#43;&#39;</span><span class="token punctuation">,</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">,</span> <span class="token builtin">next</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                        tables<span class="token punctuation">[</span>ti<span class="token punctuation">]</span> <span class="token operator">&#61;</span> <span class="token boolean">False</span>
                        <span class="token keyword">if</span> index <span class="token operator">&#43;</span> <span class="token number">1</span> <span class="token operator">&lt;</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pages<span class="token punctuation">)</span> <span class="token operator">and</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pages<span class="token punctuation">[</span>index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">&#39;tables&#39;</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">&gt;</span> <span class="token number">0</span><span class="token punctuation">:</span>
                            table_next <span class="token operator">&#61;</span> pages<span class="token punctuation">[</span>index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">&#39;tables&#39;</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
                            fi <span class="token operator">&#61;</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">.</span>join<span class="token punctuation">(</span><span class="token punctuation">[</span>item <span class="token keyword">for</span> item <span class="token keyword">in</span> table_next<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token keyword">if</span> item <span class="token keyword">is</span> <span class="token operator">not</span> <span class="token boolean">None</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
                            <span class="token keyword">if</span> re<span class="token punctuation">.</span>sub<span class="token punctuation">(</span>r<span class="token string">&#39;\s&#43;&#39;</span><span class="token punctuation">,</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">,</span> fi<span class="token punctuation">)</span> <span class="token keyword">in</span> re<span class="token punctuation">.</span>sub<span class="token punctuation">(</span>r<span class="token string">&#39;\s&#43;&#39;</span><span class="token punctuation">,</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">,</span> pages<span class="token punctuation">[</span>index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">&#39;text&#39;</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                                table <span class="token operator">&#43;&#61;</span> table_next
                        target_tables<span class="token punctuation">.</span>append<span class="token punctuation">(</span><span class="token punctuation">{<!-- --></span><span class="token string">&#39;page&#39;</span><span class="token punctuation">:</span> page_id <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token string">&#39;method&#39;</span><span class="token punctuation">:</span> <span class="token string">&#39;exact&#39;</span><span class="token punctuation">,</span> <span class="token string">&#39;table&#39;</span><span class="token punctuation">:</span> table<span class="token punctuation">,</span>
                                              <span class="token string">&#39;table-id&#39;</span><span class="token punctuation">:</span> <span class="token builtin">str</span><span class="token punctuation">(</span>page_id <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">&#43;</span> <span class="token builtin">str</span><span class="token punctuation">(</span>ti <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token punctuation">)</span>
                    <span class="token comment"># 表格可能不完整的情况</span>
                    <span class="token keyword">elif</span> first <span class="token keyword">in</span> re<span class="token punctuation">.</span>sub<span class="token punctuation">(</span>r<span class="token string">&#39;\s&#43;&#39;</span><span class="token punctuation">,</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">,</span> <span class="token builtin">next</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
                        tables<span class="token punctuation">[</span>ti<span class="token punctuation">]</span> <span class="token operator">&#61;</span> <span class="token boolean">False</span>
                        <span class="token keyword">if</span> index <span class="token operator">&#43;</span> <span class="token number">1</span> <span class="token operator">&lt;</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pages<span class="token punctuation">)</span> <span class="token operator">and</span> <span class="token builtin">len</span><span class="token punctuation">(</span>pages<span class="token punctuation">[</span>index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">&#39;tables&#39;</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">&gt;</span> <span class="token number">0</span><span class="token punctuation">:</span>
                            table_next <span class="token operator">&#61;</span> pages<span class="token punctuation">[</span>index <span class="token operator">&#43;</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token string">&#39;tables&#39;</span><span class="token punctuation">]</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
                            fi <span class="token operator">&#61;</span> <span class="token string">&#39;&#39;</span><span class="token punctuation">.</span>join<span