先自我介绍一下,小编浙江大学毕业,去过华为、字节跳动等大厂,目前阿里P7
深知大多数程序员,想要提升技能,往往是自己摸索成长,但自己不成体系的自学效果低效又漫长,而且极易碰到天花板技术停滞不前!
因此收集整理了一份《2024年最新大数据全套学习资料》,初衷也很简单,就是希望能够帮助到想自学提升又不知道该从何学起的朋友。
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上大数据知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新
如果你需要这些资料,可以添加V获取:vip204888 (备注大数据)
正文
}
可以看到数据是基于cell
被处理的,我们要在startRow
和endRow
方法中自己构建行数据
public class SheetHandler implements XSSFSheetXMLHandler.SheetContentsHandler,Procedure {
protected List row = new LinkedList<>();
private ExcleHandler excleHandler;
private int preIndex = 1;
public SheetHandler(ExcleHandler excleHandler){
this.excleHandler = excleHandler;
}
@Override
public void run() {
excleHandler.over();
}
@Override
public void startRow(int i) {
row.clear();
}
@Override
public void endRow(int i) {
excleHandler.handleRow(row,i);
}
@Override
public void cell(String cellReference, String formattedValue, XSSFComment comment) {
int index = excleCelNumToIndex(cellReference);
if(index - preIndex > 1){
for(int i = 0;i < index - preIndex - 1;i++){
row.add(“”);
}
}
row.add(formattedValue);
preIndex = index;
}
@Override
public void headerFooter(String s, boolean b, String s1) {
}
private int excleCelNumToIndex(String celNum){
int num = 0;
int result = 0;
celNum = Pattern.compile(“[\d]”).matcher(celNum).replaceAll(“”);
int length = celNum.length();
for(int i = 0; i < length; i++) {
char ch = celNum.charAt(length - i - 1);
num = (int)(ch - ‘A’ + 1) ;
num *= Math.pow(26, i);
result += num;
}
return result;
}
}
实现了一个事件处理接口,将数据封装成行数据,并且将具体的处理逻辑委托给ExcleHandler
接口,此接口基于行来处理excle数据
public interface ExcleHandler {
public void handleRow(Object row, int rowNumber);
public void over();
}
需要注意的是,poi提供给我们的读取工具会跳过某些没有编辑过的单元格,我们需要在构建行的时候自己补全缺失的数据
然后自定义一个ExcleReader
用于读取excle
package com.apex.bss.mod.util.excle;
import org.apache.poi.openxml4j.exceptions.InvalidFormatException;
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.openxml4j.opc.PackageAccess;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.util.SAXHelper;
import org.apache.poi.xssf.eventusermodel.ReadOnlySharedStringsTable;
import org.apache.poi.xssf.eventusermodel.XSSFReader;
import org.apache.poi.xssf.eventusermodel.XSSFSheetXMLHandler;
import org.apache.poi.xssf.model.CommentsTable;
import org.apache.poi.xssf.model.StylesTable;
import org.apache.poi.xssf.usermodel.XSSFComment;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.XMLReader;
import javax.xml.parsers.ParserConfigurationException;
import java.io.IOException;
import java.io.InputStream;
import java.util.LinkedList;
import java.util.List;
import java.util.regex.Pattern;
/**
- Created by Feng
*/
public class EventExcleReader {
private OPCPackage opcPackage;
private SheetHandler handler;
private ExcleTypeEnum excleTypeEnum;
public EventExcleReader(String filename, ExcleTypeEnum excleType, ExcleHandler handler) throws IllegalAccessException, InvalidFormatException {
if(excleType == ExcleTypeEnum.XLS){
throw new IllegalAccessException(“暂不支持XLS文件”);
}
opcPackage = OPCPackage.open(filename, PackageAccess.READ);
this.handler = new SheetHandler(handler);
}
public EventExcleReader(InputStream inputStream, ExcleTypeEnum excleType, ExcleHandler handler) throws IllegalAccessException, InvalidFormatException, IOException {
if(excleType == ExcleTypeEnum.XLS){
throw new IllegalAccessException(“暂不支持XLS文件”);
}
opcPackage = OPCPackage.open(inputStream);
this.handler = new SheetHandler(handler);
}
public void read(){
InputStream sheetInputStream = null;
try {
XSSFReader xssfReader = new XSSFReader(opcPackage);
StylesTable styles = xssfReader.getStylesTable();
ReadOnlySharedStringsTable strings = new ReadOnlySharedStringsTable(opcPackage);
sheetInputStream = xssfReader.getSheetsData().next();
processSheet(styles, strings, sheetInputStream);
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}finally {
if(sheetInputStream != null){
try {
sheetInputStream.close();
} catch (IOException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
if(opcPackage != null){
try {
opcPackage.close();
} catch (IOException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
}
}
private void processSheet(StylesTable styles, ReadOnlySharedStringsTable strings, InputStream sheetInputStream) throws SAXException, ParserConfigurationException, IOException{
XMLReader sheetParser = SAXHelper.newXMLReader();
sheetParser.setContentHandler(new LocalXSSFSheetXmlHandler(styles, strings, handler, false).handleEnd(handler));
sheetParser.parse(new InputSource(sheetInputStream));
}
}
然后根据具体业务实现ExcleHandler
接口处理业务逻辑即可,这里我们还可以进一步对其进行封装实现数据的分段处理
public class ExcleUtil {
public static void readExcleByReduce(InputStream inputStream, int reduce, ExcleTypeEnum excleType, BiConsumer<LinkedList<LinkedList>,LinkedList> consumer) throws IllegalAccessException, IOException, InvalidFormatException {
new EventExcleReader(inputStream, excleType, new ReduceExcleHandler(consumer).reduceBy(reduce)).read();
}
public static void readExcle(InputStream inputStream,ExcleTypeEnum excleType, BiConsumer<LinkedList<LinkedList>,LinkedList> consumer) throws IllegalAccessException, IOException, InvalidFormatException {
new EventExcleReader(inputStream, excleType, new ReduceExcleHandler(consumer)).read();
}
/**
- 分段读取类
*/
private static class ReduceExcleHandler implements ExcleHandler{
private int reduce = 1;
BiConsumer<LinkedList<LinkedList>,LinkedList> consumer;
private LinkedList<LinkedList> datas = new LinkedList<>();
private LinkedList head;
public ReduceExcleHandler(BiConsumer<LinkedList<LinkedList>,LinkedList> consumer){
this.consumer = consumer;
}
private ReduceExcleHandler reduceBy(int reduce){
this.reduce = reduce;
return this;
}
@Override
public void handleRow(Object row, int rowNumber) {
if(rowNumber <= 0){
head = new LinkedList<>((LinkedList) row);
}else {
if (datas.size() >= reduce) {
consumer.accept(datas, head);
datas.clear();
}
datas.add(new LinkedList<>((LinkedList) row));
}
}
@Override
public void over() {
consumer.accept(datas,head);
}
}
}
在上述代码中我们实现了ReduceExcleHandler
,将excle数据以reduce为间隔进行分割,分批次处理,下面是一个简单的上传并导入excle到指定表的示例
package com.apex.bss.mod.util.excle;
import net.sf.hibernate.mapping.Collection;
import net.spy.memcached.util.StringUtils;
import plugins.bean.DBUtils;
import plugins.bean.DatabaseUtils;
import sun.awt.image.ImageWatched;
import java.sql.DatabaseMetaData;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.util.LinkedList;
import java.util.List;
import java.util.function.BiConsumer;
/**
- Created by Feng
*/
public class OracleReduceConsumer implements BiConsumer<LinkedList<LinkedList>,LinkedList> {
private int successCount = 0;
private int failCount = 0;
private String insertSql;
private static DatabaseUtils databaseUtils = new DBUtils();
private final String tableName;
网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。
需要这份系统化的资料的朋友,可以添加V获取:vip204888 (备注大数据)
一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!
ents BiConsumer<LinkedList<LinkedList>,LinkedList> {
private int successCount = 0;
private int failCount = 0;
private String insertSql;
private static DatabaseUtils databaseUtils = new DBUtils();
private final String tableName;
网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。
需要这份系统化的资料的朋友,可以添加V获取:vip204888 (备注大数据)
[外链图片转存中…(img-wds7NfDG-1713211890282)]
一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!