CSV文件解析为实体:字段对应到指定列

凯文辣婆婆

已于 2023-08-29 10:29:45 修改

阅读量280

点赞数

文章标签： java

于 2023-08-23 11:52:00 首次发布

本文链接：https://blog.csdn.net/weixin_43472776/article/details/132448113

版权

CSV文件解析为实体:字段对应到指定列

场景:
需要拉取很多种csv文件的数据,但是这些csv文件都是无表头的,唯一确定的是每一列的数据是固定的;如果是一类数据倒是直接解析处理就完事儿了,面对很多类数据决定采用打注解的方式,在每一个字段上标识它需要取第几列的数据,然后进行统一处理;

自定义注解:

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface ExcelColumn {

    /**
     * 第几列
     * @return
     */
    @NotNull
    int columnIndex();
}

测试实体:

@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
public class TestInfo {
    
    @ExcelColumn(columnIndex = 0)
    private String serNo;

    @ExcelColumn(columnIndex = 1)
    private String name;
  
    @ExcelColumn(columnIndex = 2)
    private String organizationId;
    
    @ExcelColumn(columnIndex = 3)
    private String organizationName;
    
}

CsvUtil:

@Slf4j
public class CsvUtil {

    private final static String CHAR_SET = "utf-8";

    /**
     * 解析csv(无表头)
     * @param file
     * @param clazz
     * @param <T>
     * @return
     */
    public static <T> List<T> analyzeCsv(File file, Class<T> clazz) {
        Field[] fields = getClassFields(clazz, ExcelColumn.class);
        Map<Integer, Field> fieldMap = Arrays.stream(fields).collect(Collectors.toMap(it -> it.getAnnotation(ExcelColumn.class).columnIndex(), it -> it));
        List<T> list = new ArrayList<>();

        try(FileInputStream inputStream =new FileInputStream(file);
            InputStreamReader in = new InputStreamReader(inputStream, CHAR_SET);
            BufferedReader reader = new BufferedReader(in)) {
            String lineText;
            while ((lineText = reader.readLine()) != null) {
                if (StringUtils.isNotBlank(lineText)){
                    String[] values = lineText.split(",");
                    handleRow(values, list, clazz, fieldMap);
                }
            }
        }catch (Exception e){
            log.error("【csv解析】文件读取失败:", e);
        }

        return list;
    }
    
   /**
     * 递归获取有指定注解的所有字段, 包括父类(Object除外)
     *
     * @param clazz
     * @param annotationClazz
     * @return
     */
    public static Field[] getClassFields(Class<?> clazz, Class<? extends Annotation> annotationClazz) {
        List<Field> list = Lists.newArrayList();
        while (clazz != null && !Object.class.equals(clazz)) {
            List<Field> fields = Arrays.asList(clazz.getDeclaredFields());
            list.addAll(fields.stream().filter(it -> it.getAnnotation(annotationClazz) != null).collect(Collectors.toList()));
            clazz = clazz.getSuperclass();
        }
        return list.toArray(new Field[]{});
    }
    
    private static <T> void handleRow(String[] values, List<T> list, Class<T> clazz, Map<Integer, Field> fieldMap) {
        //创建一个对象
        try {
            T obj = clazz.newInstance();
            for (Map.Entry<Integer, Field> entry : fieldMap.entrySet()) {
                Field field = entry.getValue();
                int columnIndex = entry.getKey();
                String value = getValue(values, columnIndex);
                //处理数据
                field.setAccessible(Boolean.TRUE);
                field.set(obj, value);
            }
            list.add(obj);
        } catch (InstantiationException | IllegalAccessException e) {
            log.error("【csv解析】对象创建失败:{}", values, e);
        }
    }

    private static String getValue(String[] values, int columnIndex){
        try {
            return values[columnIndex];
        }catch (Exception e){
            return null;
        }
    }
}

测试代码:

    public static void main(String[] args) {
        try {
            File file = new File("C:\\Users\\Downloads\\test.csv");
            long start = System.currentTimeMillis();
            List<TestInfo> list = analyzeCsv(file, TestInfo.class);
            System.out.println("list size:" + list.size() + "  use time:"+ (System.currentTimeMillis()-start) + "ms");
        }catch (Exception e){
             throw new RuntimeException(e);
        }
    }