node中读取js文件数据_如何使用Node JS从目录中读取多个文件-CSDN博客

node中读取js文件数据

Hola Node enthusiasts!

Hola Node爱好者！

So, sometime back I was stuck at a problem where I had to read multiple CSV files present inside a directory. I couldn't find any clean and elegant code to solve this problem. And no offence to little pythons out there, I know you could do this on python in less than 10 lines of code. Anyways, let’s see how we do it.

因此，有时我陷入了一个问题，即必须读取目录中存在的多个CSV文件。我找不到任何干净优雅的代码来解决此问题。那里的小python并没有冒犯，我知道您可以用不到10行代码在python上完成此操作。无论如何，让我们看看我们是如何做到的。

Objective

目的

To read multiple CSV files present inside a folder called csvfiles which has files of other formats too(i.e not only .csv).

要读取存在于名为csvfiles的文件夹中的多个CSV文件，该文件夹也具有其他格式的文件(即不仅是.csv)。

Importing the necessary libraries

导入必要的库

var fs = require('fs');
const path = require('path');
const mysqlConnection = require('./connection');
const fastcsv = require('fast-csv');

Function to read the filenames from a given directory path

从给定目录路径读取文件名的功能

So, here I’m providing a hardcoded path to a directory. You can pass your own directory path here. The path.join( ) function takes the current working directory and creates a full path to the directory from where you want to read the CSV files. So, overall the path becomes as shown below. Please note that I’m using Ubuntu.

因此，在这里，我提供了目录的硬编码路径。您可以在此处传递自己的目录路径。 path.join()函数采用当前工作目录，并创建要从中读取CSV文件的目录的完整路径。因此，总体路径如下所示。请注意，我正在使用Ubuntu。

/home/abhishek/Desktop/Coding_Stuffs/csvfiles/

//Hard coded directory has been used.
//Put your path here...
const currDir = path.join(__dirname + '/../csvfiles/');


// Function to get the filenames present
// in the directory
const readdir = (dirname) => {
  return new Promise((resolve, reject) => {
    fs.readdir(dirname, (error, filenames) => {
      if (error) {
        reject(error);
      } else {
        resolve(filenames);
      }
    });
  });
};

Creating a CSV filter

创建一个CSV过滤器

Once you have the filenames sorted there is a CSV filter that you need in order to start parsing files. The CSV filter code filters all the CSV files while leaving any other type of files present in the directory. Here’s how we do it:

对文件名进行排序后，便需要一个CSV过滤器来开始解析文件。 CSV过滤器代码过滤所有CSV文件，同时在目录中保留任何其他类型的文件。这是我们的方法：

We are simply splitting the array of filenames returned by the readdir( ) function by the dot operator. If the file contains the csv part in it after the dot we will filter that out. As simple as that :).

我们只是通过点运算符拆分readdir()函数返回的文件名数组。如果文件中的点后有csv部分，我们将过滤掉它。就如此容易：)。

//CSV filter to filter out the csv files
//in the directory
const filtercsvFiles = (filename) => {
  return filename.split('.')[1] == 'csv';
};

Parsing the files using fast-csv

使用fast-csv解析文件

Now, we have everything that we need to read CSV files. So, we will parse each file within a for loop and store it inside a MySQL database.

现在，我们拥有读取CSV文件所需的一切。因此，我们将在for循环中解析每个文件，并将其存储在MySQL数据库中。

readdir(currDir).then((filenames) => {
  filenames = filenames.filter(filtercsvFiles);


  for (let i = 0; i < filenames.length; i++) {
    let currFilePath = currDir + filenames[i];


    //Use fast-csv to parse the files
    let csvData = [];
    fastcsv
      .parseFile(currFilePath)
      .on('data', (data) => {
        csvData.push(data);
      })
      .on('end', () => {
        csvData.shift();


        //Save the data in mysql
        query =
          'insert ignore into csvData (col1, col2, col3, col4, col5) values ?';
        mysqlConnection.query(query, [csvData], (err, response) => {
          console.log(err || response);
        });
      });
  }
});

Here is the MySQL connection file:

这是MySQL连接文件：

const mysql = require('mysql');


const mysqlConnection = mysql.createConnection({
  host: 'localhost',
  user: 'root',
  database: 'readCSVDir',
  multipleStatements: true,
});


mysqlConnection.connect((error) => {
  if (!error) {
    console.log('Database connected!');
  } else {
    console.log('Failed to connect to the databse!');
  }
});


module.exports = mysqlConnection;

Info about the MySQL database

有关MySQL数据库的信息

Create a database according to the data and provide its name in the database attribute in the mysql.createConnection( ). Then create a table to feed your data.

根据数据创建数据库，并在mysql.createConnection()的数据库属性中提供其名称。然后创建一个表来填充您的数据。

Conclusion

结论

So, that was it. Simple huh? Yeah.

就是这样。简单吧？是的

I hope I could teach you something new. Feel free to reach out if you have any doubts.

我希望我能教你一些新东西。如有任何疑问，请随时与我们联系。

Peace.

和平。