目录
参考链接:https://blog.csdn.net/u013698706/article/details/106422480
前言
复现论文,CSE-CIC-IDS2018全部有452.8GiB,所幸有处理好的CSV文件,2018的合计有6GiB左右;
四处找寻不到,CSDN会员资源下载下来里面是不对的内容,没办法自己去官网下载,过程不易,特总结一下方法,以及分享我下载的2018数据集。
我采用的下载办法,是用windows系统下的
下载
【1】 官网下载AWS工具
或者用我下载下来的(,windows64位)
【2】 在aws的安装文件夹中打开cmd命令窗口,运行:
aws s3 ls --no-sign-request "s3://cse-cic-ids2018" --recursive --human-readable --summarize
结果显示的是文件的一个总列表详细信息
C:\Users\LCF>aws s3 ls --no-sign-request "s3://cse-cic-ids2018" --recursive --human-readable --summarize
D:\Program Files\Amazon\AWSCLI\.\dateutil\parser\_parser.py:1189: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
2018-10-10 19:52:09 0 Bytes Original Network Traffic and Log data/
2018-10-10 19:52:23 0 Bytes Original Network Traffic and Log data/Friday-02-03-2018/
2018-10-10 20:00:39 225.8 MiB Original Network Traffic and Log data/Friday-02-03-2018/logs.zip
2018-10-10 20:00:51 41.7 GiB Original Network Traffic and Log data/Friday-02-03-2018/pcap.zip
2018-10-10 19:52:34 0 Bytes Original Network Traffic and Log data/Friday-16-02-2018/
2018-10-10 20:45:49 148.1 MiB Original Network Traffic and Log data/Friday-16-02-2018/logs.zip
2018-10-10 20:46:01 35.9 GiB Original Network Traffic and Log data/Friday-16-02-2018/pcap.zip
2018-10-10 19:52:41 0 Bytes Original Network Traffic and Log data/Friday-23-02-2018/
2018-10-10 20:46:10 199.8 MiB Original Network Traffic and Log data/Friday-23-02-2018/logs.zip
2018-10-10 20:46:31 55.0 GiB Original Network Traffic and Log data/Friday-23-02-2018/pcap.zip
2018-10-10 19:52:47 0 Bytes Original Network Traffic and Log data/Thursday-01-03-2018/
2018-10-10 21:41:13 217.1 MiB Original Network Traffic and Log data/Thursday-01-03-2018/logs.zip
2018-10-10 21:41:45 48.8 GiB Original Network Traffic and Log data/Thursday-01-03-2018/pcap.zip
2018-10-10 19:52:54 0 Bytes Original Network Traffic and Log data/Thursday-15-02-2018/
2018-10-10 21:41:28 142.6 MiB Original Network Traffic and Log data/Thursday-15-02-2018/logs.zip
2018-10-10 21:41:55 38.4 GiB Original Network Traffic and Log data/Thursday-15-02-2018/pcap.zip
2018-10-10 19:53:01 0 Bytes Original Network Traffic and Log data/Thursday-22-02-2018/
2018-10-10 21:41:42 195.3 MiB Original Network Traffic and Log data/Thursday-22-02-2018/logs.zip
2018-10-10 21:42:27 46.8 GiB Original Network Traffic and Log data/Thursday-22-02-2018/pcap.zip
2018-10-10 19:53:07 0 Bytes Original Network Traffic and Log data/Tuesday-20-02-2018/
2018-10-10 22:39:45 178.9 MiB Original Network Traffic and Log data/Tuesday-20-02-2018/logs.zip
2018-10-10 22:40:40 41.3 GiB Original Network Traffic and Log data/Tuesday-20-02-2018/pcap.rar
2018-10-10 19:53:14 0 Bytes Original Network Traffic and Log data/Wednesday-14-02-2018/
2018-10-11 00:44:20 133.7 MiB Original Network Traffic and Log data/Wednesday-14-02-2018/logs.zip
2018-10-11 20:22:03 37.2 GiB Original Network Traffic and Log data/Wednesday-14-02-2018/pcap.zip
2018-10-10 19:53:21 0 Bytes Original Network Traffic and Log data/Wednesday-21-02-2018/
2018-10-11 00:44:34 185.6 MiB Original Network Traffic and Log data/Wednesday-21-02-2018/logs.zip
2018-10-11 21:35:15 49.8 GiB Original Network Traffic and Log data/Wednesday-21-02-2018/pcap.zip
2018-10-10 19:53:28 0 Bytes Original Network Traffic and Log data/Wednesday-28-02-2018/
2018-10-11 00:44:47 216.1 MiB Original Network Traffic and Log data/Wednesday-28-02-2018/logs.zip
2018-10-11 22:21:03 49.6 GiB Original Network Traffic and Log data/Wednesday-28-02-2018/pcap.zip
2018-10-12 00:02:25 0 Bytes Processed Traffic Data for ML Algorithms/
2018-10-12 00:02:49 336.0 MiB Processed Traffic Data for ML Algorithms/Friday-02-03-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:03:10 318.3 MiB Processed Traffic Data for ML Algorithms/Friday-16-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:03:33 365.1 MiB Processed Traffic Data for ML Algorithms/Friday-23-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:03:59 3.8 GiB Processed Traffic Data for ML Algorithms/Thuesday-20-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:08:38 102.8 MiB Processed Traffic Data for ML Algorithms/Thursday-01-03-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:08:48 358.5 MiB Processed Traffic Data for ML Algorithms/Thursday-15-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:09:20 364.9 MiB Processed Traffic Data for ML Algorithms/Thursday-22-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:09:44 341.6 MiB Processed Traffic Data for ML Algorithms/Wednesday-14-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:10:12 313.7 MiB Processed Traffic Data for ML Algorithms/Wednesday-21-02-2018_TrafficForML_CICFlowMeter.csv
2018-10-12 00:10:33 199.6 MiB Processed Traffic Data for ML Algorithms/Wednesday-28-02-2018_TrafficForML_CICFlowMeter.csv
Total Objects: 42
Total Size: 452.8 GiB
【3】我们需要下载的是最后十个CSV文件,经过测试,逐条下载比递归下载要快。举例:
逐条下载:
需要逐个替换下载的文件
aws s3 cp --no-sign-request "s3://cse-cic-ids2018/Processed Traffic Data for ML Algorithms/Wednesday-28-02-2018_TrafficForML_CICFlowMeter.csv" D:\A_Datasets\IDS2018\Wednesday-28-02-2018_TrafficForML_CICFlowMeter.csv
命令行解读:
代表链接文件:
"s3://cse-cic-ids2018/Processed Traffic Data for ML Algorithms/Wednesday-28-02-2018_TrafficForML_CICFlowMeter.csv"
代表我的存储路径和命名
D:\A_Datasets\IDS2018\Wednesday-28-02-2018_TrafficForML_CICFlowMeter.csv
递归下载
aws s3 cp --no-sign-request "s3://cse-cic-ids2018/Processed Traffic Data for ML Algorithms/" D:\A_Datasets\IDS2018 --recursive
我的下载:
下载好的数据集进行公开:
看到下载好的激动吗?一键三联哈!关注,点赞,收藏!
链接:https://pan.baidu.com/s/10VZKII0YXczbrirRTia1Xg
提取码:lyk8
复制这段内容后打开百度网盘手机App,操作更方便哦