django基于Hadoop的气象数据的研究与应用(源码+vue+可视化大屏展示+爬虫分析+讲解等)

收藏关注不迷路!!

🌟文末获取源码+数据库🌟

感兴趣的可以先收藏起来,还有大家在毕设选题(免费咨询指导选题),项目以及论文编写等相关问题都可以给我留言咨询,希望帮助更多的人


前言

💗博主介绍:✨全网粉丝10W+,CSDN特邀作者、博客专家、CSDN新星计划导师,专注于Java/Python/小程序app/深度学习等计算机设计,主要对象是咱们计算机相关专业的大学生,希望您们都能前途无量!✨💗

👇🏻 精彩专栏 推荐订阅👇🏻

计算机毕业设计设计精品实战案例

感兴趣的可以先收藏起来,还有大家在毕设选题,项目以及论文编写等相关问题都可以给我留言咨询,希望帮助更多的人

  

程序资料获取

🌟文末获取资料🌟

一、项目技术

开发语言:Python
python框架:Django
软件版本:python3.7/python3.8
数据库:mysql 5.7或更高版本
数据库工具:Navicat11
开发软件:PyCharm/vs code
前端框架:vue.js

二、项目内容和功能介绍

  🎈1.项目内容

  
数据采集和预处理:从气象观测站、卫星、雷达等多种数据源采集气象数据,并进行数据清洗、去重、格式转换等预处理操作。
2. 数据存储和管理:将处理后的气象数据存储到Hadoop分布式文件系统(HDFS)中,并进行数据备份、恢复、权限管理等操作。
3. 数据分析和挖掘:使用Hadoop生态系统中的分布式计算框架,如MapReduce、Spark等,对气象数据进行分析和挖掘,包括数据聚合、统计、分类、预测等操作。
4. 数据可视化和展示:将分析结果以图表、地图等形式进行可视化展示,便于用户理解和使用。
  🎈2.功能介绍

  所需条件
1.Hadoop生态系统:包括HDFS、MapReduce、Spark、Hive、HBase等分布式计算和存储技术。
5. 数据采集和预处理技术:包括Flume、Kafka、Logstash等。
6. 数据分析和挖掘技术:包括机器学习算法、数据挖掘算法、统计分析方法等。
7. 数据可视化和展示技术:包括D3.js、Echarts、Leaflet等可视化工具。

预期成果
8. 成功采集多种数据源气象数据,将数据预处理。
9. 成功存入Hadoop分布式系统中,并对数据进行备份、权限管理等操作。
10. 成功使用分布式计算框架对气象数据进行分析与挖掘。
11. 将分析结果进行可视化展示,创建交互平台,绘制柱形图、折线图、雷达图等,实现图表与用户交互。

数据库设计


 Date: 16/03/2024 22:31:57
*/

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for auth_group
-- ----------------------------
DROP TABLE IF EXISTS `auth_group`;
CREATE TABLE `auth_group`  (
  `id` int NOT NULL AUTO_INCREMENT,
  `name` varchar(150) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `name`(`name` ASC) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_group
-- ----------------------------

-- ----------------------------
-- Table structure for auth_group_permissions
-- ----------------------------
DROP TABLE IF EXISTS `auth_group_permissions`;
CREATE TABLE `auth_group_permissions`  (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `group_id` int NOT NULL,
  `permission_id` int NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `auth_group_permissions_group_id_permission_id_0cd325b0_uniq`(`group_id` ASC, `permission_id` ASC) USING BTREE,
  INDEX `auth_group_permissio_permission_id_84c5c92e_fk_auth_perm`(`permission_id` ASC) USING BTREE,
  CONSTRAINT `auth_group_permissio_permission_id_84c5c92e_fk_auth_perm` FOREIGN KEY (`permission_id`) REFERENCES `auth_permission` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT,
  CONSTRAINT `auth_group_permissions_group_id_b120cbf9_fk_auth_group_id` FOREIGN KEY (`group_id`) REFERENCES `auth_group` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE = InnoDB CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_group_permissions
-- ----------------------------

-- ----------------------------
-- Table structure for auth_permission
-- ----------------------------
DROP TABLE IF EXISTS `auth_permission`;
CREATE TABLE `auth_permission`  (
  `id` int NOT NULL AUTO_INCREMENT,
  `name` varchar(255) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `content_type_id` int NOT NULL,
  `codename` varchar(100) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `auth_permission_content_type_id_codename_01ab375a_uniq`(`content_type_id` ASC, `codename` ASC) USING BTREE,
  CONSTRAINT `auth_permission_content_type_id_2f476e4b_fk_django_co` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE = InnoDB AUTO_INCREMENT = 69 CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_permission
-- ----------------------------
INSERT INTO `auth_permission` VALUES (1, 'Can add 成都表', 1, 'add_mc_chengdu');
INSERT INTO `auth_permission` VALUES (2, 'Can change 成都表', 1, 'change_mc_chengdu');
INSERT INTO `auth_permission` VALUES (3, 'Can delete 成都表', 1, 'delete_mc_chengdu');
INSERT INTO `auth_permission` VALUES (4, 'Can view 成都表', 1, 'view_mc_chengdu');
INSERT INTO `auth_permission` VALUES (5, 'Can add 城市信息表', 2, 'add_mc_city');
INSERT INTO `auth_permission` VALUES (6, 'Can change 城市信息表', 2, 'change_mc_city');
INSERT INTO `auth_permission` VALUES (7, 'Can delete 城市信息表', 2, 'delete_mc_city');
INSERT INTO `auth_permission` VALUES (8, 'Can view 城市信息表', 2, 'view_mc_city');
INSERT INTO `auth_permission` VALUES (9, 'Can add 环境表', 3, 'add_mc_huanjing');
INSERT INTO `auth_permission` VALUES (10, 'Can change 环境表', 3, 'change_mc_huanjing');
INSERT INTO `auth_permission` VALUES (11, 'Can delete 环境表', 3, 'delete_mc_huanjing');
INSERT INTO `auth_permission` VALUES (12, 'Can view 环境表', 3, 'view_mc_huanjing');
INSERT INTO `auth_permission` VALUES (13, 'Can add 系统管理员', 4, 'add_mc_supermanager');
INSERT INTO `auth_permission` VALUES (14, 'Can change 系统管理员', 4, 'change_mc_supermanager');
INSERT INTO `auth_permission` VALUES (15, 'Can delete 系统管理员', 4, 'delete_mc_supermanager');
INSERT INTO `auth_permission` VALUES (16, 'Can view 系统管理员', 4, 'view_mc_supermanager');
INSERT INTO `auth_permission` VALUES (17, 'Can add 温度DecimalField,例如Decimal5,2', 5, 'add_mc_temperature');
INSERT INTO `auth_permission` VALUES (18, 'Can change 温度DecimalField,例如Decimal5,2', 5, 'change_mc_temperature');
INSERT INTO `auth_permission` VALUES (19, 'Can delete 温度DecimalField,例如Decimal5,2', 5, 'delete_mc_temperature');
INSERT INTO `auth_permission` VALUES (20, 'Can view 温度DecimalField,例如Decimal5,2', 5, 'view_mc_temperature');
INSERT INTO `auth_permission` VALUES (21, 'Can add 用户信息表', 6, 'add_mc_user');
INSERT INTO `auth_permission` VALUES (22, 'Can change 用户信息表', 6, 'change_mc_user');
INSERT INTO `auth_permission` VALUES (23, 'Can delete 用户信息表', 6, 'delete_mc_user');
INSERT INTO `auth_permission` VALUES (24, 'Can view 用户信息表', 6, 'view_mc_user');
INSERT INTO `auth_permission` VALUES (25, 'Can add 用户关注的城市表', 7, 'add_mc_userfavoritecities');
INSERT INTO `auth_permission` VALUES (26, 'Can change 用户关注的城市表', 7, 'change_mc_userfavoritecities');
INSERT INTO `auth_permission` VALUES (27, 'Can delete 用户关注的城市表', 7, 'delete_mc_userfavoritecities');
INSERT INTO `auth_permission` VALUES (28, 'Can view 用户关注的城市表', 7, 'view_mc_userfavoritecities');
INSERT INTO `auth_permission` VALUES (29, 'Can add 天气预警表', 8, 'add_mc_weatheralert');
INSERT INTO `auth_permission` VALUES (30, 'Can change 天气预警表', 8, 'change_mc_weatheralert');
INSERT INTO `auth_permission` VALUES (31, 'Can delete 天气预警表', 8, 'delete_mc_weatheralert');
INSERT INTO `auth_permission` VALUES (32, 'Can view 天气预警表', 8, 'view_mc_weatheralert');
INSERT INTO `auth_permission` VALUES (33, 'Can add 天气分析记录表', 9, 'add_mc_weatheranalysis');
INSERT INTO `auth_permission` VALUES (34, 'Can change 天气分析记录表', 9, 'change_mc_weatheranalysis');
INSERT INTO `auth_permission` VALUES (35, 'Can delete 天气分析记录表', 9, 'delete_mc_weatheranalysis');
INSERT INTO `auth_permission` VALUES (36, 'Can view 天气分析记录表', 9, 'view_mc_weatheranalysis');
INSERT INTO `auth_permission` VALUES (37, 'Can add 天气数据表', 10, 'add_mc_weatherdata');
INSERT INTO `auth_permission` VALUES (38, 'Can change 天气数据表', 10, 'change_mc_weatherdata');
INSERT INTO `auth_permission` VALUES (39, 'Can delete 天气数据表', 10, 'delete_mc_weatherdata');
INSERT INTO `auth_permission` VALUES (40, 'Can view 天气数据表', 10, 'view_mc_weatherdata');
INSERT INTO `auth_permission` VALUES (41, 'Can add log entry', 11, 'add_logentry');
INSERT INTO `auth_permission` VALUES (42, 'Can change log entry', 11, 'change_logentry');
INSERT INTO `auth_permission` VALUES (43, 'Can delete log entry', 11, 'delete_logentry');
INSERT INTO `auth_permission` VALUES (44, 'Can view log entry', 11, 'view_logentry');
INSERT INTO `auth_permission` VALUES (45, 'Can add permission', 12, 'add_permission');
INSERT INTO `auth_permission` VALUES (46, 'Can change permission', 12, 'change_permission');
INSERT INTO `auth_permission` VALUES (47, 'Can delete permission', 12, 'delete_permission');
INSERT INTO `auth_permission` VALUES (48, 'Can view permission', 12, 'view_permission');
INSERT INTO `auth_permission` VALUES (49, 'Can add group', 13, 'add_group');
INSERT INTO `auth_permission` VALUES (50, 'Can change group', 13, 'change_group');
INSERT INTO `auth_permission` VALUES (51, 'Can delete group', 13, 'delete_group');
INSERT INTO `auth_permission` VALUES (52, 'Can view group', 13, 'view_group');
INSERT INTO `auth_permission` VALUES (53, 'Can add user', 14, 'add_user');
INSERT INTO `auth_permission` VALUES (54, 'Can change user', 14, 'change_user');
INSERT INTO `auth_permission` VALUES (55, 'Can delete user', 14, 'delete_user');
INSERT INTO `auth_permission` VALUES (56, 'Can view user', 14, 'view_user');
INSERT INTO `auth_permission` VALUES (57, 'Can add content type', 15, 'add_contenttype');
INSERT INTO `auth_permission` VALUES (58, 'Can change content type', 15, 'change_contenttype');
INSERT INTO `auth_permission` VALUES (59, 'Can delete content type', 15, 'delete_contenttype');
INSERT INTO `auth_permission` VALUES (60, 'Can view content type', 15, 'view_contenttype');
INSERT INTO `auth_permission` VALUES (61, 'Can add session', 16, 'add_session');
INSERT INTO `auth_permission` VALUES (62, 'Can change session', 16, 'change_session');
INSERT INTO `auth_permission` VALUES (63, 'Can delete session', 16, 'delete_session');
INSERT INTO `auth_permission` VALUES (64, 'Can view session', 16, 'view_session');
INSERT INTO `auth_permission` VALUES (65, 'Can add captcha store', 17, 'add_captchastore');
INSERT INTO `auth_permission` VALUES (66, 'Can change captcha store', 17, 'change_captchastore');
INSERT INTO `auth_permission` VALUES (67, 'Can delete captcha store', 17, 'delete_captchastore');
INSERT INTO `auth_permission` VALUES (68, 'Can view captcha store', 17, 'view_captchastore');

-- ----------------------------
-- Table structure for auth_user
-- ----------------------------
DROP TABLE IF EXISTS `auth_user`;
CREATE TABLE `auth_user`  (
  `id` int NOT NULL AUTO_INCREMENT,
  `password` varchar(128) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `last_login` datetime(6) NULL DEFAULT NULL,
  `is_superuser` tinyint(1) NOT NULL,
  `username` varchar(150) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `first_name` varchar(150) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `last_name` varchar(150) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `email` varchar(254) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `is_staff` tinyint(1) NOT NULL,
  `is_active` tinyint(1) NOT NULL,
  `date_joined` datetime(6) NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `username`(`username` ASC) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 6 CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_user
-- ----------------------------
INSERT INTO `auth_user` VALUES (1, 'pbkdf2_sha256$720000$39q7q4OYxekl9nvR36KsZJ$KruMyfaAhXa9XWIOFGT0KxS4BYXpEHmyL7UMf6Il+x8=', '2024-01-24 09:07:52.359635', 1, 'administrator', '', '', '', 1, 1, '2024-01-24 03:02:05.371984');
INSERT INTO `auth_user` VALUES (2, 'pbkdf2_sha256$720000$84etjADQIe7KDXWmMnuLVX$6Jc2OGkajOG565QMokHijK8n1vi4MBZKkxmSueozi2I=', '2024-03-11 12:15:39.021877', 1, '123456', '', '', '123456@qq.com', 1, 1, '2024-01-24 03:51:00.000000');
INSERT INTO `auth_user` VALUES (3, 'pbkdf2_sha256$720000$DvBQR5e6Z3xJQBjWm7pnVv$KBuGY0PI599qaLQGKhMP1kdrjVxjAOWrWN56P3sHJwE=', '2024-03-02 02:02:08.292595', 1, '666666', '', '', '123456@qq.com', 0, 1, '2024-03-02 02:02:02.150912');
INSERT INTO `auth_user` VALUES (4, 'pbkdf2_sha256$720000$OSB9bVpbSzbqPKk5Jdabv3$WEIgcIo7Wvka39Ltloe6fDkgGHERxiEBW/guy+3QOow=', '2024-03-08 07:01:23.090288', 1, '123456789', '', '', '123456@qq.com', 0, 1, '2024-03-08 07:01:16.456424');
INSERT INTO `auth_user` VALUES (5, 'pbkdf2_sha256$720000$BEJtx4S8b4p1RT0rT12J6E$RCckxcPIG1GOMjgbMg69i6RxycydLsOOBhGTSvNeAmg=', '2024-03-08 07:27:44.174081', 1, '111111', '', '', '11111@qq.com', 0, 1, '2024-03-08 07:27:38.438845');

-- ----------------------------
-- Table structure for auth_user_groups
-- ----------------------------
DROP TABLE IF EXISTS `auth_user_groups`;
CREATE TABLE `auth_user_groups`  (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `user_id` int NOT NULL,
  `group_id` int NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `auth_user_groups_user_id_group_id_94350c0c_uniq`(`user_id` ASC, `group_id` ASC) USING BTREE,
  INDEX `auth_user_groups_group_id_97559544_fk_auth_group_id`(`group_id` ASC) USING BTREE,
  CONSTRAINT `auth_user_groups_group_id_97559544_fk_auth_group_id` FOREIGN KEY (`group_id`) REFERENCES `auth_group` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT,
  CONSTRAINT `auth_user_groups_user_id_6a12ed8b_fk_auth_user_id` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE = InnoDB CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_user_groups
-- ----------------------------

-- ----------------------------
-- Table structure for auth_user_user_permissions
-- ----------------------------
DROP TABLE IF EXISTS `auth_user_user_permissions`;
CREATE TABLE `auth_user_user_permissions`  (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `user_id` int NOT NULL,
  `permission_id` int NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `auth_user_user_permissions_user_id_permission_id_14a6b632_uniq`(`user_id` ASC, `permission_id` ASC) USING BTREE,
  INDEX `auth_user_user_permi_permission_id_1fbb5f2c_fk_auth_perm`(`permission_id` ASC) USING BTREE,
  CONSTRAINT `auth_user_user_permi_permission_id_1fbb5f2c_fk_auth_perm` FOREIGN KEY (`permission_id`) REFERENCES `auth_permission` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT,
  CONSTRAINT `auth_user_user_permissions_user_id_a95ead1b_fk_auth_user_id` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE = InnoDB CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of auth_user_user_permissions
-- ----------------------------

-- ----------------------------
-- Table structure for captcha_captchastore
-- ----------------------------
DROP TABLE IF EXISTS `captcha_captchastore`;
CREATE TABLE `captcha_captchastore`  (
  `id` int NOT NULL AUTO_INCREMENT,
  `challenge` varchar(32) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `response` varchar(32) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `hashkey` varchar(40) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL,
  `expiration` datetime(6) NOT NULL,
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `hashkey`(`hashkey` ASC) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 57 CHARACTER SET = utf8mb3 COLLATE = utf8mb3_general_ci ROW_FORMAT = DYNAMIC;

四、效果图

请添加图片描述

五 、资料获取

文章下方名片联系我即可~

精彩专栏推荐订阅:在下方专栏👇🏻

毕业设计精品实战案例

收藏关注不迷路!!

🌟文末获取设计🌟
请添加图片描述
请添加图片描述
请添加图片描述

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

qq_1406299528

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值