<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[Python_Billy的博客]]></title><description><![CDATA[]]></description><link>https://blog.csdn.net/Python_Billy</link><language>zh-cn</language><generator>https://blog.csdn.net/</generator><copyright><![CDATA[Copyright &copy; Python_Billy]]></copyright><item><title><![CDATA[ch.ethz.ssh2报错java.io.IOException: Key exchange was not finished, connection is closed.]]></title><link>https://blog.csdn.net/Python_Billy/article/details/131788800</link><guid>https://blog.csdn.net/Python_Billy/article/details/131788800</guid><author>Python_Billy</author><pubDate>Tue, 18 Jul 2023 15:40:51 +0800</pubDate><description><![CDATA[ch.ethz.ssh2连接远程服务器报错java.io.IOException: Key exchange was not finished, connection is closed.]]></description><category></category></item><item><title><![CDATA[ftp查看根目录对应的linux目录位置]]></title><link>https://blog.csdn.net/Python_Billy/article/details/128530675</link><guid>https://blog.csdn.net/Python_Billy/article/details/128530675</guid><author>Python_Billy</author><pubDate>Tue, 03 Jan 2023 11:40:49 +0800</pubDate><description><![CDATA[ftp查看用户对应目录]]></description><category></category></item><item><title><![CDATA[Caused by: org.apache.thrift.TApplicationException: Required field ‘filesAdded‘ is unset]]></title><link>https://blog.csdn.net/Python_Billy/article/details/123917003</link><guid>https://blog.csdn.net/Python_Billy/article/details/123917003</guid><author>Python_Billy</author><pubDate>Sat, 02 Apr 2022 11:54:20 +0800</pubDate><description><![CDATA[hive报错Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Required field ‘filesAdded’ is unset! Struct:InsertEventRequestData(filesAdded:null)
这个需要更改hive-site.xml里的hive.metadata.dml.events为false
完整报错如下：
jav]]></description><category></category></item><item><title><![CDATA[Spark dataset.mapPartitions debug到mapPartitions的函数体]]></title><link>https://blog.csdn.net/Python_Billy/article/details/123590520</link><guid>https://blog.csdn.net/Python_Billy/article/details/123590520</guid><author>Python_Billy</author><pubDate>Sat, 19 Mar 2022 13:38:58 +0800</pubDate><description><![CDATA[Dataset dataset = null;
StructType schema = dataset.schema();
Dataset&lt;Row&gt; rowDataset = dataset.mapPartitions(
        new MapPartitionsFunction&lt;Row, Row&gt;() {
            @Override
            public Iterator&lt;Row&gt; call(Iterator&lt;Row&gt;]]></description><category></category></item><item><title><![CDATA[IDEA中添加Scala插件后还是无法新增Scala类]]></title><link>https://blog.csdn.net/Python_Billy/article/details/123182979</link><guid>https://blog.csdn.net/Python_Billy/article/details/123182979</guid><author>Python_Billy</author><pubDate>Mon, 28 Feb 2022 15:10:19 +0800</pubDate><description><![CDATA[如题，解决方法如下：
1、右键点击项目名，找到Add Framework Support

2、点击Add Framework Support，下拉选择Scala

3、在右边自己选择Create选择自己想要的Scala版本

之后在new class就有Scala类了

]]></description><category></category></item><item><title><![CDATA[Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:pom:5.1.5-jhyde in nexus-aliyun (h]]></title><link>https://blog.csdn.net/Python_Billy/article/details/121748965</link><guid>https://blog.csdn.net/Python_Billy/article/details/121748965</guid><author>Python_Billy</author><pubDate>Mon, 06 Dec 2021 15:56:35 +0800</pubDate><description><![CDATA[maven打包hive-exec报如下错误：
Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:pom:5.1.5-jhyde in nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public)
网上找的好几个都是说换掉maven的仓库，说是换成
&lt;/mirror&gt;
    &lt;mirror&gt;
    &lt;id&gt;aliyun]]></description><category></category></item><item><title><![CDATA[json.dumps遇到TypeError: Object of type date is not JSON serializable]]></title><link>https://blog.csdn.net/Python_Billy/article/details/115654639</link><guid>https://blog.csdn.net/Python_Billy/article/details/115654639</guid><author>Python_Billy</author><pubDate>Tue, 13 Apr 2021 09:48:08 +0800</pubDate><description><![CDATA[从数据库读取数据然后转为json时遇到：
TypeError: Object of type date is not JSON serializable
TypeError: Object of type Decimal is not JSON serializable
等错误，发现需要在json.dumps()时传入一个参数，如下
class DateEncoder(json.JSONEncoder):
    def default(self, obj):
        # 处理返回数据中有date类]]></description><category></category></item><item><title><![CDATA[rdd.collect()后得到的List排序问题：Exception in thread “main“ java.lang.UnsupportedOperationException]]></title><link>https://blog.csdn.net/Python_Billy/article/details/115654206</link><guid>https://blog.csdn.net/Python_Billy/article/details/115654206</guid><author>Python_Billy</author><pubDate>Tue, 13 Apr 2021 09:39:14 +0800</pubDate><description><![CDATA[跟着视频的写了一个数据统计的demo，实现按小时统计访问量。代码如下
package com.billy.test;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spar]]></description><category></category></item><item><title><![CDATA[Hive -e时的双引号和反斜杠问题]]></title><link>https://blog.csdn.net/Python_Billy/article/details/115285066</link><guid>https://blog.csdn.net/Python_Billy/article/details/115285066</guid><author>Python_Billy</author><pubDate>Sun, 28 Mar 2021 18:56:53 +0800</pubDate><description><![CDATA[今天用hive -e 测试split(column, “\.”)的时候发现有个坑，记录一下。
建表语句：
create table movie (
name string,
category string
)
row format delimited fields terminated by "\t";

数据准备movie.txt：
《疑犯追踪》	悬疑.动作.科幻.剧情
《Lie to me》	悬疑.警匪.动作.心理.剧情
《战狼 2》	战争.动作.灾难

插入语句：
load data local in]]></description><category></category></item><item><title><![CDATA[Hive一些常用配置【持续更新】]]></title><link>https://blog.csdn.net/Python_Billy/article/details/115269401</link><guid>https://blog.csdn.net/Python_Billy/article/details/115269401</guid><author>Python_Billy</author><pubDate>Sat, 27 Mar 2021 18:28:50 +0800</pubDate><description><![CDATA[&lt;!--是否以本地模式启动，默认false--&gt;
&lt;property&gt;
	&lt;name&gt;hive.exec.mode.local.auto&lt;/name&gt;
	&lt;value&gt;false&lt;/value&gt;
&lt;/property&gt;

&lt;!--让提示符显示当前库，默认false--&gt;
&lt;property&gt;
	&lt;name&gt;hive.cli.print.current.db&lt;/name&gt;
	&l]]></description><category></category></item><item><title><![CDATA[Hadoop添加snappy压缩支持]]></title><link>https://blog.csdn.net/Python_Billy/article/details/115168522</link><guid>https://blog.csdn.net/Python_Billy/article/details/115168522</guid><author>Python_Billy</author><pubDate>Wed, 24 Mar 2021 11:38:31 +0800</pubDate><description><![CDATA[启动hadoop后，使用如下命令查看压缩支持：
hadoop checknative

出现下列输出，snappy为false，则需要手动添加snappy支持
Native library checking:
hadoop:  true /opt/bigdata/hadoop/hadoop-3.1.4/lib/native/libhadoop.so.1.0.0
zlib:    true /lib/x86_64-linux-gnu/libz.so.1
zstd  :  true /lib/x86_64-li]]></description><category></category></item><item><title><![CDATA[Hadoop四个配置文件]]></title><link>https://blog.csdn.net/Python_Billy/article/details/114550213</link><guid>https://blog.csdn.net/Python_Billy/article/details/114550213</guid><author>Python_Billy</author><pubDate>Mon, 08 Mar 2021 21:23:54 +0800</pubDate><description><![CDATA[重装了下单机版的hadoop，基本配置文件忘了，这里记录一下免得以后总得找
core-site.xml:
&lt;configuration&gt;
    &lt;!-- 用来指定hdfs的老大（NameNode）的地址 --&gt;
    &lt;property&gt;
		&lt;name&gt;fs.defaultFS&lt;/name&gt;
		&lt;value&gt;hdfs://localhost:9000&lt;/value&gt;
	&lt;/property&gt;
	  &lt]]></description><category></category></item><item><title><![CDATA[LeetCode刷题：134. 加油站问题]]></title><link>https://blog.csdn.net/Python_Billy/article/details/114271674</link><guid>https://blog.csdn.net/Python_Billy/article/details/114271674</guid><author>Python_Billy</author><pubDate>Mon, 01 Mar 2021 21:27:29 +0800</pubDate><description><![CDATA[134. 加油站
难度：中等
来源：力扣（LeetCode）
链接：https://leetcode-cn.com/problems/gas-station
著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。
问题描述：
在一条环路上有 N 个加油站，其中第 i 个加油站有汽油 gas[i] 升。
你有一辆油箱容量无限的的汽车，从第 i 个加油站开往第 i+1 个加油站需要消耗汽油 cost[i] 升。你从其中的一个加油站出发，开始时油箱为空。
如果你可以绕环路行驶一周，则返回出发时加油站]]></description><category></category></item><item><title><![CDATA[ArcGIS Server Linux 10.7压缩包]]></title><link>https://blog.csdn.net/Python_Billy/article/details/113498391</link><guid>https://blog.csdn.net/Python_Billy/article/details/113498391</guid><author>Python_Billy</author><pubDate>Mon, 01 Feb 2021 10:07:58 +0800</pubDate><description><![CDATA[ArcGIS Server Linux 10.7压缩包
链接：https://pan.baidu.com/s/1GYjElpbXZN0938_vWyWSOA
提取码：nkbp

]]></description><category></category></item><item><title><![CDATA[beeline连接hive报错：Error: Could not open client transport with JDBC Uri: jdbc:hive2://.....]]></title><link>https://blog.csdn.net/Python_Billy/article/details/105063337</link><guid>https://blog.csdn.net/Python_Billy/article/details/105063337</guid><author>Python_Billy</author><pubDate>Tue, 24 Mar 2020 08:17:09 +0800</pubDate><description><![CDATA[beeline连接hive2的时候报错，报错信息为Error: Could not open client transport with JDBC Uri: jdbc:hive2://192.168.81.131:10000: java.net.ConnectException: 拒绝连接 (Connection refused) (state=08S01,code=0)
google搜了很久试了...]]></description><category></category></item><item><title><![CDATA[hadoop启动HDFS命令]]></title><link>https://blog.csdn.net/Python_Billy/article/details/104796165</link><guid>https://blog.csdn.net/Python_Billy/article/details/104796165</guid><author>Python_Billy</author><pubDate>Wed, 11 Mar 2020 14:28:06 +0800</pubDate><description><![CDATA[启动命令：/hadoop/sbin/start-dfs.sh

停止命令：/hadoop/sbin/stop-dfs.sh
]]></description><category></category></item><item><title><![CDATA[JAVA, SCALA, HADOOP, SPARK里的JAVA_HOME和环境变量设置]]></title><link>https://blog.csdn.net/Python_Billy/article/details/104790131</link><guid>https://blog.csdn.net/Python_Billy/article/details/104790131</guid><author>Python_Billy</author><pubDate>Wed, 11 Mar 2020 09:57:16 +0800</pubDate><description><![CDATA[/etc/profile里添加的内容：

export JAVA_HOME=/jdk
export HADOOP_HOME=/hadoop
export SCALA_HOME=/scala
export SPARK_HOME=/spark
export PATH=$PATH:$JAVA_HOME/bin
export PATH=$PATH:$SPARK_HOME/bin
export PATH=$...]]></description><category></category></item><item><title><![CDATA[Windows下安装Python的Scrapy模块]]></title><link>https://blog.csdn.net/Python_Billy/article/details/98475609</link><guid>https://blog.csdn.net/Python_Billy/article/details/98475609</guid><author>Python_Billy</author><pubDate>Sun, 04 Aug 2019 22:28:25 +0800</pubDate><description><![CDATA[Windows下Scrapy安装教程：

pip install wheel(也可能不需要)
pip install lxml
pip install pyOpenSSL
进入官网：http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted ，找到 Twisted-19.2.1-cp36-cp36m-win_amd64.whl ，（这里同样找到对应自己的p...]]></description><category></category></item><item><title><![CDATA[Python json中文显示unicode字符]]></title><link>https://blog.csdn.net/Python_Billy/article/details/98070290</link><guid>https://blog.csdn.net/Python_Billy/article/details/98070290</guid><author>Python_Billy</author><pubDate>Thu, 01 Aug 2019 16:11:35 +0800</pubDate><description><![CDATA[Python json文件
当使用json.dumps将文档转化为json格式时，文档内容里有中文，这样会导致显示成\u95f9\u949f这样，因此需要在dumps函数里设置参数ensure_ascii=False即可。

...]]></description><category></category></item></channel></rss>