- 博客(85)
- 资源 (1)
- 收藏
- 关注
原创 spark java.lang.OutOfMemoryError: Java heap space
spark java.lang.OutOfMemoryError: Java heap space经常遇到此问题,这个说得很详细http://stackoverflow.com/questions/21138751/spark-java-lang-outofmemoryerror-java-heap-space
2015-11-14 17:57:11
1861
原创 sql替换NOT IN语句
进来项目中用到很多sql的语句,其中有个需求:有两个表table1、table2,需要找出table1中存在,而table2中不存在的数据。要求不能使用NOT IN语句。举例:两表的数据如下: 现在要从table1中选出记录,记录的f3字段不在table2中出现。一、简单的使用NOT INselect f1,f2,f3 from tabl
2015-11-12 21:52:54
3833
原创 spark问题归纳
spark问题归纳一、说明以下遇到的spark问题都是本人在实际过程中碰到的,可能并不适合所有的情况,可做参考,有异议欢迎讨论。二、问题列表2.1 Application history not found (app-20150821200352-0022) No event logs found for application in hdfs:
2015-08-24 11:05:59
1069
原创 欢迎使用CSDN-markdown编辑器
欢迎使用Markdown编辑器写博客本Markdown编辑器使用StackEdit修改而来,用它写博客,将会带来全新的体验哦:Markdown和扩展Markdown简洁的语法代码块高亮图片链接和图片上传LaTex数学公式UML序列图和流程图离线写博客导入导出Markdown文件丰富的快捷键快捷键加粗 Ctrl + B 斜体 Ctrl + I 引用 Ctrl
2015-08-01 17:25:28
544
原创 Spark体系及RDD
Spark体系架构Spark体系架构包括如下三个主要组件:数据存储API管理框架接下来让我们详细了解一下这些组件。数据存储:Spark用HDFS文件系统存储数据。它可用于存储任何兼容于Hadoop的数据源,包括HDFS,HBase,Cassandra等。API:利用API,应用开发者可以用标准的API接口创建基于Spark的应用。
2015-07-31 20:42:18
900
原创 Spark API
Spark RDD API使用说明(一)1、aggregate1.1 函数声明defaggregate[U: ClassTag](zeroValue: U)(seqOp: (U, T) => U, combOp: (U, U)=> U): U1.2函数说明aggregate函数通过两个函数来操作RDD。第一个reduce函数(seqOp)对每个partition聚合,然后将初
2015-07-31 20:16:38
736
原创 Spark初探
Spark初探一、Spark简介及特性1.1什么是SparkApache Spark是一个围绕速度、易用性和复杂分析构建的大数据处理框架。最初在2009年由加州大学伯克利分校的AMPLab开发,并于2010年成为Apache的开源项目之一。Spark为我们提供了一个全面、统一的框架用于管理各种有着不同性质(文本数据、图表数据等)的数据集和数据源(批量数据或实时的流数据)的大
2015-07-31 19:57:18
693
原创 Spark streaming 初探
Spark streaming 初探 一、原理和运行场景Spark Streaming运行原理图如下:Spark Streaming把Kafka、HDFS、Socket等系统作为流处理的数据来源,把输入的数据流用时间切片的方式把数据分为一个个小的Batch,然后把这些小的Batch交给Spark引擎处理。官方wordcount示例代码:package o
2015-07-28 16:07:02
1024
原创 Word Ladder
Given two words (beginWord and endWord), and a dictionary, find the length of shortest transformation sequence from beginWord to endWord, such that:Only one letter can be changed at a timeEach int
2015-06-21 17:21:59
661
原创 LeetCode Word Break
LeetCode Word BreakGiven a string s and a dictionary of words dict, determine if s can be segmented into a space-separated sequence of one or more dictionary words.For example, givens = "l
2015-06-10 09:57:42
726
原创 LeetCode Maximal Square
LeetCode Maximal SquareGiven a 2D binary matrix filled with 0's and 1's, find the largest square containing all 1's and return its area.For example, given the following matrix:1 0 1 0 01
2015-06-05 15:06:40
574
原创 LeetCode Contains Duplicate II Contains Duplicate III
Contains Duplicate II Contains Duplicate III一、Contains Duplicate II Given an array of integers and an integer k, find out whether there there are two distinct indices i and j in the
2015-06-02 15:27:53
714
原创 LeetCode Course Schedule
LeetCode Course ScheduleThere are a total of n courses you have to take, labeled from 0 to n - 1.Some courses may have prerequisites, for example to take course 0 you have to first take co
2015-05-31 10:41:54
1088
原创 LeetCode Word Search II
LeetCode Word Search IIGiven a 2D board and a list of words from the dictionary, find all words in the board.Each word must be constructed from letters of sequentially adjacent cell, where "
2015-05-28 14:37:06
1155
原创 LeetCode Add and Search Word - Data structure design
Add and Search Word - Data structure design Design a data structure that supports the following two operations:void addWord(word)bool search(word)search(word) can search a literal word o
2015-05-21 14:59:53
703
原创 LeetCode Implement Trie (Prefix Tree)
LeetCode Implement Trie (Prefix Tree)Implement a trie with insert, search, and startsWith methods.Note:You may assume that all inputs are consist of lowercase letters a-z.题目要求是实现字典
2015-05-19 10:30:15
651
原创 LeetCode Minimum Size Subarray Sum
LeetCode Minimum Size Subarray SumGiven an array of n positive integers and a positive integer s, find the minimal length of a subarray of which the sum ≥ s. If there isn't one, return 0 inste
2015-05-16 22:03:16
855
原创 LeetCode LRU Cache
LeetCode LRU CacheDesign and implement a data structure for Least Recently Used (LRU) cache. It should support the following operations: get and set.get(key) - Get the value (will always be
2015-05-15 15:52:09
685
原创 vs2010 error link2019无法解析的外部符号
vs2010 error link2019无法解析的外部符号一、动机自己写的Dll库,然后对其进行引用,出现了题中所示的错误。使用了以下步骤的解决办法。1、引入dll的头文件:属性-》c/c++-》常规-》附加包含目录,在框中加入dll的头文件2、引入lib库目录:属性-》链接器-》常规-》附加库目录,在框中加入生成dll对应的lib库目录(注意这里加入的是文件目录)3、引
2015-05-11 21:36:07
9749
原创 LeetCode House Robber
LeetCode House RobberYou are a professional robber planning to rob houses along a street. Each house has a certain amount of money stashed, the only constraint stopping you from robbing each of
2015-05-10 17:08:03
757
原创 Windows下C/C++连接MySQL
Windows下C/C++连接MySQL一、安装MySQL二、连接MySQL本例子使用vs2010在Win7系统下连接本机MySQL服务器。1、用vs2010创建win32控制台应用程序2、配置工程2.1、配置头文件找到MySQL安装目录下的include文件夹:打开工程配置属性窗口--> C/C++ --> 常规 --> 附加包含目录,把mysql
2015-05-09 14:10:02
1154
原创 LeetCode:Distinct Subsequences
Distinct SubsequencesGiven a string S and a string T, count the number of distinct subsequences of T in S.A subsequence of a string is a new string which is formed from the original string b
2015-04-01 18:51:34
551
原创 LeetCode : Partition List
Partition List Given a linked list and a value x, partition it such that all nodes less than x come before nodes greater than or equal to x.You should preserve the original relative order of
2015-03-31 09:29:09
574
原创 LeetCode:Construct Binary Tree from Preorder and Inorder Traversal
Construct Binary Tree from Preorder and Inorder TraversalGiven preorder and inorder traversal of a tree, construct the binary tree.Note:You may assume that duplicates do not exist in the t
2015-03-29 08:58:14
947
原创 Binary Tree Maximum Path Sum
LeetCode :Binary Tree Maximum Path SumGiven a binary tree, find the maximum path sum.The path may start and end at any node in the tree.For example:Given the below binary tree,
2015-03-27 15:21:35
886
原创 LeetCode Min Stack
LeetCode Min StackDesign a stack that supports push, pop, top, and retrieving the minimum element in constant time.push(x) -- Push element x onto stack.pop() -- Removes the element on top of
2015-03-21 10:33:28
698
原创 LeetCode Repeated DNA Sequences
LeetCode Repeated DNA SequencesAll DNA is composed of a series of nucleotides abbreviated as A, C, G, and T, for example: "ACGAATTCCG". When studying DNA, it is sometimes useful to identify repe
2015-03-10 19:45:55
742
原创 LeetCode Rotate Array
LeetCode Rotate ArrayRotate Array Total Accepted: 5488 Total Submissions: 29811 My Submissions Question Solution Rotate an array of n elements to the right by k steps.For example, with n = 7
2015-03-03 15:16:48
1514
转载 跟我一起学习VIM - The Life Changing Editor
跟我一起学习VIM - The Life Changing Editor转载出处:http://ju.outofmemory.cn/entry/79671前两天同事让我在小组内部分享一下VIM,于是我花了一点时间写了个简短的教程。虽然准备有限,但分享过程中大家大多带着一种惊叹的表情,原来编辑器可以这样强大,这算是对我多年来使用VIM的最大鼓舞吧。所以分享结束之后,将这篇
2015-01-28 12:42:20
2011
原创 Rotate Image
LeetCode Rotate ImageYou are given an n x n 2D matrix representing an image.Rotate the image by 90 degrees (clockwise).Follow up:Could you do this in-place?关键地方在于就地。而不用新的内存。
2015-01-25 20:33:04
609
原创 Gray Code
LeetCode Gray CodeThe gray code is a binary numeral system where two successive values differ in only one bit.Given a non-negative integer n representing the total number of bits in the code
2015-01-22 13:55:01
674
原创 Unique Binary Search Trees
LeetCode Unique Binary Search Trees I IIⅠ第一题主要用到了数学推导的方法。令数组arr[i]表示i个节点时BST的数目。则arr[0]=arr[1]=1,arr[2] = 2.对于已知的arr[0...n]求arr[n+1],此n+1个节点中有个是根节点,令它的左子树个数L(n),右子数节点个数R(n),则左子树有arr[L(n)]中
2015-01-21 21:43:28
502
原创 Best Time to Buy and Sell Stock I II III
Best Time to Buy and Sell Stock I II IIII对于第一题,本质意义是求出数组中某两个元素(day[i],day[i])的最大差值。但是,要求j>iint maxProfit(vector &prices) { if(prices.size() ==0) return 0;
2015-01-19 23:44:10
703
原创 LeetCode Add Two Numbers
LeetCode Add Two NumbersYou are given two linked lists representing two non-negative numbers. The digits are stored in reverse order and each of their nodes contain a single digit. Add the two n
2015-01-07 19:56:13
635
原创 LeetCode Triangle
LeetCode TriangleGiven a triangle, find the minimum path sum from top to bottom. Each step you may move to adjacent numbers on the row below.For example, given the following triangle[
2015-01-04 16:10:53
599
原创 LeetCode Recover Binary Search Tree
LeetCode Recover Binary Search TreeTwo elements of a binary search tree (BST) are swapped by mistake.Recover the tree without changing its structure.Note:A solution using O(n) space is
2015-01-01 23:58:05
588
转载 Linux 中将VIM改装成强大C的IDE
By:吴垠Date:2007-09-07Version:0.5Email:lazy.fox.wu#gmail.comHomepage:http://blog.csdn.net/wooinCopyright:该文章版权由吴垠和他可爱的老婆小包子所有。可在非商业目的下任意传播和复制。对于商业目的下对本文的任何行为需经作
2014-12-31 11:36:29
1072
原创 LeetCode Longest Palindromic Substring
LeetCode Longest Palindromic SubstringGiven a string S, find the longest palindromic substring in S. You may assume that the maximum length of S is 1000, and there exists one unique longest pa
2014-12-30 00:17:29
670
原创 LeetCode Longest Common Prefix
LeetCode Longest Common PrefixWrite a function to find the longest common prefix string amongst an array of strings.很傻逼的逐个比较法。。string longestCommonPrefix(vector &strs) { string r
2014-12-23 17:34:34
792
原创 LeetCode Valid Parentheses
LeetCode Valid ParenthesesGiven a string containing just the characters '(', ')', '{', '}', '[' and ']', determine if the input string is valid.The brackets must close in the correct order
2014-12-21 17:45:58
476
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人