ibingoogle-CSDN博客

原创 BigDL 运行 LeNet5 on MNIST 发现的 BUG

1. 部署Hadoop 2.8.3, Spark 2.1.02. 编译和部署BigDL 编译command: ./make-dist.sh -P spark_2.x3. 运行脚本LeNet5-on-MNIST-example.sh:#!/bin/bash$SPARK_HOME/bin/spark-submit \ --master yarn \ --deploy-mode cli...

2018-03-08 02:32:11 731

原创 LeetCode 119. Pascal's Triangle II

题目链接：https://leetcode.com/problems/pascals-triangle-ii/Given an index k, return the kth row of the Pascal's triangle.For example, given k = 3,Return [1,3,3,1].基本思路：1. 关于什么是Pa

2017-01-07 02:36:33 247

原创 LeetCode 118. Pascal's Triangle

题目链接：https://leetcode.com/problems/pascals-triangle/Given numRows, generate the first numRows of Pascal's triangle.For example, given numRows = 5,Return[ [1], [1,1], [1,2,1],

2017-01-07 02:13:09 315

原创 LeetCode 150. Evaluate Reverse Polish Notation

题目链接：https://leetcode.com/problems/evaluate-reverse-polish-notation/Evaluate the value of an arithmetic expression in Reverse Polish Notation.Valid operators are +, -, *, /. Each operand m

2017-01-04 04:29:26 273

原创 Spark1.6.3 Driver端 task运行完成源码分析

0.背景0.1当完成task调度：Executor就会收到serialized的 task，先deserialize 出正常的task，然后运行task得到结果，并通过状态更新的RPC告知SchedulerBackend0.2这里主要分析GoarseGrainedSchedulerBackend.scala的情况1.GoarseGrainedSchedule

2017-01-03 11:04:30 456

原创 Spark1.6.3 on Yarn 调度总览

先放图再分析, 估计后续的2.0版本也差不多，应该不会有大变动0. Scheduling OverView1. Stage选择DAGScheduler.scala接收到一个Job提交后，会根据computing chain和 RDDs之间的dependencies自后向前建立DAG图并划分若干stages。然后从前向后依次提交stage，因为stages之间具有依赖关系，因

2017-01-02 10:56:03 581

原创 Spark1.6.3 Driver端 task调度源码分析

0.背景当一个Action操作触发了Spark Job提交后：1. Spark首先会在DAGScheduler.scala 中对其stages进行递归分析2. 然后针对某个可执行的stage，根据其partitions的大小生产tasks，并封装成TaskSet3. 最后调用taskSchedulerImpl.scala的submitTasks(taskSet) 方法发送backe

2017-01-02 08:53:26 988

原创 LeetCode 284. Peeking Iterator

题目链接： https://leetcode.com/problems/peeking-iterator/Given an Iterator class interface with methods: next() and hasNext(), design and implement a PeekingIterator that support the peek()ope

2016-07-19 14:04:04 235

原创 LeetCode 374. Guess Number Higher or Lower

题目链接： https://leetcode.com/problems/guess-number-higher-or-lower/We are playing the Guess Game. The game is as follows:I pick a number from 1 to n. You have to guess which number I picke

2016-07-18 00:51:12 330

原创 LeetCode 62. Unique Paths

题目链接：https://leetcode.com/problems/unique-paths/A robot is located at the top-left corner of a m x n grid (marked 'Start' in the diagram below).The robot can only move either down or right at

2016-07-17 04:12:59 194

原创 Linux下, Hadoop2.6.0源代码导入Eclipse

环境：Linux: Ubuntu14.04 Java: JDK1.7 Eclipse: eclipse-standard-kepler-SR2-linux-gtk-x86_64.tar.gz Hadoop：version-2.6.0 Maven: 3.0.51.安装JDK,Eclipse,Maven请自行百度，下

2016-01-14 15:30:00 1147

ibingoogle的博客