spark | scala | 线性代数库Breeze学习

最新推荐文章于 2023-07-27 19:27:35 发布

茁壮小草

最新推荐文章于 2023-07-27 19:27:35 发布

阅读量5.6k

点赞数 2

文章标签： spark scala Breeze

本文链接：https://blog.csdn.net/xxzhangx/article/details/74066679

版权

本文介绍了在Spark开发中如何利用Breeze库进行线性代数运算，包括矩阵、向量的基本操作，如创建、转换、元素访问及赋值，以及数值计算、线性代数函数的使用，是学习Spark数学库的好资料。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

最近在学习spark，对线性代数库Breeze做了学习，介绍下常用的函数

前提，使用Breeze库，必须导入相关的包

import breeze.linalg._
import breeze.numerics._

最基础的操作：矩阵，向量，数组的转换

1、DenseMatrix.zerosDouble

全为零的n*m的矩阵，Double类型

2、DenseVector.zerosDouble

全为零的n个数组成的向量，Double类型

3、DenseVector.onesDouble

全为1的n个数组成的向量，Double类型

4、DenseVector.fill(n){5.0}

产生向量，长度为n，用5.0来填充

5、DenseVector.range(start,stop,step)
DenseVector.rangeD(start,stop,step)

产生序列向量
6、DenseVector.linspace(start,stop,numvals)

产生向量，有numvals个数的向量

7、DenseMatrix.eyeDouble

产生n*n的矩阵，对角为1，Double类型

8、diag(DenseVector(1.0,2.0,3.0))

产生主对角元素为1.0,2.0,3.0的矩阵

9、DenseMatrix((1.0,2.0),(3.0,4.0))

产生矩阵

10、DenseVector(1,2,3,4)

产生向量

11、DenseVector(1,2,3,4).t

向量转置

12、DenseVector.tabulate(3){i => 2*i}

scala> DenseVector.tabulate(3){i => 2*i}

res33: breeze.linalg.DenseVector[Int] = DenseVector(0, 2, 4)

结果为:0,2,4

13、DenseMatrix.tabulate(3,2){case(i,j) => i+j}

scala> DenseMatrix.tabulate(3,2){case(i,j) => i+j}
res34: breeze.linalg.DenseMatrix[Int] =
0  1
1  2
2  3

行列数相加

14、new DenseVector(Array(1,2,3,4))
从数组创建向量

scala> new DenseVector(Array(1,2,3,4))
res35: breeze.linalg.DenseVector[Int] = DenseVector(1, 2, 3, 4)

15、new DenseMatrix(2,3,Array(11,12,13,21,22,23))
从数组创建矩阵

scala> new DenseMatrix(2,3,Array(11,12,13,21,22,23))
res36: breeze.linalg.DenseMatrix[Int] =
11  13  22
12  21  23

16、DenseVector.rand(4)
得到0到1的随机向量，长度为4

scala> DenseVector.rand(4)
res37: breeze.linalg.DenseVector[Double] = DenseVector(0.9838289972536518, 0.798555117073358, 0.30308183931925403, 0.7958095551517774)

17、DenseMatrix.rand(2,3)
得到0到1的随机矩阵

scala> DenseMatrix.rand(2,3)
res38: breeze.linalg.DenseMatrix[Double] =
0.3891370890132193  0.06732600444704517  0.2136759825764527
0.587145241786718   0.8670050354290917   0.5494899108312414

Breeze元素访问

1、指定位置

scala> val a = DenseVector(1,2,3,4,5)
a: breeze.linalg.DenseVector[Int] = DenseVector(1, 2, 3, 4, 5)

scala> a(2)
res44: Int = 3

2、向量子集

scala> a(1 to 2)
res40: breeze.linalg.DenseVector[Int] = DenseVector(2, 3)

scala> a(1 until 2)
res41: breeze.linalg.DenseVector[Int] = DenseVector(2)

scala> a.slice(1,2)
res42: breeze.linalg.DenseVector[Int] = DenseVector(2)

3、按照指定步长取子集

scala> a(3 to 1 by -1)
res45: breeze.linalg.DenseVector[Int] = DenseVector(4, 3, 2)

4、指定开始位置至结尾

scala> a(2 to -1 )
res48: breeze.linalg.DenseVector[Int] = DenseVector(3, 4, 5)

5、最后一个元素

scala> a(2 to -1 )
res48: breeze.linalg.DenseVector[Int] = DenseVector(3, 4, 5)

6、矩阵指定列

scala> val a = DenseMatrix((1,2,3,4,5),(3,4,5,6,7),(5,6,7,8,9))
a: breeze.linalg.DenseMatrix[Int] =
1  2  3  4  5
3  4  5  6  7
5  6  7  8  9

scala> a(::,2)
res1: breeze.linalg.DenseVector[Int] = DenseVector(3, 5, 7)

Breeze元素操作

1、a.reshape(3,2)
调整矩阵形状

scala> val a = DenseMatrix((2,3),(3,4),(6,7))
a: breeze.linalg.DenseMatrix[Int] =
2  3
3  4
6  7


scala> val a = DenseMatrix((2,3),(3,4),(6,7))
a: breeze.linalg.DenseMatrix[Int] =
2  3
3  4
6  7

2、a.toDenseVector
矩阵转成向量

scala> val a = DenseMatrix((2,3),(3,4),(6,7))
a: breeze.linalg.DenseMatrix[Int] =
2  3
3  4
6  7

scala> a.toDenseVector
res3: breeze.linalg.DenseVector[Int] = DenseVector(2, 3, 6, 3, 4, 7)

3、lowerTriangular
下三角矩阵

scala> val b = DenseMatrix((1,2,3,4,5,6),(2,3,4,5,6,7),(3,4,5,6,7,8))
b: breeze.linalg.DenseMatrix[Int] =
1  2  3  4  5  6
2  3  4  5  6  7
3  4  5  6  7  8

scala> lowerTriangular(b)
res7: breeze.linalg.DenseMatrix[Int] =
1  0  0
2  3  0
3  4  5

4、upperTriangular
上三角矩阵

scala> upperTriangular(b)
res8: breeze.linalg.DenseMatrix[Int] =
1  2  3
0  3  4
0  0  5

5、b.copy
复制矩阵

scala> b.copy
res9: breeze.linalg.DenseMatrix[Int] =
1  2  3  4  5  6
2  3  4  5  6  7
3  4  5  6  7  8

6、diag(a)
取对角线元素

scala> val c = DenseMatrix((1,2,3),(2,3,4),(3,4,5))
c: breeze.linalg.DenseMatrix[Int] =
1  2  3
2  3  4
3  4  5

scala> diag(c)
res12: breeze.linalg.DenseVector[Int] = DenseVector(1, 3, 5)

7、c(1 to 4 ) := 5.0
子集赋值，将c中的第2个数到第五个数赋值为5.0

scala> val d = DenseVector(1,2,3,4,5,6,7,8,8)
d: breeze.linalg.DenseVector[Int] = DenseVector(1, 2, 3, 4, 5, 6, 7, 8, 8)

scala> d(1 to 4) :=  5
res18: breeze.linalg.DenseVector[Int] = DenseVector(5, 5, 5, 5)

scala> d
res21: breeze.linalg.DenseVector[Int] = DenseVector(1, 5, 5, 5, 5, 6, 7, 8, 8)

8、d(1 to 4) := DenseVector(1,2,3)
子集赋向量

scala> d(1 to 4):=DenseVector(1,2,3,4)
res26: breeze.linalg.DenseVector[Int] = DenseVector(1, 2, 3, 4)

scala> d
res27: breeze.linalg.DenseVector[Int] = DenseVector(1, 1, 2, 3, 4, 6, 7, 8, 8)

9、a(1 to 3,1 to 3) :=5
矩阵赋值

scala> b(1 to 2,1 to 3):= 6
res30: breeze.linalg.DenseMatrix[Int] =
6  6  6
6  6  6

scala> b
res31: breeze.linalg.DenseMatrix[Int] =
1  2  3  4  5  6
2  6  6  6  6  7
3  6  6  6  7  8

10、a(::,2) := 5
矩阵列赋值

scala> b(::,2) := 7
res32: breeze.linalg.DenseVector[Int] = DenseVector(7, 7, 7)

scala> b
res33: breeze.linalg.DenseMatrix[Int] =
1  2  7  4  5  6
2  6  7  6  6  7
3  6  7  6  7  8