重学 PyTorch 第二天:Tensor 的基本操作

重学 PyTorch Tensor 的基本操作

基本算数操作

作为一个张量计算库,算数操作是最基本的修养。PyTorch 中有一百来个算数操作算子。

逐元素操作算子

PyTorch 中,Tensor 的逐元素操作可分为函数式操作原位操作,在函数名中用 _ 后缀区分了二者,比如 tensor.abs() 是一个函数式操作,不改变 tensor 而将改变体现在返回值中,而 tensor.abs_() 是一个原位操作,既修改了原 tensor 又将这个修改后的值返回。

这里的操作都是逐元素进行的,元素之间没有关联。基本上每个 API 都有函数式和原位两个版本,也有作为 torch 模块的函数的过程化编程和作为 Tensor 的方法的面向对象式编程,其实面向过程和面向对象的唯一区别是第一个参数。每个算子一共有至少三种写法。

算子名说明
abs计算 input 中每个元素的绝对值.
absoluteAlias for torch.abs()
acosinverse cosine 大家熟悉的 arccos input.
arccosAlias for torch.acos().
acoshinverse hyperbolic cosine of input.
arccoshAlias for torch.acosh().
addAdds other, scaled by alpha, to input.
addcdivPerforms the element-wise division of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
addcmulPerforms the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input.
angleComputes the element-wise angle (in radians) of the given input tensor.
asinReturns a new tensor with the arcsine of the elements of input.
arcsinAlias for torch.asin().
asinhReturns a new tensor with the inverse hyperbolic sine of the elements of input.
arcsinhAlias for torch.asinh().
atanReturns a new tensor with the arctangent of the elements of input.
arctanAlias for torch.atan().
atanhReturns a new tensor with the inverse hyperbolic tangent of the elements of input.
arctanhAlias for torch.atanh().
atan2Element-wise arctangent of \text{input}{i} / \text{other}{i}inputi/otheri with consideration of the quadrant.
arctan2Alias for torch.atan2().
bitwise_notComputes the bitwise NOT of the given input tensor.
bitwise_andComputes the bitwise AND of input and other.
bitwise_orComputes the bitwise OR of input and other.
bitwise_xorComputes the bitwise XOR of input and other.
bitwise_left_shiftComputes the left arithmetic shift of input by other bits.
bitwise_right_shiftComputes the right arithmetic shift of input by other bits.
ceilReturns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element.
clampClamps all elements in input into the range [ min, max ].
clipAlias for torch.clamp().
conj_physicalComputes the element-wise conjugate of the given input tensor.
copysignCreate a new floating-point tensor with the magnitude of input and the sign of other, elementwise.
cosReturns a new tensor with the cosine of the elements of input.
coshReturns a new tensor with the hyperbolic cosine of the elements of input.
deg2radReturns a new tensor with each of the elements of input converted from angles in degrees to radians.
divDivides each element of the input input by the corresponding element of other.
divideAlias for torch.div().
digammaAlias for torch.special.digamma().
erfAlias for torch.special.erf().
erfcAlias for torch.special.erfc().
erfinvAlias for torch.special.erfinv().
expReturns a new tensor with the exponential of the elements of the input tensor input.
exp2Alias for torch.special.exp2().
expm1Alias for torch.special.expm1().
fake_quantize_per_channel_affineReturns a new tensor with the data in input fake quantized per channel using scale, zero_point, quant_min and quant_max, across the channel specified by axis.
fake_quantize_per_tensor_affineReturns a new tensor with the data in input fake quantized using scale, zero_point, quant_min and quant_max.
fixAlias for torch.trunc()
float_powerRaises input to the power of exponent, elementwise, in double precision.
floorReturns a new tensor with the floor of the elements of input, the largest integer less than or equal to each element.
floor_divide
fmodApplies C++’s std::fmod entrywise.
fracComputes the fractional portion of each element in input.
frexpDecomposes input into mantissa and exponent tensors such that input = mantissa × 2 exponent \text{input} = \text{mantissa} \times 2^{\text{exponent}} input=mantissa×2exponent.
gradientEstimates the gradient of a function g : R n → R g : \mathbb{R}^n \rightarrow \mathbb{R} g:RnR in one or more dimensions using the second-order accurate central differences method.
imagReturns a new tensor containing imaginary values of the self tensor.
ldexpMultiplies input by 2 other 2^\text{other} 2other.
lerpDoes a linear interpolation of two tensors start (given by input) and end based on a scalar or tensor weight and returns the resulting out tensor.
lgammaComputes the natural logarithm of the absolute value of the gamma function on input.
logReturns a new tensor with the natural logarithm of the elements of input.
log10Returns a new tensor with the logarithm to the base 10 of the elements of input.
log1pReturns a new tensor with the natural logarithm of (1 + input).
log2Returns a new tensor with the logarithm to the base 2 of the elements of input.
logaddexpLogarithm of the sum of exponentiations of the inputs.
logaddexp2Logarithm of the sum of exponentiations of the inputs in base-2.
logical_andComputes the element-wise logical AND of the given input tensors.
logical_notComputes the element-wise logical NOT of the given input tensor.
logical_orComputes the element-wise logical OR of the given input tensors.
logical_xorComputes the element-wise logical XOR of the given input tensors.
logitAlias for torch.special.logit().
hypotGiven the legs of a right triangle, return its hypotenuse.
i0Alias for torch.special.i0().
igammaAlias for torch.special.gammainc().
igammacAlias for torch.special.gammaincc().
mulMultiplies input by other.
multiplyAlias for torch.mul().
mvlgammaAlias for torch.special.multigammaln().
nan_to_numReplaces NaN, positive infinity, and negative infinity values in input with the values specified by nan, posinf, and neginf, respectively.
negReturns a new tensor with the negative of the elements of input.
negativeAlias for torch.neg()
nextafterReturn the next floating-point value after input towards other, elementwise.
polygammaAlias for torch.special.polygamma().
positiveReturns input.
powTakes the power of each element in input with exponent and returns a tensor with the result.
quantized_batch_normApplies batch normalization on a 4D (NCHW) quantized tensor.
quantized_max_pool1dApplies a 1D max pooling over an input quantized tensor composed of several input planes.
quantized_max_pool2dApplies a 2D max pooling over an input quantized tensor composed of several input planes.
rad2degReturns a new tensor with each of the elements of input converted from angles in radians to degrees.
realReturns a new tensor containing real values of the self tensor.
reciprocalReturns a new tensor with the reciprocal of the elements of input
remainderComputes Python’s modulus operation entrywise.
roundRounds elements of input to the nearest integer.
rsqrtReturns a new tensor with the reciprocal of the square-root of each of the elements of input.
sigmoidAlias for torch.special.expit().
signReturns a new tensor with the signs of the elements of input.
sgnThis function is an extension of torch.sign() to complex tensors.
signbitTests if each element of input has its sign bit set (is less than zero) or not.
sinReturns a new tensor with the sine of the elements of input.
sincAlias for torch.special.sinc().
sinhReturns a new tensor with the hyperbolic sine of the elements of input.
sqrtReturns a new tensor with the square-root of the elements of input.
squareReturns a new tensor with the square of the elements of input.
subSubtracts other, scaled by alpha, from input.
subtractAlias for torch.sub().
tanReturns a new tensor with the tangent of the elements of input.
tanhReturns a new tensor with the hyperbolic tangent of the elements of input.
true_divideAlias for torch.div() with rounding_mode=None.
truncReturns a new tensor with the truncated integer values of the elements of input.
xlogyAlias for torch.special.xlogy().

除了上述函数,PyTorch 还为我们重载了 Tensor 的很多运算符,这些运算符都符合 Python 的语义。PyTorch 的开发者为了 be Pythonic 还是费了相当大心思的,这也是 PyTorch 能很好融入 Python 生态的原因。

运算符说明
+add
-sub
*mul
/div
//truediv
@matmul

规约算子

规约是函数式编程中的重要概念。

算子名说明
argmaxReturns the indices of the maximum value of all elements in the input tensor.
argminReturns the indices of the minimum value(s) of the flattened tensor or along a dimension
amaxReturns the maximum value of each slice of the input tensor in the given dimension(s) dim.
aminReturns the minimum value of each slice of the input tensor in the given dimension(s) dim.
aminmaxComputes the minimum and maximum values of the input tensor.
allTests if all elements in input evaluate to True.
anyTests if any element in input evaluates to True.
maxReturns the maximum value of all elements in the input tensor.
minReturns the minimum value of all elements in the input tensor.
distReturns the p-norm of (input - other)
logsumexpReturns the log of summed exponentials of each row of the input tensor in the given dimension dim.
meanReturns the mean value of all elements in the input tensor.
nanmeanComputes the mean of all non-NaN elements along the specified dimensions.
medianReturns the median of the values in input.
nanmedianReturns the median of the values in input, ignoring NaN values.
modeReturns a namedtuple (values, indices) where values is the mode value of each row of the input tensor in the given dimension dim, i.e. a value which appears most often in that row, and indices is the index location of each mode value found.
normReturns the matrix norm or vector norm of a given tensor.
nansumReturns the sum of all elements, treating Not a Numbers (NaNs) as zero.
prodReturns the product of all elements in the input tensor.
quantileComputes the q-th quantiles of each row of the input tensor along the dimension dim.
nanquantileThis is a variant of torch.quantile() that “ignores” NaN values, computing the quantiles q as if NaN values in input did not exist.
stdIf unbiased is True, Bessel’s correction will be used.
std_meanIf unbiased is True, Bessel’s correction will be used to calculate the standard deviation.
sumReturns the sum of all elements in the input tensor.
uniqueReturns the unique elements of the input tensor.
unique_consecutiveEliminates all but the first element from every consecutive group of equivalent elements.
varIf unbiased is True, Bessel’s correction will be used.
var_meanIf unbiased is True, Bessel’s correction will be used to calculate the variance.
count_nonzeroCounts the number of non-zero values in the tensor input along the given dim.

比较算子

allcloseThis function checks if all input and other satisfy the condition:
argsortReturns the indices that sort a tensor along a given dimension in ascending order by value.
eqComputes element-wise equality
equalTrue if two tensors have the same size and elements, False otherwise.
geComputes \text{input} \geq \text{other}input≥other element-wise.
greater_equalAlias for torch.ge().
gtComputes \text{input} > \text{other}input>other element-wise.
greaterAlias for torch.gt().
iscloseReturns a new tensor with boolean elements representing if each element of input is “close” to the corresponding element of other.
isfiniteReturns a new tensor with boolean elements representing if each element is finite or not.
isinTests if each element of elements is in test_elements.
isinfTests if each element of input is infinite (positive or negative infinity) or not.
isposinfTests if each element of input is positive infinity or not.
isneginfTests if each element of input is negative infinity or not.
isnanReturns a new tensor with boolean elements representing if each element of input is NaN or not.
isrealReturns a new tensor with boolean elements representing if each element of input is real-valued or not.
kthvalueReturns a namedtuple (values, indices) where values is the k th smallest element of each row of the input tensor in the given dimension dim.
leComputes \text{input} \leq \text{other}input≤other element-wise.
less_equalAlias for torch.le().
ltComputes \text{input} < \text{other}input<other element-wise.
lessAlias for torch.lt().
maximumComputes the element-wise maximum of input and other.
minimumComputes the element-wise minimum of input and other.
fmaxComputes the element-wise maximum of input and other.
fminComputes the element-wise minimum of input and other.
neComputes \text{input} \neq \text{other}input=other element-wise.
not_equalAlias for torch.ne().
sortSorts the elements of the input tensor along a given dimension in ascending order by value.
topkReturns the k largest elements of the given input tensor along a given dimension.
msortSorts the elements of the input tensor along its first dimension in ascending order by value.

谱算子 Spectral Ops

stftShort-time Fourier transform (STFT).
istftInverse short time Fourier Transform.
bartlett_windowBartlett window function.
blackman_windowBlackman window function.
hamming_windowHamming window function.
hann_windowHann window function.
kaiser_windowComputes the Kaiser window with window length window_length and shape parameter beta.

杂类数学算子

atleast_1dReturns a 1-dimensional view of each input tensor with zero dimensions.
atleast_2dReturns a 2-dimensional view of each input tensor with zero dimensions.
atleast_3dReturns a 3-dimensional view of each input tensor with zero dimensions.
bincountCount the frequency of each value in an array of non-negative ints.
block_diagCreate a block diagonal matrix from provided tensors.
broadcast_tensorsBroadcasts the given tensors according to Broadcasting semantics.
broadcast_toBroadcasts input to the shape shape.
broadcast_shapesSimilar to broadcast_tensors() but for shapes.
bucketizeReturns the indices of the buckets to which each value in the input belongs, where the boundaries of the buckets are set by boundaries.
cartesian_prodDo cartesian product of the given sequence of tensors.
cdistComputes batched the p-norm distance between each pair of the two collections of row vectors.
cloneReturns a copy of input.
combinationsCompute combinations of length rr of the given tensor.
corrcoefEstimates the Pearson product-moment correlation coefficient matrix of the variables given by the input matrix, where rows are the variables and columns are the observations.
covEstimates the covariance matrix of the variables given by the input matrix, where rows are the variables and columns are the observations.
crossReturns the cross product of vectors in dimension dim of input and other.
cummaxReturns a namedtuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim.
cumminReturns a namedtuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim.
cumprodReturns the cumulative product of elements of input in the dimension dim.
cumsumReturns the cumulative sum of elements of input in the dimension dim.
diagIf input is a vector (1-D tensor), then returns a 2-D square tensor
diag_embedCreates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input.
diagflatIf input is a vector (1-D tensor), then returns a 2-D square tensor
diagonalReturns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape.
diffComputes the n-th forward difference along the given dimension.
einsumSums the product of the elements of the input operands along dimensions specified using a notation based on the Einstein summation convention.
flattenFlattens input by reshaping it into a one-dimensional tensor.
flipReverse the order of a n-D tensor along given axis in dims.
fliplrFlip tensor in the left/right direction, returning a new tensor.
flipudFlip tensor in the up/down direction, returning a new tensor.
kronComputes the Kronecker product, denoted by \otimes⊗, of input and other.
rot90Rotate a n-D tensor by 90 degrees in the plane specified by dims axis.
gcdComputes the element-wise greatest common divisor (GCD) of input and other.
histcComputes the histogram of a tensor.
histogramComputes a histogram of the values in a tensor.
histogramddComputes a multi-dimensional histogram of the values in a tensor.
meshgridCreates grids of coordinates specified by the 1D inputs in attr:tensors.
lcmComputes the element-wise least common multiple (LCM) of input and other.
logcumsumexpReturns the logarithm of the cumulative summation of the exponentiation of elements of input in the dimension dim.
ravelReturn a contiguous flattened tensor.
renormReturns a tensor where each sub-tensor of input along dimension dim is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm
repeat_interleaveRepeat elements of a tensor.
rollRoll the tensor input along the given dimension(s).
searchsortedFind the indices from the innermost dimension of sorted_sequence such that, if the corresponding values in values were inserted before the indices, when sorted, the order of the corresponding innermost dimension within sorted_sequence would be preserved.
tensordotReturns a contraction of a and b over multiple dimensions.
traceReturns the sum of the elements of the diagonal of the input 2-D matrix.
trilReturns the lower triangular part of the matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.
tril_indicesReturns the indices of the lower triangular part of a row-by- col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates.
triuReturns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.
triu_indicesReturns the indices of the upper triangular part of a row by col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates.
vanderGenerates a Vandermonde matrix.
view_as_realReturns a view of input as a real tensor.
view_as_complexReturns a view of input as a complex tensor.
resolve_conjReturns a new tensor with materialized conjugation if input’s conjugate bit is set to True, else returns input.
resolve_negReturns a new tensor with materialized negation if input’s negative bit is set to True, else returns input.

线性代数算子 BLAS

addbmmPerforms a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension).
addmmPerforms a matrix multiplication of the matrices mat1 and mat2.
addmvPerforms a matrix-vector product of the matrix mat and the vector vec.
addrPerforms the outer-product of vectors vec1 and vec2 and adds it to the matrix input.
baddbmmPerforms a batch matrix-matrix product of matrices in batch1 and batch2.
bmmPerforms a batch matrix-matrix product of matrices stored in input and mat2.
chain_matmulReturns the matrix product of the NN 2-D tensors.
choleskyComputes the Cholesky decomposition of a symmetric positive-definite matrix AA or for batches of symmetric positive-definite matrices.
cholesky_inverseComputes the inverse of a symmetric positive-definite matrix AA using its Cholesky factor uu: returns matrix inv.
cholesky_solveSolves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix uu.
dotComputes the dot product of two 1D tensors.
eigComputes the eigenvalues and eigenvectors of a real square matrix.
geqrfThis is a low-level function for calling LAPACK’s geqrf directly.
gerAlias of torch.outer().
innerComputes the dot product for 1D tensors.
inverseAlias for torch.linalg.inv()
detAlias for torch.linalg.det()
logdetCalculates log determinant of a square matrix or batches of square matrices.
slogdetAlias for torch.linalg.slogdet()
lstsqComputes the solution to the least squares and least norm problems for a full rank matrix AA of size (m \times n)(m×n) and a matrix BB of size (m \times k)(m×k).
luComputes the LU factorization of a matrix or batches of matrices A.
lu_solveReturns the LU solve of the linear system Ax = bA**x=b using the partially pivoted LU factorization of A from torch.lu().
lu_unpackUnpacks the data and pivots from a LU factorization of a tensor into tensors L and U and a permutation tensor P such that LU_data, LU_pivots = (P @ L @ U).lu().
matmulMatrix product of two tensors.
matrix_powerAlias for torch.linalg.matrix_power()
matrix_rankReturns the numerical rank of a 2-D tensor.
matrix_expAlias for torch.linalg.matrix_exp().
mmPerforms a matrix multiplication of the matrices input and mat2.
mvPerforms a matrix-vector product of the matrix input and the vector vec.
orgqrAlias for torch.linalg.householder_product().
ormqrComputes the matrix-matrix multiplication of a product of Householder matrices with a general matrix.
outerOuter product of input and vec2.
pinverseAlias for torch.linalg.pinv()
qrComputes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that \text{input} = Q Rinput=QR with QQ being an orthogonal matrix or batch of orthogonal matrices and RR being an upper triangular matrix or batch of upper triangular matrices.
svdComputes the singular value decomposition of either a matrix or batch of matrices input.
svd_lowrankReturn the singular value decomposition (U, S, V) of a matrix, batches of matrices, or a sparse matrix AA such that A \approx U diag(S) V^TAUdia**g(S)V**T.
pca_lowrankPerforms linear Principal Component Analysis (PCA) on a low-rank matrix, batches of such matrices, or sparse matrix.
symeigThis function returns eigenvalues and eigenvectors of a real symmetric or complex Hermitian matrix input or a batch thereof, represented by a namedtuple (eigenvalues, eigenvectors).
lobpcgFind the k largest (or smallest) eigenvalues and the corresponding eigenvectors of a symmetric positive definite generalized eigenvalue problem using matrix-free LOBPCG methods.
trapzAlias for torch.trapezoid().
trapezoidComputes the trapezoidal rule along dim.
cumulative_trapezoidCumulatively computes the trapezoidal rule along dim.
triangular_solveSolves a system of equations with a square upper or lower triangular invertible matrix AA and multiple right-hand sides bb.
vdotComputes the dot product of two 1D tensors.

下标操作

随机操作

序列化与反序列化

杂项

compiled_with_cxx11_abiReturns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1
result_typeReturns the torch.dtype that would result from performing an arithmetic operation on the provided input tensors.
can_castDetermines if a type conversion is allowed under PyTorch casting rules described in the type promotion documentation.
promote_typesReturns the torch.dtype with the smallest size and scalar kind that is not smaller nor of lower kind than either type1 or type2.
use_deterministic_algorithmsSets whether PyTorch operations must use “deterministic” algorithms.
are_deterministic_algorithms_enabledReturns True if the global deterministic flag is turned on.
is_deterministic_algorithms_warn_only_enabledReturns True if the global deterministic flag is set to warn only.
set_deterministic_debug_modeSets the debug mode for deterministic operations.
get_deterministic_debug_modeReturns the current value of the debug mode for deterministic operations.
set_float32_matmul_precisionSets the internal precision of float32 matrix multiplications.
get_float32_matmul_precisionReturns the current value of float32 matrix multiplication precision.
set_warn_alwaysWhen this flag is False (default) then some PyTorch warnings may only appear once per process.
is_warn_always_enabledReturns True if the global warn_always flag is turned on.
_assertA wrapper around Python’s assert which is symbolically traceable.

说明

PyTorch 的算子大多为异步惰性求值,尤其是 GPU 版本。

算子只定义了入口,真正的实现依赖于不同的后端,常用的有 cpucudamkldnnnpu 等等。

知道这些的意义是,不用管这些,爱怎么编程就怎么编就可以了,PyTorch 引擎会帮你搞定一切的。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值