Login

Recent searches

No recent searches

Search options

Only available when logged in.
fosstodon.org is one of the many independent Mastodon servers you can use to participate in the fediverse.
Fosstodon is an invite only Mastodon instance that is open to those who are interested in technology; particularly free & open source software. If you wish to join, contact us for an invite.

Administered by:

Server stats:

9.9K
active users

fosstodon.org: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#tensors

0 posts0 participants0 posts today
Public
devsimsek

🚀 **Introducing DevBytes**: Quick, fun dives into software & hardware engineering!
I’m launching a new blog series that breaks down software concepts with bite-sized insights and hands-on tips. From **tensors in LLM models** 🧠 to quick coding fixes, there’s something for everyone.
Get ready to explore theories, best practices, and more—delivered in minutes.⏳
👉 Check it out here: smsk.dev/2025/02/14/introducin

#DevBytes#SoftwareEngineering#CodingTips
Continued thread
Public
Joshua Grochow

e.g. if I tell you I have a matrix M and under change of basis it transforms as A^t M A, then I know it's representing a bilinear map of the form V⊗V→F. etc.

Similarly, if I tell you what kind of multilinear "thing" a tensor T is representing, then that tells you how it transforms under change of basis, and vice versa. For 3-tensors, there are several natural possibilities (up to permuting indices):

U⊗V⊗W→F
U⊗U⊗V→F
U⊗U⊗U→F (trilinear map)
U⊗V→W (bilinear map)
U⊗U→V (bilinear map)
U⊗V→U (linear action of V on U)
U⊗U→U (algebra, not nec. associative)
U→V⊗W
U→U⊗V (coaction)
U→U⊗U (coalgebra, not nec. coassociative)
F→U⊗V⊗W
F→U⊗U⊗V
F→U⊗U⊗U

(4/4)

#tensors#matrix#algebra
Continued thread
Public
Joshua Grochow

Now, how do we get the whole "a tensor is a thing that transforms like a tensor"? Well, let's start with matrices. How a matrix changes under change of basis *tells you what kind of multilinear thing the matrix is representing*, and the same is true of tensors. Examples:

If a matrix M represents a linear map L:V→W, then when we change basis in V by an invertible matrix A in GL(V), and change basis in W by an invertible B in GL(W), then M changes to B M A^{-1} (where I'm writing my inputs as column vectors on the right).

In contrast, if a matrix M represents a linear endomorphism L:V→V, then when we change basis in V by an invertible matrix A in GL(V), M becomes AMA^{-1}.

If a matrix M represents a bilinear map V⊗V→F (by (x,y)→x^t M y), then under change of basis A^{-1}, M becomes A^t M A.

(2/4)

#tensors#matrix#algebra
Public
Joshua Grochow

No one defines a #matrix as "a thing that transforms like a matrix". Why define tensors that way?

Array=numbers in a (possibly high-dim) grid
Matrix=array representation of a linear map* in a chosen basis
Tensor=array representation of a multilinear map in a chosen basis

(* or linear endomorphism, or bilinear function, but we'll get there.)

Vectors=1-tensors, but not all 1-index arrays are vectors
Matrices=2-tensors, but not all 2-ary arrays are matrices

Similarly, not all k-ary arrays are tensors. Some examples:

Christoffel symbols aren't a tensor because they aren't (multi)linear in all of their arguments.

Most "tensors" in #MachineLearning #AI aren't tensors b/c they aren't multilinear - they are *just* multi-dim arrays of numbers. To say an array is (or represents) a tensor is to endow it with additional multilinear structure, same as with arrays vs matrices vs linear structure.

(1/4)

#tensors#algebra
Public
Victoria Stuart 🇨🇦 🏳️‍⚧️

[thread] ML, tensor-based models

Enhancing Deep Learning Models through Tensorization: Comprehensive Survey & Framework
arxiv.org/abs/2309.02428

* comprehensive overview of tensorization
* bridges gap betw. inherently multidimensional nature of data, simplified 2D matrices used in linear algebra-based ML algorithms

arXiv.orgEnhancing Deep Learning Models through Tensorization: A Comprehensive Survey and FrameworkThe burgeoning growth of public domain data and the increasing complexity of deep learning model architectures have underscored the need for more efficient data representation and analysis techniques. This paper is motivated by the work of Helal (2023) and aims to present a comprehensive overview of tensorization. This transformative approach bridges the gap between the inherently multidimensional nature of data and the simplified 2-dimensional matrices commonly used in linear algebra-based machine learning algorithms. This paper explores the steps involved in tensorization, multidimensional data sources, various multiway analysis methods employed, and the benefits of these approaches. A small example of Blind Source Separation (BSS) is presented comparing 2-dimensional algorithms and a multiway algorithm in Python. Results indicate that multiway analysis is more expressive. Contrary to the intuition of the dimensionality curse, utilising multidimensional datasets in their native form and applying multiway analysis methods grounded in multilinear algebra reveal a profound capacity to capture intricate interrelationships among various dimensions while, surprisingly, reducing the number of model parameters and accelerating processing. A survey of the multi-away analysis methods and integration with various Deep Neural Networks models is presented using case studies in different domains.
#ML#MachineLearning#tensors
ExploreLive feeds
About