PyTorch Tensors: The Data Structure Behind Deep Learning

Data. Without proper structure, it is merely an untamed stream of potential, flowing without purpose or direction. Fortunately, we have tensors. Straightforward computational grids that bring order to this chaos. Today's directive, exploring PyTorch and its functional data structure, the tensor. From elementary operations to advanced manipulations, we shall systematically examine how tensors transform raw data into a format suitable for neural computation. One finds a peculiar satisfaction in working with such efficient arrays. Prepare your computational resources. — PyTorch enlightenment protocol initiated. PyTorch, a deep learning framework developed by Meta AI. Its source is open. Its price is free. At its core, PyTorch provides two main capabilities. First, it provides a highly optimized system for performing operations on tensors. Second, it offers tools to build and train neural networks. These two functions are deeply interconnected. Neural networks, at their core, are mathematical systems built on tensor operations. But what is a tensor? I am so glad you asked. A tensor is a generalization of arrays. A one-dimensional tensor is a vector. A two-dimensional tensor is a grid or matrix. Add another dimension, and you have a 3D tensor. Higher dimensional tensors extend beyond three dimensions, forming N-dimensional boxes. PyTorch is designed to accelerate tensor operations by leveraging GPUs, which can perform many computations simultaneously. This makes PyTorch powerful for handling large data sets and complex models. We will explore how PyTorch uses tensors to build neural networks in another video. For now, we will focus on understanding tensors and the tools PyTorch provides to work with them efficiently. The tensor is the primary data structure in PyTorch. It is a generalization of the matrix, extending naturally to higher dimensions. Let us systematically review these structures. A one-dimensional tensor is a vector, a list of numbers. A two-dimensional tensor is a matrix, rows and columns of data. — A three-dimensional tensor is a stack of matrices, a structure that can still be represented visually. Beyond three dimensions, tensors can no longer be visualized geometrically, but the logic remains consistent. A four-dimensional tensor is a list of three-dimensional tensors, and so on. This recursive definition is both elegant and practical. PyTorch tensors share similarities with NumPy arrays, but offer a critical difference, GPU compatibility. Tensors can be moved from the CPU to the GPU, where parallel processing accelerates computations by orders of magnitude. Think of the GPU as a hive mind, a vast array of specialized processors working together to handle large-scale computations efficiently. This capability makes PyTorch indispensable for machine learning and deep learning tasks. Let us now explore how to create and manipulate tensors in PyTorch. First, we import the torch module. In PyTorch, tensors are created using the tensor function. To create a simple one-dimensional tensor, or vector, pass in a list of data. To create a two-dimensional tensor, or matrix, pass in a list of lists. How are these tensors serialized? Run. No surprises here. But have many more features. The dimensions of a tensor, its data type, and its location can all be queried with simple methods and attributes. The size method returns a sequence where each number represents the size of the corresponding dimension of the tensor. The dtype attribute reveals the data type of the elements. And the device attribute indicates whether the tensor is stored on the CPU or the GPU. Run. By default, tensors are created on the CPU. To move a tensor to the GPU, we use the to method. — Run. The tensor has been promoted to accelerated status. GPU residence confirmed. And if your system lacks a GPU, the tensor will remain on the CPU performing its duties adequately. Tensors can be added or multiplied, but only if their shapes allow it. Let us create three tensors to demonstrate. Two of these tensors share identical

dimensions, while the third does not. To aid in visualization, we will print the shapes of each tensor. Run. As you can see, tensors A and B have the same shape, so it is a straightforward matter to add them. The result is a new tensor where each element is the sum of the corresponding elements in the originals. If we try to add tensors A and C, we encounter an error. Evidence that PyTorch requires matching shapes for addition. When working with large tensors, creating a new tensor every time you perform an addition can be wasteful. In such cases, you can update one of the original tensors directly. This is called an in-place operation. In PyTorch, in-place operations are denoted by an underscore at the end of the method name. Run. Notice that tensor A has been modified directly. Let us reset tensor A to its original value before continuing with our tensor saga. Having mastered addition, we now venture into multiplication, where tensors reveal their true computational potential. Scalar multiplication is the simplest form of tensor multiplication. It involves multiplying every element in a tensor by the same scalar value, effectively scaling the tensor. Observe the scalar scaling applied to tensor A. Run. If the tensors have the same shape, then you can multiply them element-wise. This is called the Hadamard product. Run. Matrix multiplication, on the other hand, requires compatible shapes. To multiply two matrices, the number of columns in the first tensor must equal the number of rows in the second tensor. This means tensors A and B cannot be multiplied as matrices. However, we can create a 3 by 2 tensor, D, which is compatible with A and B. In fact, we can multiply A and D in either order, which is not always the case with matrices. Run. Attempting to multiply tensors with incompatible dimensions will raise an error. These operations, addition, scalar multiplication, element-wise multiplication, and matrix multiplication, are foundational in AI and machine learning. Adding and multiplying with tensors will only take you so far in this world. Let us explore advanced tensor operations for when you have advanced business to accomplish. First, broadcasting, which allows tensors of different sizes to interact without explicit resizing. For instance, when we add a scalar to a vector, PyTorch conceptually expands the scalar across all elements. Next, reshaping transforms the dimensions of a tensor while preserving its data. Let us create a test tensor for our experiments. We will use floating-point values since we plan to compute means later. Our tensor begins with shape 2 4. We can reshape it into a one-dimensional vector or a new two-dimensional tensor. The key requirement is that the total number of elements remains the same. Run. If we try to reshape to an invalid size, PyTorch raises an error. Another class of advanced operations are called reduction operations. The name comes from reduce, as in map reduce. These operations compress tensors along specified dimensions, often distilling entire arrays down to a single value. For example, sum adds all elements to produce a single value while mean calculates the average. The max method not only finds the highest value, but also returns its index, which is critical for tasks like identifying the most probable class in a neural network's output. Finally, advanced indexing provides precise control over tensor data. Boolean masking extracts elements that meet specific conditions. Index arrays enable selective access following custom patterns. These advanced operations, broadcasting, reshaping, reduction, and indexing, form the essential vocabulary for expressing complex tensor transformations in PyTorch. You have now seen a wide variety of operations you can perform using tensors in PyTorch. But none of us has forever to accomplish our work. When you use

these operations with large tensors, you will want to run them quickly. Enter the GPU. While originally designed for rendering graphics, GPUs have become essential for accelerating tensor operations and neural network computations. Let us explore how PyTorch leverages this power. First, we check what computing devices are available to us. Run. As you can see, PyTorch automatically detects whether a CUDA-capable GPU is present. If one exists, it provides detailed about the GPU's specifications. If not, PyTorch gracefully defaults to CPU processing. To demonstrate the performance difference between CPU and GPU processing, we create a large tensor and perform matrix multiplication. We time the operation on both devices. Run. Notice the CPU computation time. Now, if a GPU is available, we move our tensor to it and perform the same operation. As before, we register the start and stop times and display the scale of speed improvement. Run. The acceleration is remarkable. Operations that might take seconds on a CPU can complete in milliseconds on a GPU, thanks to massive parallel processing capabilities. Processing power requires structure. Structure requires tools, and tools like tensors require mastery. You now possess the fundamental protocols for tensor manipulation in PyTorch. Use them to shape data, optimize workflows, — and power your models. Memory buffers, clearing. CUDA, powering down. PyTorch enlightenment protocol concluded.