Welcome to the section in which we will look at linear algebra operations.
So just right here we have TensorFlow Linalg.
And the very first operation we'll be looking at will be that of the matrix multiplication.
So right here we have matmul.
Essentially what this matrix multiplication does is it takes in two matrices.
And then um computes the matrix multiplication of those two matrices.
So here we have TensorFlow Linalg matmul.
And let's just copy this part.
There we go.
And then we have this um examples which we have right here.
So let's go ahead and test this out.
We have ab AB take this off.
And then let's set a let's get back up here and try to copy this.
There we go.
We have let's say this is a.
So this is our a.
Yeah.
We have a and then we have um well that's a and then we have B, which is going to be the same as a.
So let's copy this paste out here we have A and B which are exact same.
So that's it.
Now let's um b now let's go ahead and print this output out.
We have print out the map or matrix multiplication of A and b.
So let's run that and see what we get.
You see we have this error.
So we have uh matrix size incompatible.
Now the reason why we're having this is simple.
Let's first of all print out a shape.
And then we print out B shape C here we have 4545.
Now if you um are new to this um matrix multiplication and linear algebra in general, you should take
note of the fact that when multiplying two matrices, the number of columns of the first matrix will
or should always match the number of rows of the second matrix.
So when you have a matrix like, um, or when you have two matrix like A and B, where this there is
no match.
That's why you have this kind of error.
So let's, let's um, try to solve this problem.
Let's, let's say we change this from five columns to four columns.
Or we could change this from five four rows to five rows.
So b here let's copy this.
Increase number of rows.
Take this off.
Increase number of rows.
And that's it okay.
So now you see it's going to work because we have four by five and five by five.
And this is our output.
Now it should be noted that if instead of um um the math model of A and B, we try to print out, we
try to print out A times B like this, you would find that we'll get an error.
So the math model produces no error.
But A times B produces an error.
Again here we have incompatible shapes.
Now the reason why we're having this is because with this A times B it's an element wise multiplication.
That is we're going element by element.
That's three.
And this corresponding here we multiply this one and this corresponding.
Here we multiply this nine.
With this corresponding we multiply and so on and so forth.
Uh, and so if we have this tool or matrices which do not have the exact same shape, or where we cannot
even broadcast one to uh, given shape, then we would have an error.
So you could see clearly why we get an an error.
Because, um, if we're trying to compare this seven year or we're trying to multiply the seven which
is corresponding, and a, we're not going to have that corresponding value.
And so that's why we get this error.
And so in that case this will work.
Let's get back here.
This will work if um we take this off.
So if we take this off and run that you would find that we have no error.
So obviously we have three times three like this.
Three times three is nine.
So that's why we have nine here.
Uh right here then if we take some random position this two times two, we have four.
That's why we have four right here.
So that's it.
Now let's take this let's comment this get back to our linear or linear model.
We get an error.
That's obviously because the number of columns for the first doesn't match the number of rows for the
second.
So again we're going to copy this and paste our year.
Run that again.
And now everything should be fine.
Now let's keep let's make this simpler.
So we could explain uh matrix multiplication in simple terms.
Let's take this off.
Take three by three.
So we'll have three by three here.
And here we want to have two by three.
So let's, let's just two by three.
That's two rows.
Take that off and then by three that's fine.
So let's run that again and see what we what we get.
So you see that works two by three three by three it produces two by three.
Now let's take this example and show you how we get the output.
So right here we have this input which is two by three.
Or we have a which is two by three.
And we have b which is three by three.
And then the way we get the outputs is simple.
We take one times three plus five.
That's it here this one one times three plus five.
Times zero.
Plus one times one.
You see that?
And then we get the we we put the output here.
So now we have one times three which is three plus five times zero which is zero.
So we have three plus zero.
And then we have one times one which is one plus one.
So that's how we get this first position.
Then for the next position let's let's bundle this up.
So that's for the first position.
For the next position we have again one times one.
Plus five times one plus one times five.
So one times one is one, five times one is five.
So we have one plus five plus one times five which is five.
So plus five again.
And that's how we we obtain all those different values.
So we could bond those bundle this other one up one times nine.
Um is nine five times seven is um 3535 plus nine is 44.
Um well let's just put that out.
So we have nine, nine plus 35 and then plus one.
So nine plus 35.
Plus one.
That's fine.
Now for the next, um, we have seven times 321 plus zero.
So we have 21 plus zero plus zero.
So we have essentially here we have 21.
And then for the next we have seven times one um which is seven plus three plus zero.
So seven times one seven plus three plus zero that's ten.
And then for the next we have seven times nine.
So here we have seven times nine which is 63 plus um 2163 plus 21.
That's 84.
So here we have 84 plus zero.
So here we have 84.
So finally we will have this output which is three.
Uh from here we will have um 11.
We will have um 30 or 45 will have 21.
Ten and then 84.
So that's it.
And then 84.
Getting back to the code, you could verify that.
You see here we have 411, 45, 21, ten and 84.
Um, this actually four, three plus one is four.
So that was an error.
So here we have four.
That's it.
Um, we we now understand how we get this output, which is quite simple.
And also we could get the shape before the calculations.
Now notice how because these two are the same.
See these two are the same.
The output is this outer um dimensions.
So the number of rows here will be the number of rows of the output.
And the number of columns here will be will be the number of columns of the output.
So here we have two by three two by three output.
Because here we have two rows.
And here we have three columns.
And these two obviously must match.
Also notice how if we didn't have this say let's suppose we did not, we did not have um because this
is three by three.
So let's suppose we do not have this last row.
So we, we only had this output which was that is two by three.
And here we had two by three.
Then it means that when we take one by three plus five by zero, then what we want to have this one
by its corresponding here it wouldn't match.
And so this makes sense that the number of rows here, the number of rows should match with the number
of columns here.
Because that's exactly how matrix multiplication is computed.
Now getting back, we could obviously calculate the matrix multiplication by simply doing a at um b.
So if you do a at B you see you still have the exact same output.
Our next operation will be the transpose.
So let's suppose we have again a let's let's let's get let's copy this a we add from here.
Let's just copy this B if we take this well what will be the transpose will not be very clear with the
A is much clearer since it's not a square matrix.
So let's let's print that out.
Um, we have A151730.
Let's get back to the board.
151 um, 730.
Let's copy this.
There we go.
We simply copy that, paste it out, and then there we go.
Let's take the scroll down.
So that's it.
So now we have this matrix right here.
Let's take this off.
We have this matrix.
And what the transpose does is simple.
So here if we have um a matrix like this one, if we have a matrix like this one where we have 151730,
then to obtain the transpose, all we need to do is make sure that all the rows.
Now become Collants.
So all the rows become columns, and all the columns obviously will become rows.
So this row here will become or the second row becomes our second column.
So we have we have this next one right here.
So now we'll have 151.
So we go from 151 as rows now to 151 as column.
Then 730.
We have 730 as column.
Notice how the the columns this 171 have now become the rows.
So 171 is now a row.
Um five three which was a column is now a row, one zero is now a row.
So this is what we call the transpose operation.
And using TensorFlow again it's quite easy to make use of this um operation.
So let's get back here.
And let's say we do um let's print out spring.
Let's print out um transpose.
Um a so let's run that and see what we get.
So we have A151 um, 730.
Now we have 1753 and one zero exactly as we expect.
Um, that's it for the transpose.
It's a simple operation.
We could take this off and see how we could change this.
So we're going from an output or an input, which is that's a we go from an input which is one by three
to an output which is three by one.
Also what happens with the transpose is the number of rows, um, changes to become number of columns
and number of columns changes to become number of rows.
So we go from 1 to 3 now to three one.
Also note that when you're carrying out this, um, matrix multiplication, if you get back to the documentation,
you might have noticed that we had this transpose A, transpose B.
So let's get again, get back here to the code, um, and say we want to um, compute this matrix multiplication.
So let's say we want to have um let's print that out.
Um, Linalg Matmul.
And then we have a and B and let's, let's suppose that we wanted to, to multiply A with the transpose
of B.
Well, in this case, um, our A is one three.
And so if we're, if we were to have that's B okay B is three three.
So that that should be fine.
So let's say we have a um at B13 by three three should give us an output which is one three.
So that's that makes sense now.
So A is 13B is three three.
And we have that output which is um three or rather one three.
Now if we compute the transpose of this three three matrix it's still going to be three three because
the it's going to be the same.
Um, because number of rows equal number of columns.
So it's not going to change anything.
So let's suppose we want to multiply a, let's get the the matrix multiplication of a and b transpose.
In that case we would have a at TensorFlow.
Um transpose and then b.
So we run that and see we get 17 1227.
Now if we were to do it in another way we could do TensorFlow Linalg.
Uh linalg.
Uh, matrix multiplication.
Um, and then here we specify a, we specify b, and then just as in the documentation where we had
transpose B um set to to false initially we could now set that to true.
So we have transpose B set to true.
So essentially what we're saying here is we multiply a and b with the difference that b is the is is
transpose instead.
So we're getting a times B transpose.
So we run that.
And you see we have the exact same output.
So this shows you two different ways of carrying out this um exact same computation.
Again um transpose and B um, didn't stop or didn't bring an error because the number of rows equal
the number of columns.
So if we had b, let's say we we had changed B.
Let's take this off.
So now let's say b is three by two on that.
And then we get back here we should get an error because um two by three or rather this a a which is
one by three doesn't match.
Um two by three.
Let's make that much clearer.
Let's run A and B side by side and print out the shape.
You see here that this computation could actually work.
But when you do the transpose, when you do the transpose of B, that's when you do, um, the transpose
transpose of B, and then you get a shape.
You get a shape.
You find that one three cannot match with two three.
Because number of columns here is different from number of rows we have here.
That's why we're getting this error.
And so if we just had b.
Obviously we should have no error.
So far we have been using the TensorFlow transpose method.
We could simply change this by doing.
If you want to use a linear method, or here we have matrix uh matrix transpose.
That's fine.
So let's run that again.
And normally everything should work.
Um, as expected from this point we'll look at how to carry out matrix multiplication for um tensors
which are greater than 2D.
So let's suppose we have a 3D tensor.
Um let's take some example of a 3D tensor which we had done previously.
Let's get back up.
We have this 3D tensor right here.
Let's get back from here.
We paste that out um just below.
So let's paste it out.
Here we have this tensor.
Well let us call this a.
So let's suppose we have this tensor A right here we have this tensor a.
There we go.
We have our tensor A.
And then we want to multiply this tensor A by uh another tensor b.
So let's say let's take let's copy this again and paste out here we have tensor A.
We have tensor B which for now are exactly the same tensors.
That's fine.
We'll just run this and then print out a shape.
We'll print out B shape.
And then we try to uh multiply this two.
So well let's just print out the shapes first and then we do a or TensorFlow linalg.
Um matmul a comma b.
So you see again we have this error.
But the reason why we're having this error is because when you're doing matrix multiplication over batches
of, um data, we consider each and every batch and try to carry out the matrix multiplication for that
batch.
But then for each and every batch we have here, if you look at this, let's reduce this so you can
see the two at once.
If you look at this right here, you'll notice that this is two by three.
So this is here two by three.
Here we have two by three and then here we also have two by three.
And obviously because the number of columns in this case, which is three, doesn't match the number
of rows, in this case which is two, we get that error.
So if we convert this into because here we have um looking at the shape we have for x, here we have
for two by three matrices.
Here we have for two by three matrices.
So two by three.
If we change this um from two by 3 to 2 by two.
So let's make sure that each and every one of this now is two by two.
Let's get back.
We have two by two, two by two.
There we go.
Two by two.
And then finally we have two by two for this last, um, element.
So we run that again.
We get a shape do matrix multiplication and we have an output.
So because this is two by two this two matches with this two.
And that's fine.
So again as we're seeing the matrix multiplication where we have uh a 3D tensor like this one is carried
out in such a way that each and every um, block of 2D, there is some sort of block wise matrix multiplication.
So this block is multiplied with this block, this block multiplied with this block.
Um, then this block, multiply with this block, then this last block.
Multiply with this last block.
Now to test this out, you could simply say, what if we do a zero.
So let's get a zero.
If we do a zero uh let's print out a zero.
And then um we print out B zero.
So we get the first block or the first two blocks.
Here we have one, two, three, four, five.
And here we have 12035.
Negative one 1235312035.
Negative one.
Which is what we expect.
Now let's do a zero um times a zero times B zero.
We run that.
You expect the output to be exactly what we had for the first block here.
So we have 712 -218, 31 negative five, which is exactly what we expect here.
Now if we change this to let's say two two run that again.
We expect to have this exact output.
See that.
So this tells us that uh, when we multiplying um, two 3D um, tensors, we are simply carrying out
a block wise, uh, matrix multiplication.
For our next method we check out the barn part method.
Let's get to barn part.
So here we have barn part.
Scroll to the top.
What it does is it copies a tensor, setting everything outside the central band in each innermost matrix
to zero.
Now, um, you have this this explanation and maybe this, um, definition here, which looks a bit
complicated.
And, and so we're going to break this up.
So here we have also this you see this definition.
Uh, we told that the band of this matrix, um, is in band m n times the input matrix.
And then the in band is defined or the indicator function is defined as such.
So this function essentially what it does is it's um capable of maintaining all the values of a given
input while turning some of the values all to zeros, just as it was, um, defined here.
So let's go ahead and copy our this input, this simple example and then um, see how we could obtain
the output.
So let's get back here.
Uh, we could just take we could just take what we had already from here.
So let's just, let's just copy an example.
We had, um, let's take a three by three matrix.
Well, let's just copy this and then we'll just turn this into a three by three matrix.
So getting back here we have spaces out.
Then we copy this out and paste out.
That's fine.
Zero.
Um well let's not let's not set this to zero.
Let's put one and then let's say three okay.
So five or let's say nine okay.
So that's it.
So this is our input matrix A.
And what we want to get is um linear band part of A while specifying the the lower number to be zero
and the upper number to be zero.
So we will come and copy the definition and see how we could produce or generate the output.
We'll go step by step and see how this output here is generated.
So we have this condition right here.
Let's let's take this off so we could see only the conditions.
See we have this conditions.
And now because our num lower is zero here this condition is never going to be verified.
So this is false.
And then if we continue here our num upper is zero.
So this condition is never going to be verified.
Two.
So here already we have this condition which is never going to be verified.
And this condition which is never going to be verified.
Now for the next for the other conditions, for this other conditions, it will depend on the position
of that element.
So if we take this position here we have m.
Let's um put this down.
Here we have m equals zero and then n equals zero because this is the zeroth row and the zeroth column.
So because m equals zero and n equals zero um m minus n we have m minus n uh which is equal um zero.
We have n minus m which is equal to zero.
And then m minus n which is zero.
So we have is zero less than or equal zero.
Yes that's true.
So now we will have here true here we have true.
That's true.
Is zero less than or equal to um zero.
That's true.
So it means that uh because this is an R.
Remember with the r, um, any of them need to be true for us to have true.
So the r is only false when we have this condition false and this condition false.
So because one of this is true, our final output here is true.
So let's pick this up again.
We have um at the center.
Now we have true.
And then um on this other side because one is true, we also have true.
You see we have true.
Now for our overall condition to be true because we have the end, because here we have true here to
the left we have true, and to the right we have true.
Um, these two must be true.
And that's the case.
So overall we have true.
So finally we have true.
And because we have true this value of one, if, you know, if you look here is maintained.
If it were false, then the value would not be maintained.
Now let's take a different example.
Let's say when m is equal zero and n equal one.
If m equals zero and n equal one, that's where this zeroth row and we are at column one.
So we fall in this value five um m minus n is um negative one and n minus m is one.
So that's it.
Let's take this here.
Um n minus n negative one n minus m one.
Now the again we have this false because our num lower is zero.
So we have false we have false.
And then for m minus n m minus n is negative one is less than zero.
So this is true.
So that's fine.
Here we have n minus m which is one.
It is um not less than um zero.
It's not less than or equal to zero.
So this tends to be false.
So here we have false.
And so this is false true.
This is ah so this this is true.
And here we have false false.
And it's ah so it's false.
See now here we have true false because we have.
And the two must be true false to have true.
So we end up with a false.
And because we end up with a false the five is changed to zero.
So that's how we, we we we get this um, position here.
Now you'll note that for the rest we will have something similar where generally, um, it's only when
m is equal, n is only going to be where m is equal to n that we are going to have um true in the output.
And so we have an output where all the values which are not on the diagonal are always zeros.
So that's the this um, band part method.
When we set num lower to zero and num upper to zero, if we get back here you'll see that we have some
special cases.
So these are some useful special cases.
Let's take back this.
You see that running this?
Now we have for this.
First the upper we have the upper triangular part which is nonzero.
So um, first of all, if you look at a this is a the upper triangular part remains intact, whereas
the lower triangular part is all turned to zeros.
Now essentially what this means is if you have an input like this one, let's say we have um 356, um,
102 um 113.
And then here we have two.
This is one.
When we say we want to have the upper triangular part.
Let's reduce this when we say we have the upper triangular part.
Um, that is all this um parts here uh be non zeros, whereas all the, the lower triangular part is
set to zero.
Then what that simply means is, um, you break up your matrix into two parts, so we into two triangles.
So you have this triangle.
And you also have this, um, triangle.
Well, let's get back because this is this is the upper triangle, actually.
So this is the upper triangular part.
This is a lower triangular part.
So we have this two triangles.
And then we have this diagonal.
So as we have seen already uh, when we set when we have zero zero.
The lower triangle and this upper triangle is zero.
Whereas when we have, um, let's take this other condition, when we have negative one zero, we have
your the lower triangle which is set to zero, whereas this remains intact and the diagonal also remains
intact.
And then uh, when we have um, zero negative one.
Well this would be this is zero negative one.
When we have negative one zero we instead have the upper part which is set to zero as you could see
here.
So when we have zero negative one or rather negative one zero when we have negative one uh negative
one zero is this upper part which is set to zero.
Now there are some special cases.
Or in practice you may want to have an output which is having um, let's say this upper part, this
upper triangular part set to zero or maybe the lower triangular part set to zero.
And in that case, uh, the band part method comes very handy.
Another method which we have is the determinant, um, which helps us compute the determinant of a square
matrix.
So it's important to know that this is a square matrix.
That is M should be equal to n.
So the number of rows should be equal number of columns.
Um that's um a very simple method which is which you could use depending on what problem you're trying
to solve.
Then we also have the inverse.
So let's go to I, we have the inverse.
There we go.
Now note that the inverse of a matrix is one which is defined, um, such that the inverse of a matrix
is one which is um defined such that a inverse.
Times a.
So the inverse of a is such that a inverse times A gives you the identity matrix.
So if A if we have A to be a two by two matrix, which is like um, say um x um x x x whatever value
we have in here, multiplying this with this inverse will give you the identity matrix which is one,
zero, um, zero and one.
And so if you have a matrix like this one and you want to find a matrix such that when you multiply
with this, um, matrix, it gives you the identity matrix, then you could make use of this TensorFlow
um inverse method.
So that's it.
So you could compute the inverse of one or more square invertible matrices or the adjoints.
So that's it.
The next method we'll be looking at is the Insum.
The Insum operator offers a clever way of carrying out matrix computations by taking into consideration
just the shapes of the different matrices of the different tensors.
To better understand how the Insum operator works, we shall take several examples.
So let's start with the matrix multiplication.
Right here we have a b and then we carry out matrix multiplication of of a and b.
And then we obtain c.
Obviously, because the number of columns of a equal number of rows of b, this matrix multiplication
is valid.
And we have this output which is of shape three by five.
So here is the output we get.
Now if we were to use the insum operator to get the value of c, what would make use of will be the
shapes of our um different matrices.
So this for example is three by four and then b is um four by five.
Obviously the output c is three by five.
And so if here we have I g.
So we take this to b I g we put a comma.
And then here we have g k.
Then our output is going to be I k.
Notice how this j is the same as this g.
So because we we these two must be the same.
We maintain the j.
But we have your I and then k as output.
So here we have I um k.
And so if we want to translate that into code we will have um TensorFlow I insum.
And then we will have the string which takes I j comma j k.
So we want to multiply a which which has a shape.
And then by b.
And then the output is going to be I k which is now c.
So we pass in a and b and it understands that this is a and this is for b.
So let's run this and then get this output I k.
So you see here we have the exact same output as um with the matrix multiplication.
Now another example we could take is the dot product.
So let's suppose that we have um two vectors.
So let's let's um set a and b again let's just copy this.
So we have two vectors a um just one d.
So here we have or we take this off.
Well let's just take all this off.
Take this, put this here.
And then uh, we create our 1D tensor.
So we have this 1D tensor which is our vector.
And then we want to do a dot b.
So dot product we want to get the dot product of a and b.
Um here we have say zero.
Let's say one let's say three.
And then let's say nine okay.
So well let's just put this a small number.
So if we were to calculate this manually we would have two times zero plus which is zero.
So we'd have zero plus um six times one which is six plus five times three which is 15 plus two times
two which is four.
So in that case we are going to be having 21 plus four.
That's 25 okay.
So if we want to um compute the dot product, what we'll do here is we would make use of the Insum.
So we have Insum.
And then because we have a here which has a shape I because it's not it's not 2D it's 1D.
So we just have I and then we have B which is um shape j c.
So um like before here we had this comma separating the two.
So let's put that comma.
You see this is a this is B.
And then the output is going to have no shape because it's going to have uh it's going to be a scalar.
So because it's going to be a scalar we put nothing um in front of the arrow.
And then we just simply put in A and B we run that and there we go.
We have 90.
We are expecting to get 25, but we instead got 90.
Now we have a a year, uh, which has shape four and then we have B which has shape um four.
And then when we define this we act as if A and B, um although 1D have different shapes.
So we, we act as if we could have six here or we could have nine or we could have whatever number,
uh, and which is not the case because, um, for us to carry out the dot product, this tool that's
A and B must have exactly the same shape.
So if we have um, shape four then here we should have shape of four.
And so that's why we have to change this g into I because this two must be the same.
This is similar to what we had with the matrix multiplication where this j and this j or essentially
we make sure the color here is j and the row um number is g.
So make to let the handsome operator understand that this is um.
This condition must be verified.
And if this condition is verified then we have matrix multiplication.
So getting back here, you see if we have I, I and then we output nothing meaning that we just, um,
sum up or we just multiply and then sum up all the values since we'll get a scalar.
Um, then if you run this again you find that we have the output 25.
Now talking about let's get back let's get back here.
I'll take this off and piece that out.
Let's, let's say we had J.
So let's say we had J.
If we had J here then we'll be instead computing the outer product.
Now with the outer product is is is different because we could have something of different length.
We could have one uh, and that would still work.
So if you run that you'll still have an output because I and J could be different.
But if you get here and you have one and you run this, you should get an error.
You see, um, that's an error.
Which makes sense because this is are the same.
So let's get back, run that again and get back here.
So that's it.
So you see that's that's what we have now if we do if after this that's after this I, we do let's say
we want to have an output I, we run that you see we have the output that we have here is still another
1D tensor.
And when you look carefully you will notice that what all what we have here is simply is simply gotten
by multiplying each and every element here.
So if you take this zeroth element multiply it by zero.
That's two times zero.
We have zero six times one.
We have six five times three we have 15 two times two we have four.
And then when you sum all this up we have now 25.
So essentially when you specify that the output should be I like this.
All what you're saying is just do the element wise multiplication and give me whatever output we get.
But when you take this off, what you're saying is, um, generate this because after multiplying you
will generate this.
And then because there is no output, I want a scalar.
And since I want the scalar, I'm going to sum all this up to produce that scalar.
So when you run this now you see we have the 25.
Then if we say I j we run this.
You see, um, our script contains a level J not present the inputs.
Well, this makes sense because obviously you cannot have, uh, you cannot introduce J when it's not
present in the input.
So let's have I run that again.
Um, output subscripts contain a level appearing more than once.
So here we told that we have this level I uh, which appeared in more than once.
Anyway, um that's it.
Let's take this off.
We have our dot product, and then now we get back to our outer product for the outer product, um,
here we have two, six, five two and then zero one, three two.
Let's say let's take this off for now.
So 26520132 to compute the outer product of this tool will um, rearrange this this way.
So let's say we have this matrix or we have this tensor and then we we have this other one.
Let's turn this like this.
Let's get this transpose from here.
Now we do zero times two zero times two.
We have zero zero times six we have zero.
Well we have all zeros here.
And then one times two we have two.
We just maintain this 2652 um six um 18.
18.
Three times five is 15.
Three times two is six.
Two times two is four.
Two times six is 12.
Um, two times five is ten.
Two times two is four.
So this is now our outer product.
And this is how it's computed very easily.
Getting back to the code, if we have this as our a and this as b, then if we say we want to have ij.
So we want to generate this 2D matrix um which takes into consideration or which takes as input a and
B and then our computer's outer product and we run this.
You see we have this matrix which is similar to what we just generated with the difference that this
is the transpose of what we got.
Well, getting back you see this is note that this is um, let's just copy all this so we could compare
on this on the board.
Take this slightly up.
As you can see, it's exactly the same, um, as the transpose of what we just computed.
So what this tells us is when we, when we have this tool like this, the A acts as a column matrix
and the B acts as the row matrix if we have ij.
So this is this here, this a is a column matrix and B is a row matrix.
So instead of 2652 where we're supposed to put it out like this.
So it was supposed to be 2652 and then 0132.
So that when you multiply now you have all the, um, answers, you or you have exact same response
as expected.
Um, if you want to have the, the original or what we had when we calculate computed this originally,
you could instead have.
Well this is AB, this is AB.
And then here you could do G.
So you run that again.
And you see we have exactly what we got um initially.
So this if is, if it's meant to be in order that's IJ then this is going to be our column matrix.
And then this is going to be our row matrix.
So let's say now that we have C.
So this is our C.
Um let's go ahead and compute the sum of all elements in C.
If we want to compute this we'll simply make use of the reduce sum method.
So reduce sum let's print that out.
Run that again.
And then we see that we have 90.
Now.
Well let's print out C.
So we have C printed out and we have the sum of 90.
Now what this tells us is that um initially or right here when we had computed this and then we had
put out um, or when we had IJ, you see, remember when we had IJ um, we, we ran this and then we
had 90.
It was simply because we had computed the outer product instead of the dot product.
And then we had summed up all the elements in this outer product.
Because now if you do if you get back here, let's take this off, print it out.
And then instead of, um, coming up with this 2D tensor, if we come up with a with a scalar by by
saying we don't want, uh, or by simply not putting anything in the output, and you print this out,
then you see, now that you get the exact same 90 which was expected.
So when you sum all this and then when you just do this directly, you, you get that same 90.
So that's why initially we had made an error um, here.
So instead of 25 we got 90 because of that error we made.
That's where computing the outer product instead of the dot product for the next one is going to be
the transpose.
So let's suppose that we have um, this matrix C um let's say we have this matrix C or well let's just,
let's just go, go and copy out um, a again, let's try to get the, the transpose of this B we'll
copy A or B doesn't matter.
So let's say we get this is our b.
So that's our b.
And then we want to get the the transpose of B using the Insum operator.
Then you could pause at this level and try to figure that out.
So what we'll do is we'll say okay B is IJ remember B is ij.
But there is no comma here because we are dealing with only a one matrix or only one tensor.
So the output is going to be GI.
That's all.
So and then you pass in a.
So you run that we get this error.
Expect input to have rank two but got one.
While looking at this you see we put A instead of B.
So when you replace and put B you find that now it works.
So this here has been transposed.
Notice how we have 29030.
And here we have 29030.
Um 368 negative two 2368.
Negative two two.
So that's it.
We've, we've um, simply gotten the transpose of our input matrix for the next.
We'll look at the element wise multiplication.
So let's come.
One copy.
Um, A and B again where we have A and B, we copy that, we get down and then simply paste that out.
Remember since this element wise multiplication this two must have the exact same shape.
So let's take this off.
Take this off, take this off.
And that's fine.
So now we have two three by four matrices.
If we do the usual element wise multiplication a times b.
Then you should have, obviously, for each and every position, the product of the two elements.
So if you take this negative two times six we should have -12.
So that's it.
Now for the with the end sum operator we could do n sum.
Now we have for a a is ij and then b must be ij.
Because we said the they should have the same exact same shape.
And then the output should also be ij.
So let's run that and see what we get.
We have ij ij expecting two.
Okay.
So we we forgot to put A and B here run that again.
And now our output as you could see from here is exactly the same output we got from here.
So we have 450 406 and so on and so forth.
So we see we have exact same output and that's how we obtain this element wise multiplication.
Let's now see what happens when we have say 3D tensors.
So let's get back again here we just duplicate all this.
Well let's just duplicate all this copy paste and send this just up right here.
We have now this um 3D tensor.
So here we have two.
We add a batch dimension, we have two by three by four.
So we have two, um, three by four tensors.
So this is the first one.
And then this is the second one.
So this is the this is a.
Now here we have a and then multiplied by here is multiplied by the b which is two four by five tensors.
So this are the two four by five tensors.
And this obviously is going to give us two three by five tensors.
Now just as we had seen already with a, with a usual or with the usual matrix multiplication, we had
ijk which gave us IK.
Here we're going to have b I j b j k and we're just going to give us b I k.
So the only thing we're going to add here is simply just this b.
Let's get back and try to um repeat that.
Let's get back here.
We had this um a and B hope it's the exact same.
No that's not the same.
Let's get back to where we computed the matrix multiplication.
There we go.
We had this here.
Um, let's just copy all this.
So that's it.
And instead of this here, we're going to put this out okay.
So now we have a, we have a which is going to be made up of this tool two of this.
So um, let's get back.
There we go.
Get from here.
And that's fine.
So that's it.
So here we have this.
And then we repeat this.
So let's just repeat this piece that out.
And that's fine.
So now we have we open this up here.
And then we close this at the end.
We close this right here.
So now that we have this uh our 3D tensor we repeat the same for B.
There we go.
We have a and then b and we print out their shapes.
So let's now do insum and then we have uh rather we have b I j and then we have b j k b j k.
And then we now have b I k.
So let's run that and see what we get.
So here we have a b c we have this output.
Um it's obviously supposed to be the same output because there are same uh matrices or same tensors
which we've repeated here.
We have 33, 69, 77 negative six.
Exactly as we expect a practical case of where we could apply or make use of the ensemble operator is
with the transformer model.
So right here we have the softmax of QQ transpose divided by square root of DK times v.
Anyway, you don't need to understand what's going on.
Just take note of the fact that we have k or k transpose, and that we want to multiply KR by k transpose
using the Insum operator.
So it's the same as just saying we want to multiply a by b transpose.
So let's say we have a which is this 3D tensor.
Let's copy that and paste out here we have a which is this 3D tensor.
And then we we we want to do a um times a transpose.
So we want to have a um times a transpose.
So now we're multiplying a with a.
And then we have transpose B set to true.
So what we're saying here is we want this to be transposed before we multiply with a let's print out
this and see what we obtain.
There we go.
You see, we have our expected output.
Now let's suppose that we wanted to use the Or.
Let's use the Insum operator.
So right here we would have print and sum.
And then we simply specify that here we're going to have because let's let's print out a um a shape.
Let's print this out here.
Um let's print out a shape.
So as we're saying here we're going to have two by three by four.
So it's b I j there we have big.
And then for the next one is going to be b.
But j I or rather b um j k because this, this could be two different matrices or two different tensors.
Now before putting out the string let's just um repeat or let's just put out the shapes here so it's
clearer.
So here we could have 234.
And then we could multiply this.
We could multiply this by um two or the transpose of 234.
Now what's interesting to note is the fact that because this column is already fixed, it means that
here too, it has to be fixed.
Because when you transpose this we're going to have two, four, three.
We constrain that the batch, uh remains fixed.
So when this is transposed.
So this is going to be now or let let's just put this out as the transpose.
Let's say the transpose multiply by the transpose of this.
So after transposing let's copy this paste this out here.
After transposing we expect to get 243.
Let's take that off.
We get back here okay.
So we have 234 by 243.
So we have to ensure that this number of columns matches uh with this um number of columns for the first
matrix.
Whereas this three year doesn't need to match with the other.
So we could have um two, three, four and then um, 234.
And here after transpose is going to be like uh, two 430.
Anyways this for this for match.
So that's fine.
So that said, what we are seeing is because this three could be any value.
It could be one, it could be whatever value.
Um, we have to be careful with the way we, we, we note that because in this case we took 234 uh,
two three 4th May be tempted to, to use the exact same, um letters for those two.
So getting back here we have I, um j k this is, this is I, this is J and then this is k, and then
the transpose of that's multiplied by the transpose of um 254.
So now we have I again, that's the same I or let's say batch batch batch I, batch I j.
And then here we also have batch.
There we go.
And then now we have k.
So we're calling this k because this five could be whatever.
It must not be.
Um exactly this three.
So it must not be I.
So we have k.
And then for the four we have j because it must match.
Remember after we transpose let's let's print this out again.
This is what happens after we transpose.
After we transpose we'll be having j k and this j.
Now we match with this j.
So it's important that we take note of that.
Let's take that off.
And then now we have a and a.
So that's it.
Let's run this and see what we get.
You see we have 138 um 16 104 and so on and so forth.
Well actually this is the reduced sum.
So let's, let's go ahead and put the arrow and then say we want to have b.
Now the, the this is very important because you have to be careful with the way this is going to be
or the output is going to be generated.
Here we have B, we have I and then we have k because this j and this j is going to be eliminated.
So we have by j bc and our by k slightly different from the usual matrix multiplication because here
the second uh matrix or second tensor is going to be transposed.
So that's it.
So run that see what we get.
You see we have exact same output as uh what we had before here.
So 69 852 821 zero.
That's it.
Um, exactly what we expect.
So that's it for the sum.
Operator.
Hope this, um, section was was clear.
And see you in the next section where we shall look at other, uh, common methods in TensorFlow.