‫But now this has been an easy and unrealistic example.
‫In reality, our cost function was much larger and we had many more constraints.
‫The standard form of our cost function was J equals one half delta.
‫Your global transposed.
‫Then this is your h double bar matrix delta, your global plus small f transposed times, delta, you
‫global.
‫And this small f transposed that was essentially this one here in the concourse, our horizon period
‫equals 10.
‫But you could have also made it bigger, like 15 or 20, perhaps.
‫But let's stick with 10 in that case.
‫How many dimensions would you have in your cost function?
‫Well, Delta U equals Delta Delta and Delta A..
‫And your horizon equals 10.
‫So your delta, your global, would equal all that.
‫So in total, you have 20 independent variables, so you have a 20 dimensional domain.
‫Of course, this is not something you can visualize, but I recommended you to think about it differently.
‫In terms of a table with 20 columns, each column is reserved for one specific independent variable,
‫and that variable could range from minus infinity to plus infinity.
‫And so instead of visualizing something, in my opinion, a better way is to think about it like this.
‫What's the unique combination of all the variables here that would give you the smallest j possible
‫out of all the possible values that each variable can have?
‫There is one unique combination of values that would give you the smallest cost function possible.
‫Any other combination here would give you a bigger j.
‫That's essentially minimization of the cost function.
‫And so your solution delta you global, which is in red here.
‫That's that unique combination that gives you the smallest j possible and then the red case.
‫It does not take constraints into account.
‫However, the green case does take the constraints into consideration.
‫So in green, this unique combination, it gives you the smallest cost function possible while respecting
‫all your constraints at the same time.
‫The red solution might give you a smaller j, but it's possible that it violates some constraints.
‫And now, if we want to constrain our Delta, Delta and Delta values, then we did it like this.
‫We constrained our Delta Delta between Delta Delta Minimum and Delta Delta Maximum.
‫And in the same way, we constrained our Delta A between Delta Minimum and Delta a maximum.
‫So we had a system of inequality constraints.
‫We determined some kind of minimum and maximum bounds, and we wanted to keep our delta.
‫Delta and Delta A between those bounds.
‫But since your horizon period equals 10 and since in the cost function, you had 20 independent variables,
‫then you had to have these constraints for all of them.
‫And so you had here Delta Delta zero in Delta A0.
‫Then you had those constraints for Delta Delta one end Delta A1 up until Delta Delta nine in Delta eight
‫nine.
‫So as you can see, we had a lot of constraints.
‫When you don't have constraints, then finding the solution that minimizes your cost function was easy.
‫We used the gradient method for that.
‫We took the gradient of our cost function and we made it equal to zero.
‫So this is your cost function here.
‫And so that means that this one here, it looks like this.
‫This is the gradient of the cost function and I make it equal to zero.
‫And then from that, you could make some rearrangements.
‫You would have a double bar times delta.
‫You global equals minus F and that would give you Delta.
‫Are your global equals minus h double bar, inverse times small F?
‫That was your solution.
‫Without the constraints, we could use the gradient method because the gradient means the change of
‫the cost function with respect to all of its independent variables.
‫When you're dealing with a quadratic cost function whose h double bar matrix is positive, definite
‫and when all your NPC weights are positive, so CU S and R they all have to be positive, then the gradient
‫of J equals zero.
‫Where your Minion point is in the concourse.
‫I treated the concept of positive definite matrices extensively, but in a nutshell you saw that if
‫h double bar is quadratic and positive, definite with positive NPC weights, then your cost function
‫looked like this.
‫It only had one global minimum point.
‫That's where your gradient of J equals zero.
‫Your cost function j could range from this j midpoint up until plus infinity.
‫So mathematically, you could write that J belongs to J Min and Plus Infinity.
‫There was no way that this cost function could have a smaller value than this J Min value, but when
‫your h double bar was not positive definite, then your cost function looked like this.
‫You see that you don't have a minimum point there.
‫Your cost function would go from minus infinity to plus infinity.
‫So your j it would belong to this interval here from minus infinity to plus infinity.
‫You did have a place where your gradient was zero, but that was not your minimal point.
‫We called it cell point, and the same logic would apply in higher dimensions to in our car and run
‫courses.
‫The H double bar was quadratic and positive, definite.
‫Also, it was symmetric.
‫As we discussed in the very first car course, a symmetric matrix is like this one in blue.
‫The side elements are mirrored around the diagonal.
‫You see, you have four, four here, five, five here, six six here.
‫That's a symmetric matrix.
‫The cool thing about symmetric matrices was that they were equal to their transpose, so your h double
‫bar transpose equals your h double bar.
‫That fact allowed us to have this form for the cost function gradient.
‫So in conclusion, our cost function was quadratic positive, definite symmetric and the NPC weights
‫were positive.
‫That's why we could use this gradient method here.