Hi everyone.
Welcome back.
So in this video we're going to be starting to implement the API that we talked about the tech similarity
API and for that I need a new folder here called Text similarity.
So I'm going to go into the to the folder here so slash text similarity and we're basically going to
shirk building this from scratch right so that any of the previous go and just to get used to the idea
of developing an API from scratch and you don't have to reference or code.
So the first thing is we're going to be using Blocher and our compose as usual cerned going to do touch
darker compose dynamo.
So on the or on the outer directory slash text similarity is do touch darker composed.
And then around to also do the two directories that we have.
So I'm going to do make there actually web and make directory D-B.
So we have two new ones where we have the Web and the D-B.
The Web is going to have the the API.
And then is going to have as usual the Mungo D-B where we can store the information about the users
and their passwords and so on.
OK.
All right.
So a couple of things though.
Let's first go into this first open to your right to atom.
OK.
So we're going to open Adam in this directory and let's fill in the doc or compose file.
Right.
It shouldn't be too hard to finish.
So let's close these welcoming guides.
Make sure they don't they don't show us again and yet.
So that's close to that.
And then we go to the doctor gumbos file and he's going to be very similar to the previous ones.
We're going to write version and then three.
So our version history.
And then we're going to have services inside these services or we're going to have one cold web and
I can build it so I'm going to build from dot slash Schweppes.
So from this directory over here and I'm going I also opens up it's
and then it's going to be a list.
So it's going to be a list of five thousand to five that's I'm going to map the ports 5000 to five that
I'm also going to do some links.
So because I need to depend on another another one so I'm going to do lent D-B.
So it's going to basically depend on this D-B over here.
So that depend on this service service here.
So for this one we're going to build it right from as you might have guessed dot slash D-B.
We're going to have a docker file that we should build for.
Okay so that's it for the for the or composer very similar to the previous one that we built in previous
years now.
Now the second thing we want to do is to do the doc files right.
So let's go into to do the doc or file for D-B which is much easier right.
Very.
Only one line of code where we get the Mongo D-B from the the doctor that we discussed previously.
So I'm going to go into D-B and then I'm going to touch in your file called Duck duck or file a case
Doctor file just like this and then I'll see the doctor file is now showing yours.
I'll open it and then it's only a one line or even a write from Mungo.
And then our version was 326 point four.
Now if at that time of this video there's a newer version.
Feel free to put in your version here.
Shouldn't be too different in terms of the API.
All right so we've finished the doctor file for this one over the database.
So the next thing is to do the date the doctor file for it the Web site or the API.
So I'm going to go into web SOS.
So go back first and then go into web site and then when I get in touch logger fire file make sure the
pronunciation is and the spelling is correct.
So now we're going to see that in Web.
We have a new doc file here.
So the doc files here and we're going to write this one is going to be a little bit longer and we're
going to add an additional thing in this in this API that we haven't added in previous ones.
So if you remember when we were discussing here so let's go back to this chart protocol we were discussing
that we're going to be doing detection right.
The similarity detection between two documents but we never really discussed how we're going to compare
these two documents together.
And the answer is actually we're going to be using natural language processing library code Spacey netspace
is a very easy and natural language processing model that has already been tried it has a lot of models
already trained and are going to show you how to use it it's very very simple to use.
The API is very simple to use.
And so we're basically going to be comparing the two strings using a new library called Spacey.
I'm going to show you how to use this library in just a few minutes.
Let's go back to our virtual machine.
So the first thing is again we need Python right the first thing.
This is a python API.
So we're going to Python through that.
The second thing is I'm going to choose the work directory as we always do.
I'm going to go to slash slash slash app.
Gates We always do that or it's a convention.
Next we're going to copy requirements.
Text but we don't already have requirements that text.
So that's what we're going to be doing next.
So we have to remember that we still don't have the requirements on texts.
So for now we don't have it but we're going to be making it in a couple of seconds in this directory.
So recopying requires that text into the work directory which is less user slashed or slash.
And then what I'm going to do is I'm going to do a run.
So run it install and then I'm going to use no cash.
Right.
Make sure there's no cache directory and then recursive and then requirements not text.
Ok so I'm going to do this install no cache directory and then recursive and then requirements that
exit the file that I want to install next I'm going to do the copy.
To got someone to copy this entire file over here into Sless use or slash or slash app.
And then the fine.
Well now the final thing usually before if you remember we we do a basically command and then Python
and then app up.
Right.
So for now this is perfectly fine.
Right.
But the thing is Stacee already has some models right.
So what we're going to be doing is basically install that.
So if I go to the desktop here you'll see that install one of the models that Stacy was already trained
on.
So usually in machine learning or data science you have a model and you train it right.
And this training process takes a lot of computation power.
You need a strong GP you to do all these computations.
But for example the library's PC already trained our model.
So this is the model over here it's called English core web S M and then 2.0 point zero.
So this is a library here you can download it from the Web site off of species or if I go to your to
Firefox and then I write to speccy download and then I'm going to write.
So you're Spacey and then download and then models and then click enter.
Someone I find here models and languages you can see are already downloaded from here.
So if you click here and then you're going to have a bunch of available models right so this one here
and core web as.
So this is the one that I downloaded and you can download it from here using Piscean bib or you can
download it right away from the get help repository.
Right.
So if you click here and then basically this is the library that so if you look here it's only 35 megabytes.
And this is the smallest model.
Right.
And if you want to download it right away to your device then you can go to get help repositories or
you can write get help and then speccy models.
Spacey dash models.
And then you click here on the Gitau blink.
And then you to go for releases so you want you want to download their latest releases.
So you're going to
and then you're going to go to slash releases slash download and then slash the model name that you
want.
So in this case what I downloaded is the smallest model of them which is eat an underscore core underscore
web underscore s.m so I'm going to write an underscore core underscore Web so you go to space the models
get to the top so get dot com slash explosions last speccy models and then slash releases slash and
download slash and underscore core underscore web underscore s m Dasht 2.0 point zero and then slash
and underscore core underscore where underscore and Dasht 2.0 point zero Tarbuck GC and he should find
it on the page and then if you click enter then it should automatically show you the link.
So here is the file.
So this is the underscore core underscore web underscore as.
And this is 3:35 when six megabytes and this is exactly this file over here.
So you're just going to basically save the file.
But for now since I already downloaded it I will do that.
OK.
So why are we discussing this.
So these models are already pre-trained and they're good for as I say you can use and right away to
to use or to predict some whether the similarity of two documents is similar or not.
So what we're going to do is we're going to take this package over here and get a copy it and then going
to go into tech similarity.
So this is our new API.
And then I'm going to go into web.
And then I'm going to paste this model here because I'm going to be using this more frequently.
Right.
OK so now let's go back here.
Now I want to say that I want to download or to to to pick install So this is basically if you go back
here this model is just up is either is a Python package.
So you want to install it.
So I go piss.
So I'm going to go back and do my doctor file here and then I'm going to write Pipp that first of all
run to run a command and then get installed and then I'm going to go to the rectory as my model which
is in this directory here so I'm going to do a dot slash event underscore core underscore web underscore
as and then dash 2.0 or point zero and then that TAR that GZ and then that's it.
So I basically told Pipp to install this library over here.
Now there are other ways that you can go get it from the Speccy Web site here to download this model
right away using pick rice or Pipp install space something so there is a command here which is.
So I'll show it here.
Bison dash cam Spacey download and then space and underscore core underscore where underscore grass
and this automatically downloads the the library for you and sets everything right.
But for me I'd prefer it to have my model off line not online because the website might break at any
point.
So I prefer to have the model locally of elbow's shouldn't have open that.
So I prefer to have the model available at all times so that you don't have to open it or download it
before you want to run this program right because the Web site might be broken so it's better to keep
everything locally.
Even the models you're going to be using.
OK so this is the docker file.
Right.
This is what we're going to be using the model that we're going to be using with Spacey to predict whether
two documents are similar or not.
Now this if you remember I said a few seconds ago that we still need to do the requirements that X-Fi
So let's do that.
So we've got to go touch requirements that text and then we have that here.
And then we have a couple of requirements so first of all we need of course Flast right as we always
discuss with the flask restful right flask restful.
We also need pi Mungo to communicate with the Mongo D-B we need be script because the script is used
to store the passwords to hash the passwords and store them as hashed passwords.
And finally we're going to need a new library called Spacey.
So Spacey is Espey ace UI.
So let me perhaps zoom in here.
So Spacey and space is basically the library where we'll be using for natural language processing.
OK.
So are we going to save that and we're done with the requirements text and the overall architecture
of the project.
So we got to pause here and we're going to come back in a future in the future videos and start implementing
the app up your way.
Right so we have one last file we haven't done worse which is a touch at the Y.
And this is where the main logic of our API is stored.
If you remember everything here is in this API.
It is in this A.
After you whyfor.
So yeah we're going to stop here and we're going to pick up in the next video.
So until the next video had become.