# Python Tutorial: How I Manage Multiple Projects, Virtual Environments, and Environment Variables

## Метаданные

- **Канал:** Corey Schafer
- **YouTube:** https://www.youtube.com/watch?v=cY2NXB_Tqq0
- **Дата:** 06.09.2017
- **Длительность:** 20:48
- **Просмотры:** 159,814
- **Источник:** https://ekstraktznaniy.ru/video/12513

## Описание

In this Python Programming Tutorial, I will be showing how I personally manage multiple projects, virtual environments, and environment variables. If you have ever worked on multiple projects, you probably know that it can be a hassle to keep all of your dependencies and project information separate. The strategy I have adopted over the years allows me to easily separate out my environments in such a way that they can be easily exported, activated, and also automatically activate the correct environment variables. Let's get started...

The workflow I follow:
https://conda.io/docs/user-guide/tasks/manage-environments.html

Anaconda Intro Video:
https://youtu.be/YJC6ldI3hWk

The original conda_auto_env code I modified can be found here:
https://github.com/chdoig/conda-auto-env/blob/master/conda_auto_env.sh

My modified version of conda_auto_env:
https://github.com/CoreyMSchafer/code_snippets/blob/master/conda_auto_env.sh


✅ Support My Channel Through Patreon:
https://www.patreon.com/cor

## Транскрипт

### <Untitled Chapter 1> []

hey there how's it going everybody in this video I'm going to walk through how I personally handle multiple projects virtual environments and environment variables now when you're working on multiple projects at once it's easy to get your packages and dependencies mixed up and also it's hard to keep track of virtual environments and also you might have environment variables that are overriding each other and things like that so in this video I'll show you how I keep all of these separate and easy to work with so before I get started I just want to point out that this is how I personally manage these projects and environments it's definitely not the only way to do this and I'm not claiming it's the absolute best way to do this but it's something that has worked well for me up until this point so I figured I'd walk through how I do this and maybe it'll give you some ideas for how you can manage your projects but if you don't like this workflow then like I said there are plenty of other ways that you can accomplish this same thing so you have plenty of options now also I will be going over how I do this on my Mac and I apologize if you're on a Windows machine some of these steps will be similar on Windows but it's likely that a lot of this will be pretty different but I will leave a link to the page where I got a lot of these ideas for this workflow and I believe that they list the equivalent Windows commands on that website so I'll leave a link to that page in the description section below and with that said let's go ahead and get started okay so first of all I personally do a lot of work with data science libraries and packages

### use the anaconda distribution and the conda package [1:20]

so I use the Anaconda distribution and the Conda package manager now if you don't know what anaconda is it's basically an open source Python distribution that makes it easy to get up and running with different libraries and packages and it comes with its own package manager called Conda now Conda is similar to pip but certain things can be more easily installed through Conda now Conda isn't necessarily a replacement for pip so you can use them both interchangeably personally I actually use pip about 99% of the time and if there's something not available through pip then I will install it through Conda now Conda can also manage

### manage virtual environments [1:55]

virtual environments and that is what I use in my projects so we'll see that in just a bit so I do have a separate video on installing anaconda and the basics of using Conda and I'll leave a link to that video in the description section below now I know that some people don't like anaconda for some reason but I've never really understood why that is but like I said this is just how I personally do things and my work from day to day okay so from this point on I'm just going to create a new project from scratch and we'll walk through the process of making sure that it is separate from our other projects so I'll go ahead and create a new project here called my project and we'll create an

### create an empty directory [2:32]

empty directory just called my underscore project and then we will CD into that directory okay so now I'm

### create my virtual environment using conda [2:42]

going to create my virtual environment using Conda so we need to figure out what version of Python and also what packages we'd like to use in this project so for example let's say that we

### create a project using the latest version of python [2:52]

wanted to create a project using the latest version of Python and the packages let's say flask SQL alchemy numpy and pandas so to do this we can say Conda create to create a new virtual environment and we want this name to be since I called this my project we'll call this my project underscore env and now we can pass in our starting dependencies so like I said we'll do flask SQL alchemy also numpy and pandas so then it's going to list all the dependencies for all those packages and if we want to proceed and we can just hit yes okay so once that's finished up it says here that we can activate this environment using this source activate command here and to deactivate we can do source deactivate so I'm gonna go ahead and clear my screen and activate this environment and once you have the environment activated it should add this to your prompt here in some parentheses saying so that you know that you're within your virtual environment so now that we're within our virtual environment if I do a pip list then we can see the packages that we installed and with the environment activated you can continue to add or remove any packages that you want but when you get to the point where you have everything that you need I'd like to create an environment Yama file that captures everything about our environment and can be used to build the same environment from scratch so let me go ahead and do this so that you can see what it looks like so I'll clear my screen and now to export this environment we can say Conda you I envy export and we want to export this to an environment dot yeah mole file now some people just use yml and some people use yam all spelled out either way should work fine so once that environment file is created let's go ahead and see what that looks like so you can either open it up or in a text editor or you can just display it here within the terminal so on Mac or Linux I'm just going to splay it on the terminal by adding that file so we can see that within this file here if I scroll down that it has the name of our environment and all of the dependencies with the exact version so someone else could grab this file and create the exact environment with just one command so now that we've seen what this environment file looks like I'm going to clear my screen so we could actually package up that environment yam will file with our project and then if someone wanted to replicate that environment all they would need to do is create their own project directory drop in that file and then they could do a Khanda env create just create with a dash F saying that they want to create from my file and then pass in that file now I'm not going to run this now since we already have that environment created but you can see how easy it would be to create that environment from scratch just using that file ok so now let's look at how we can

### manage our environment variables [5:52]

manage our environment variables now environment variables are used in many different projects to you know set database URLs or hold secret keys or set a Python path and all kinds of different information so the environment variables allow us to access sensitive data without putting it directly in our projects code for everyone to see now some people just use global environment variables that they share amongst all different projects and that can work but then you have to be careful with naming and making sure that they're not overwriting each other so to separate these out the Conda environment the Conda environments come with some useful scripts that you can create so to create these scripts we need to put them in a specific location so we need to navigate to where our environment is located on the file system and to do this we need to first find out where our environment is located and to find that location we can say Konda env list and this is going to list out all of the

### list out all of the environments [6:48]

environments currently on my machine and we can see that this one is the one that we just created this my project env now this is the location of this environment of this virtual environment so now I'm going to change directory to the location of that virtual environment and now that we're within our virtual environment directory we're going to

### create two different directory trees [7:09]

create two different directory trees so that we can set environment variables now this is going to seem a little weird but I'll put a link to the instructions in the description section below

### create an activate directory [7:21]

now first we're going to create an activate directory and this is going to hold scripts that will run whenever our environment is activated so to create this directory we'll say make dirt - P that means that the directory doesn't have to currently exist and we're going to say etc' khandha and this is going to be called activate dot d and now we're

### create a deactivate directory [7:42]

going to create a deactivate directory which is just the exact same except instead of activate D its deactivate D okay so now we have created this activate directory and this deactivate directory now within the activate directory we're going to create a file so I'll do a touch and then within that directory we're going to call this e and V underscore FAR's Sh and then we're going to create that exact same file in the deactivate directory also okay so now we can create the environment variables that we want to set within the activate script and then unset those environment variables within the deactivate script so let's edit these files and add some variables now you can edit those files and any text editor but I'm going to use my sublime text editor to open up that directory that we just created so I am here in my virtual environment directory and we created this etc' conda and here's our activate directory and our deactivate so I'm going to open up this activate env VARs and this deactivate env VARs now within the activate script let's create a fake database URI environment variable so at the top I'm going to specify that this is a bash script by saying bin - SH and then underneath here I'll do an export of database underscore URI and set that equal to okay so this is kind of a actually I think this should be post SQL not entirely sure if that's the right connection string but anyways this is just a fake connection string to a test database here so this is something that you would use a an environment variable for is something like a connection to a database with the use username and password here so that you don't have to have the username and password actually within your projects code so whenever we activate our environment or virtual environment then it's going to set this database URI now within our deactivate script let's go ahead and unset this variable so again it will specify that this is a bash script with bin that's SH and then we will unset that database URI okay so once I have both of those files saved I'm going to quit out of our text editor here and now let me deactivate

### deactivate and reactivate our environment [10:10]

and reactivate our environment to see if those set properly so to deactivate this environment I can just say source D activate and you can see that now our environment our virtual environment has been deactivated and now if I reactivate that with source activate and that was called my underscore project underscore env and now let's echo out that database URI just to see if that was activated or see if this was set whenever we activated our virtual environment so that is database URI and you can see there that environment variable was set when we activated our virtual environment so

### deactivate this virtual environment [10:54]

now if I deactivate this virtual environment say source the activate now if I try to echo out this database URI then you can see that now that doesn't exist because we unset that whenever we deactivated our virtual environment okay so far we have our Python version our dependencies and the environment variables all separated out so for example I have another test project here that I've created in my projects directory called Cori MS underscore web so let me nag navigate to that project really quick and that is Cori MS under school or web and now let's look at the files I have available in here oh and actually I'm in the wrong location this is the virtual environment directory I really want to be in my projects directory kori MS underscore web so now if we look at the files that we have in here then we can see that we have an environment dot yam will file in here so let's take a quick look at that so we've got a simple environment yamo file here where we can see that the name of this virtual environment is kori ms underscore web and in our dependencies one thing to note here is that we're using Python 2:7 instead of Python 3 so you can have different versions of Python between different environments and we can see that the environment is using the same dependencies that we listed before except we're also including a sigh PI down here at the bottom now I've already created this environment from this Yambol file so let's navigate to the location of this and virtual environment and see if any variables are being set so to do that we can say khandha env list just like we did before and we can see it lists out all of our virtual environments here so I want to navigate to this virtual environment folder so I will CD to that directory and now let's open this up in a text editor so that we can see if we have any activate and deactivate scripts within here so I do have an activate script and I deactivate script and it's another database URI so this is basically the same thing that we had before except here at the end instead of test DB we're using a different database this core ems database so that's why it's beneficial to keep these environments separate so when we activate our other project it will set that and that environment variable to its database and when we deactivate and activate our core ems project it will correctly set the database to this database so let's activate this query MS environment really quick so if I close out of my text editor and we activate this with source activate core EMS under school web and then we echo out that database URI then we can see that our database URI now has that correct database for that project and also if we check our version of Python so I'll run a Python here you can see that this is using Python 2 7 for this project so we're able to switch

### switch between versions of python [14:02]

between versions of Python as well which is extremely beneficial for different projects ok so we're almost finished up now for most people you could stop here and you've got a great way of separating out your projects and environments from each other but there's one more thing that I usually like to set up so when you're working on multiple projects sometimes it can be easy to forget to deactivate one project environment before working on another or maybe you'll try to run tests and be wondering why all your tests are failing only to realize that you haven't activated the virtual environment for that project and it's not seeing the correct dependencies and environment variables now my personal preference is to have something in place that checks when you navigate to our project whether that project has an environment DM will file and to activate that environment if it exists now there are programs out there that will do something like this for you so maybe you've heard of things like Auto env or dirty env and tools like that but for Conda I just prefer to use my own function to do this now I modified this function from some code that I found online and I will put a link to the original function in the description section below so that they're sure to get credit but basically to do this we need to write a bash function that checks for the virtual environment and automatically activates it if it finds one and then we need to set an environment variable called prompt command that will run this custom function before each bash prompt so let's open up my bash underscore profile file and we can see how we can create this function and environment variable so I'm just going to open this up in nano and this is in my home directory and it is the dot underscore bash profile I'm sorry that's dot bash underscore profile so if I scroll down here to the bottom then we can see this function and this environment variable okay so here's the function it's call kondeh auto envy and it checks if the environment dot llamo file exists and then it is grabbing the environment name from that environment dot yeah mol file and then it's checking if we're currently in that environment by checking this Conda prefix environment variable and then if we are not in our environment already then it does a source activate on that environment name that it plucked from the environment yeah mo file and like I said this is just a modified function that I grabbed offline and I will post a link to that in the description below now I have my prompt command environment variable here commented out where I'm exporting this because I didn't want the environments to be automatically activating during the video and you wondering what was going on so now I can uncomment out this line and then if we save this I'll do a control X and ay Y to save and then I'll save that now like I said when I added that function to that prompt command variable that will run that function before every prompt it's just a special bash environment variable that does that so now to make sure that environment variable is set properly I'm going to quit out of mic terminal and then open this back up and I'll make this text a little larger here so now if we navigate to our project so within projects I'm navigating to this my project directory now I expected that to activate automatically ok it's because I misspelled environment here so I'm going to rename this file move that and spell this correctly so that is environment Djamel ok now you can see whenever I rename that to environment and spelled that correctly then that virtual environment did get activated automatically so we can see that we're in our my project a virtual environment right now so now we should have access to those environment variables that get set when this is activate and again I'm spelling these wrong sorry about that okay so now we can see that we are connected to our test database here or we have that database environment variable set so now if I navigate to my other project which is called core EMS web as soon as I navigate into there then it sets that core EMS web environment variable so now if we check that same database URI environment variable again we can see that now it's set to that different database so with that prompt command and Armit variable running that Khanda auto env function then we can see that the environment automatically gets activated so it changes the Python that we're using its using the project's dependencies and also it sets the environment variables just by navigating to the project folder that contains the environment yamo file so you don't need to deactivate the other project that you're working on or anything like that beforehand so I like this feature because it keeps you from needing to remember to switch back and forth between the correct environments or even needing to remember the name of your environment since they are in the environment yeah mol file it just does everything for you automatically okay so I think that is going to do it for this video I hope that this gave you some good ideas for how you can juggle multiple projects even if you're working on projects with different Python versions or different Python packages and a lot of different environment variables it should be easy to keep everything separate using a strategy like this now like I said this is how I personally do this but even if you aren't using anaconda or the Conda environments then you can still do something like this using virtual env and tools like Auto env so definitely see what's easiest for you to put together and see if you can get a similar workflow because most likely you'll need to learn how to handle multiple projects at some point in your career so if anyone has any questions about what we covered in this video then feel free to ask in the comment section below and I'll do my best to answer those and if you have a preferred method for how you do this then I'd love to hear about that also now if you enjoy these tutorials and would like to support them then there are several ways you can do that the easiest ways to simply like video and give it a thumbs up and also it's a huge help to share these videos with anyone who you think would find them useful and if you have the means you can contribute through patreon and there's a link to that page in the description section below be sure to subscribe for future videos and thank you all for watching