Setting environment variables in Python is an essential skill for developers, allowing them to manage and streamline the configuration of their applications.
You’re able to set, store and easily modify these variables without altering the application’s core functionality.
You can set environment variables in Python using the os module, which provides a simple interface for working them. By using this module, developers can quickly set and get environment variables, making their programs more versatile and ideally suited to changing development or production environments.
Throughout this article, we’ll discuss effective techniques for setting and accessing environment variables within your Python code and leveraging their potential to produce cleaner, more flexible applications.
From using the os module to various command line tools, this guide will provide valuable insights to streamline your development process and promote best practices across your projects.
Let’s dive in!
What is an Environment Variable?
Environment variables are variables set outside the program in its running environment by the user, operating system (OS), etc.
They are key-value pairs used by operating systems to store information about the environment, such as paths and configuration settings.
Some common environment variables related to Python include PATH, PYTHONPATH, and PYTHONHOME.
Other use cases include API keys, database credentials, and other sensitive information stored outside of the code to enhance security and maintainability.
For example, if we’re pulling data from an API like the GoogleNews API in our video below, we can use environment variables to store the API keys.
By doing this, we avoid hardcoding the keys.
Scope of Environment Variables
There are two types of environment variables: system variables and user variables. System variables are globally accessible for all users, while user environment variables are only accessible to the specific user account.
For example, the PATH variable is one of the critical system environment variables that determine the directories where the OS searches for executables, like the Python interpreter.
To effectively work with Python, it’s essential to add the interpreter’s location to the PATH environment variable.
This allows you to run Python commands from the command line without specifying the full path to the Python executable. You can typically add Python to the PATH when installing it, or you can manually modify it later.
How to Set Environment Variables in Python
There are two methods you can use in setting environment variables in Python. You can either use the OS module or the python-dotenv library.
Let’s look at how they work:
1. How to Set Environment Variables Using the OS Module
The OS module in Python provides a convenient and portable way to interact with the operating system. The os.environ dictionary is an essential part of this module, as it allows Python programs to access and modify environment variables.
It represents the environment variables of the operating system. We can use this dictionary to get, set, and modify environment variables in Python.
Let’s take a look at how it works:
1. Setting Environment Variables
To set an environment variable, you can assign a value to a specific key in the os.environ dictionary. Make sure that the value is always a string. Here’s an example:
import os
os.environ['MY_VARIABLE'] = 'hello'You can also use it to modify the value of an existing environment variable.
2. Retrieving Environment Variables
To retrieve environment variables, you can use the os.environ dictionary as a regular Python dictionary. To get the value of a specific environment variable, simply use the variable’s name as the key.
For example:
import os
home_dir = os.environ['HOME']If the environment variable exists, it returns its value. If it doesn’t, it returns a KeyError exception.
Alternatively, you can use the os.getenv() and os.environ.get() methods to access environment variables, avoiding a KeyError when the variable does not exist:
#Both of these environment varaibles don't exist.
user = os.getenv('API_USER')
password = os.environ.get('API_PASSWORD', 'default value')Output:
None
default valueThey return None if the variable doesn’t exist. You can also set a default value as the second argument in both functions if the environment variable doesn’t exist.
Remember that the OS module is an important tool for managing environment variables in Python programs. By using the os.environ dictionary to set and get variables, you can confidently interact with the operating system in a portable and clear manner.
2. How to Set Environment Variables Using the Dotenv Package
The Python-dotenv package is a valuable tool that can help you manage environment variables in your Python applications. It allows you to store key-value pairs in a .env file, keeping sensitive information separate from your application code and preventing accidental leaks or hard-coding of variables.
Also, setting multiple environment variables with the os.environ library can take quite a bit of time. By using a .env file, you can load multiple environment variables into your running environment immediately after your program starts running.
Let’s look at how you can use it:
1. Loading Environment Variables with Load_Dotenv
To begin, ensure that the Python-dotenv package has been installed. If you haven’t done so already, you can install it using the pip install python-dotenv command on your terminal.
Once installed, you can import the load_dotenv function from the package and use it to load the key-value pairs from your .env file as environment variables. These variables can then be accessed using Python’s os module.
Here’s a brief example:
from dotenv import load_dotenv
import os
load_dotenv()
access_token = os.getenv("ACCESS_TOKEN")
secret_token = os.getenv("SECRET_TOKEN")In this example, the .env file should be located in your project directory and should contain the key-value pairs for ACCESS_TOKEN and SECRET_TOKEN.
Note: When using environment variables in your Python project, it’s important to keep security in mind. Since the .env file may contain sensitive information, such as API keys or secrets, it is essential that you exclude it from version control systems such as Git.
To do this, simply add the .env file to your .gitignore file:
# .gitignore
.envBy doing so, you ensure that the .env file is not accidentally committed to your Git repository, keeping your sensitive data secure. You can read more about this module in our article on Python-Dotenv: Setting Environmental Variables.
How to Set Environment Variables via the Command Line
You can set environment variables for your program directly from your command line interface. This method provides an alternative to setting environment variables via Python.
You can also use them to set permanent environment variables system-wide. Here’s how you can achieve this in various operating systems:
1. How to Set Environment Variables in Windows
In Windows, you can set environment variables permanently using the setx command in the Command Prompt. Here’s an example of setting the PYTHONPATH:
setx PYTHONPATH "C:\path\to\your\python\directory"This will make the PYTHONPATH environment variable permanent. To temporarily set an environment variable, use the set command:
set PYTHONPATH=C:\path\to\your\python\directoryAlternatively, you can set environment variables through the System Properties window, under “Environment Variables.“
After setting the environment variable, you can retrieve them using the echo %<variable-name>% format. For example:
set PYTHONPATH=C:\path\to\your\python\directory
echo %PYTHONPATH%Output:
To list all environment variables, you can simply run the set command in the terminal.
2. How to Set Environment Variables in Unix (Linux & macOS)
In Unix-based operating systems such as Linux and MacOS, the export command is used to set environment variables. Here’s an example of setting the PYTHONPATH:
export PYTHONPATH="/path/to/your/python/directory"This will set the environment variable for the current session. Once you terminate the shell, the environment variable will be deleted. To make it permanent, you can add the export command to your shell configuration file (e.g., .bashrc, .zshrc, or .profile).
To get the value of an environment variable, use the echo command:
echo $PYTHONPATHTo list all the environment variables, you can use the printenv command.
What is the Scope of an Environment Variable in Python?
In Python, environment variables’ scope is limited to the process they belong to and any subprocesses spawned by the parent process.
When you modify an environment variable within a Python script, its changes only affect the current process and the spawned subprocesses, but not the calling process or the entire system.
For example, you can temporarily modify the current process’s environment using Python’s os module:
import os
# Store initial PATH value
initial_path = os.environ['PATH']
# Set a new PATH value for the process
os.environ['PATH'] = '/new/directory:' + initial_path
# Perform actions using the updated PATH
# Restore initial PATH value
os.environ['PATH'] = initial_path
It’s important to note that modifying environment variables within a Python script will not change the overall system or user environment permanently.
This temporary modification mainly helps in managing dependencies and enabling specific behaviors for sub-processes, such as changing the Python startup behavior or configuration.
Advanced Environment Management in Python
Having navigated the foundational concepts of environment management in Python, you’ve established a strong footing in ensuring project consistency and avoiding dependency conflicts.
However, as we venture into more intricate and large-scale projects, a deeper understanding becomes paramount. This section builds upon that foundation, diving into the nuances and sophisticated techniques that cater to the multifaceted demands of complex Python endeavors.
1. Handling Newlines in Environment Variables
When working with environment variables in Python, you might encounter issues with newline characters. Newlines can cause unexpected behavior, especially when passing environment variables to subprocesses. Therefore, it’s essential to handle them correctly.
You can remove newlines using the str.strip() function:
import os
raw_value = os.environ.get("MY_VARIABLE", "")
clean_value = raw_value.strip()
2. Default Values in Python Code
It’s common to use default values for environment variables in case they aren’t set in the system environment. This can be done using the os.environ.get() method or the os.getenv() function:
import os
# Using os.environ.get()
default_value = "default"
my_var = os.environ.get("MY_VAR", default_value)
# Using os.getenv()
another_var = os.getenv("ANOTHER_VAR", default_value)
Both methods retrieve the value of the environment variable if it exists, else return the specified default value.
3. Working with Child Processes
When creating subprocesses in Python, you might need to pass environment variables to them. The subprocess module provides an easy way to do this:
import os
import subprocess
# Set environment variables
os.environ["API_USER"] = "username"
os.environ["API_PASSWORD"] = "secret"
# Pass environment to child process
env = os.environ.copy()
result = subprocess.run(["your_command"], env=env)
In the above example, the os.environ.copy() function is used to create a copy of the current environment, which is then passed to the child process.
This method enables the child process to inherit the environment variables from the parent process while avoiding inadvertent changes to the global system environment.
Final Thoughts
The importance of configuring environment variables in Python can’t be overstated. Through the utilization of the os and python-dotenv modules, this process remains both accessible and efficient.
In a digital landscape where adaptability is key, Python’s elegant approach to managing environment variables underscores its position as a language of choice for pragmatic and effective programming.
That’s the end of our article on environment variables in Python. If you enjoyed be sure to read through our article on How to Delete Files from Python: 6 Easy Methods Explained.
Frequently Asked Questions
What is the proper way to set environment variables permanently in Linux?
In Linux, you can set environment variables permanently by adding export statements to the ~/.bashrc or ~/.profile file. Open the file with a text editor and append the following line:
export MY_VARIABLE=my_value
Save and exit the file. To apply the changes, either restart the terminal or use the command source ~/.bashrc or source ~/.profile.
What is the method to print all environment variables in Python?
To print all environment variables in Python, you can iterate through the os.environ dictionary after importing the os module:
import os
for key, value in os.environ.items():
print(f"{key}: {value}")
Do Python environment variables work in a virtual environment?
Yes, environment variables in Python work can work in a virtual environment. You can access and set them in the same way as in a regular environment.
You can even add the environment variable into the virtual environment’s activate.bat file for easy loading. This way, every time you load up the virtual environment, the program sets the environment variable.
You can append the export statement at the bottom of the script. For example:
export KEY = VALUEAre environment variables set with Python?
No, environment variables set with Python are not permanent. They do not persist outside of the Python process.
Once the script or shell stops running, the environment variables are deleted or reverted to the system defaults.
