The Ultimate PATH Management Guide

24 Jan 2017

Basics of Environment Variables and PATH

Each process has an environment, a set of variables that the process can read or write. When the process spawns another process, the new process inherits the environment of its parent. These values are nevertheless local: the child inherits a copy of the variables of the parent at creation-time. If the parent later changes his variables, the change will not be reflected on the child's variables.

One particularly interesting environment variable is called PATH: it defines the paths to search to find an executable specified by name. The value of this variable is a list of paths separated by a separator (: on Unix-based OSes like Mac OS X and Linux, ; on Windows).

This is particularly relevant on the command line: when you type the name of a program (for instance ls in Unix), it will search the paths listed in the PATH variable (from left to right) until it finds one containing a ls executable.

We'll now see how each OS handles environment variables and the PATH. One particular question that will interest us is how the "default" value for the environment variables (and the PATH in particular) are set.

Windows

In Windows, the initial values for environment variables are stored in the registry.

These are separated between system values and user values. If a variable has both a system and user value, the user value takes precedence, excepted for the PATH, where both values get concatenated (user PATH, ;, system PATH).

• User environment variables are stored in HKEY_CURRENT_USER\Environment.
• System environment variables are stored in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment.

explorer.exe will immediately reflect any change to these values, and any program launched by it will inherit these values.

All other programs, on the other hand, need to be restarted. A few comments about programs of interest:

• The Launchy app launcher automatically updates the environment before launching a program.

• The ConEmu console emulator is by default configured to hijack the opening of new consoles and open them in the existing ConEmu window instead (this is a good thing). The problem that is when intercepting consoles opened by explorer.exe, the environment seems to be set to that of the ConEmu window (i.e. the environment when a console was first launched). See this issue.

How to change the environment

• Locally in a console: use the built-in set command. You can clear a variable by assigning it nothing: set X=.

• Globally in a console: use setx (built-in Windows 7+, download for previous versions)

• From the explorer, follow these instructions.

• Use the excellent Rapid Environment Editor

Running scripts on the command line

On Windows, it is customary to append an extension to executables binaries (.exe) and scripts (.bat, .sh, .py, ...). However, typing this extension to run the program is not very desirable.

There is an environment variable called PATHEXT which holds a list of all extensions that can be run on the command line. Basically, if a file has its extension in this list, you can type its name ommitting the extension and it will run as though you double clicked it. There might be some additional finesse here, but it will be like this most of the time. The order of the extensions inside PATHEXT is important if there are two files with the same name but different extensions. Extensions earlier in the list have more priority.

Unix

Both Mac OS X and Linux are Unix-based OSes. As such, the basic principles of how environment variables are handled is similar. I'll explain this here before talking about the peculiarities of each OS.

In Unix, a process does not automatically inherit all the environment variables of its parents. It only inherits those that are explicitly exported.

In a shell script, or on the command line, we can mark a variable X for export by writing export X. This can also be combined with the definition of the variable: export X="some stuff". You can un-export something with export -n. Removing purely and simply a variable is done with unset.

On Unix, environment variables are initialized by shell scripts. Some scripts are executed each time a shell is run. Which scripts are run depends on whether the shell is a login shell and wether it is interactive.

The rules are rather complex, but this page does a good job at summarizing them.

Also beware that your scripts may source other files, including those that mentionned in that guide.

When taking inheritance into account, usually the right thing to do is to define your environment variables in ~/.profile or equivalent. One caveat: the file won't get sourced when running a command remotely via SSH (that is, when running a command through ssh without creating an interactive shell).

Reloading the environment

Reloading the environment in an existing shell after you edited .profile can be tricky. One easy solution would be to re-source .profile, but there are a few caveats:

• Sourcing your .profile should not have unwanted side-effects.

• Often variables are not just set, but are modified. This is typically the

  case for PATH, where you append to the existing definition.

  In this case, if you reload the .profile, you'll end up with many duplicate entries in your PATH, and you'll fail to clear entries you want removed.

Hence: no side-effects; set variables, don't modify them.

Mac OS X

OS X has a fantastic little utility called path_helper that will help you manage the PATH. Basically what it does is read the files in /etc/paths.d and appends their content to the PATH (one entry per line). Files are read in lexicographic order, so you can control the order of entries in the PATH.

Ironically, path_helper is not itself on the path, so you must call it through its fully qualified name /usr/libexec/path_helper. The output of path_helper is a shell command to set the PATH. This command must itself be evaluated.

I suggest to define the following shell alias for ease of use:

alias refresh_path='eval `/usr/libexec/path_helper -s`'

The -s part is just to ensure a bash-compatible command is generated.

By default, OS X calls path_helper in /etc/profile so you do not need to add it yourself.

Oh and by the way, path_helper does the same thing it does to PATH to MANPATH (which holds the paths used to find manpages) but with the /etc/manpaths.d directory.

Final word of caution: /etc/paths.d requires root permission to write into. However, the files must be readable without root permissions to be used by path_helper. Hence you might need to do a chmod 644 on the files you create in that directory.

See this webpage for more details about path_helper

Setting the PATH for GUI applications

Ways of defining the PATH for GUI applications on OS X have a history of breaking. You may read on the net about ~/.MacOSX/environment.plist or launchd.conf, but those no longer works.

"The" solution, at the time of writing, is to create a file ~/Library/LaunchAgents/environment.plist, enter a bunch of boilerplate and include launchtl setenv commands to set environment variables. See this explanation for details.

After a change, you can make it take effect with launchctl load ~/Library/LaunchAgents/environment.plist (or wait for a reboot).

Once applied, new applications started from the Finder (or Spotlight) will inherit the environment variables set in environment.plist.

Note that what environment.plist does is simply run a script. You could use that to actually source your .profile or invoke path_helper. Two solutions that work with this idea (but doing rather more complicated things) are to be found here and here.

The difficulty to achieve something so basic is truly baffling.

Linux

There are too many desktop managers for Linux to cover. Cursory searches seem to indicate that this aspect isn't too well thought-out however.

However, Linux should, in theory, be more scrupulous than OS X about inheriting the PATH, even in GUI applications. Which means that, editing .profile then login out and login in should work as a last resort.

Some distributions (notably Debian, Ubuntu, Fedora) have files called ~/.pam_environment and /etc/environment where they recommend you define your environment variables. The benefits of this way of doing things are rather unclear, since it doesn't seem to enable anything that couldn't be done by defining variables in .profile.