Difference Between environment and env_file in Docker Compose

When spinning up services with Docker Compose, you'll inevitably need to set environment variables. Docker Compose offers two main avenues for this: the environment key and the env_file directive. While they both get variables into your containers, they approach it differently. Let's break down which to use and when.

1. The environment Key: Direct & In-File

This method lets you define environment variables directly within your compose.yml file.

services:
  myservice:
    environment:
      - VAR1=value1
      - VAR2=value2
      # Or using mapping syntax
      VAR3: value3
      VAR4: value4

Pros:

• Visible: Variables are right there in your compose file.
• Service-Specific: Great for configuration unique to a single service.
• Interpolation Power: Values can use Docker Compose's variable interpolation (e.g., API_URL: http://localhost:${HOST_PORT}).
• Runtime Substitution: Supports substituting variables from your shell at runtime.

Cons:

• Clutter: Can make your docker-compose.yml lengthy.
• Sensitive Data Risk: Secrets become visible in your compose file (bad for version control!).
• Less Reusable: Not easily shared across different services or projects.

2. The env_file Directive: External & Organized

This tells Docker Compose to load variables from one or more external files.

services:
  myservice:
    env_file:
      - ./common.env
      - ./service-specific.env

And your common.env might look like:

VAR1=value1
VAR2=value2

Pros:

• Cleaner Compose: Keeps your compose.yml tidy.
• Reusable: Easily share configurations across multiple services or even projects.
• Better for Secrets: Keeps sensitive data out of the main compose file (though remember, .env files with secrets shouldn't be committed to Git!).
• Organized: Group variables by purpose into different files.

Cons:

• Indirect: Variables aren't immediately visible in the compose file.
• No Interpolation (in the file): Docker Compose interpolation (like ${VARIABLE}) doesn't work inside the .env file itself.
• Extra Files: You'll have a few more files to manage.

Key Differences at a Glance

• Precedence Rules:
  1. Variables set via environment override those from env_file.
  2. If you list multiple env_files, files listed later override those listed earlier.
• Syntax:
  • env_file: Uses a simple KEY=value format. No quotes are needed around values. Comments start with #.
  • environment: Uses YAML list or map syntax.
• Location:
  • environment: Defined directly in docker-compose.yml.
  • env_file: Variables live in external .env files.
• Interpolation:
  • environment values can use Docker Compose interpolation (e.g., VAR: ${HOST_VAR}).
  • The content of .env files cannot use this interpolation (e.g., VAR=${OTHER_VAR} inside the .env file won't work as expected).

Best Practices: When to Use Which

• Use env_file for:
  • Shared configurations (e.g., database credentials used by multiple services).
  • Environment-specific settings (e.g., dev.env, prod.env).
  • Sensitive values you want to keep out of your docker-compose.yml (and out of version control if they are truly secret).
• Use environment for:
  • Service-specific overrides for values perhaps set in an env_file.
  • Variables whose values depend on Docker Compose interpolation.
  • A few, rarely changing, non-sensitive, service-specific values.

The Power Combo:

Often, the best approach is to combine them!

services:
  app:
    env_file:
      - ./common.env
    environment:
      - SPECIFIC_VAR=override_this_value
      - DEBUG_MODE=${DEBUG_FROM_SHELL:-false} # Interpolation example

This gives you the maintainability of env_files with the flexibility to override or add specific, interpolated values directly in your compose file.

By understanding these distinctions, you can craft cleaner, more secure, and more maintainable Docker Compose setups. Happy composing!