Where .env Went Wrong

82 points62 comments4 days ago
chacham15

This article reads like more of an ad than anything else.

> Environment variables only deliver values

Yes, the problem .env files try to solve is having "environment variables" be injectable from a file so that different applications can have different environment variables by default.

> A string is not a schema

Yes, input validation is an application concern. The application should know what these values represent / how to parse them and error if they're invalid.

I could go on, but the article is all about trying to use a hammer as a screwdriver and complaining that the hammer doesnt work.

show comments
eigencoder

Hm, I haven't seen these issues personally. We only have one `.env` file and it's just for local secrets. Configuration emphatically does not go in `.env` and ideally is in docker compose and defined in code (we use Pydantic Settings).

show comments
leftnode

I suppose I'm spoiled by the Symfony framework which has a robust Secrets Component [0].

It allows you to generate a public/private key for each environment: test, dev, and prod (by default). The 'prod' private key can't be committed to the repository. Once the cryptographic keys are generated, you can encrypt any secret you want, and the framework will automatically decrypt it and allow you to access it as an environment variable at runtime.

The encrypted values for all environments are committed to the repository. For example, the file for the `$STRIPE_SECRET_KEY` environment variable for the 'dev' environment is a file that has these contents (shortened for readability):

    <?php // dev.STRIPE_SECRET_KEY.c33678

    return "\x1Dq\x11B6\x5B\xFE7\x9B\xA4\xFE...";
This solves several problems:

1. Keys for the 'dev' and 'test' environments can easily be shared with the team because their public and private keys values are committed to the repository. No more sharing secrets file over Slack or through some other mechanism.

2. The private key for the 'prod' environment can be stored in a 3rd party vault so it can be made available to the production servers. You can also decrypt 'prod' secrets during deployment to reduce the decryption overhead for each request.

3. At worst, agents would have access to the secrets in the 'test' and 'dev' environments only because they couldn't decrypt 'prod' values locally.

4. It forces good practices to ensure you're not using the same secret value for production and non-production environments.

It's made secret management so easy I don't even give it a second thought. Do other web frameworks support a system like this?

[0] https://symfony.com/doc/current/configuration/secrets.html

ghshephard

I'm guessing my brain has been permanently warped by working in a hashistack environment for the last 5+ years. All of these KV pairs are always either in a consul (bog-standard config information) or vault (secrets/credentials) in a consul-template associated with the job - and rendered at run time. And, of course - critically - the consul-template updates those credentials and values as they are routinely rotated.

I did a ^F for "update" and "dynamic" - didn't find any hits in that doc. Managing your dynamic environment values is table-stakes in most large deployments - and I didn't see any reference in `secretspec` as to how they plan to do that.

osinix

Having a crowded and sloppy environment, is not a good way to go. When you log in to your account, you should not source a bunch of environment files by default. It is always a good practice to have a simple environment, just whatever you needed be sourced. If you need extra, just source them before using them. Keep your home clean.

c-hendricks
show comments
ctippett

I'm sure there are many better alternatives to .env, but its ubiquity and support across various tools makes it super convenient.

I'm using 1Password's .env integration[1] and although the UX is a bit clunky, I really like it. My API keys are secure, tools that ordinarily support .env just work and there's a team-sharing feature too (although I'm yet to use it). It's pretty neat.

[1] https://www.1password.dev/environments

ramses0

I wish there was better support for something like `SECRET_CMD="..."` (ie: "run this command to get/refresh secrets"), but that's just baking arbitrary command execution into your development pipeline.

Personally, I tend to have a `source ./source-me-auth` which does stuff like `DB_PASSWORD="$( pass show blah.com | head -1 )"`, but that still means I end up with secrets dangling around in my environment. The "source-me" file is safe to share, but my runtime environment is tainted (for good and bad...).

I saw something w.r.t. the way open-claw handles things where you basically say `export DB_PASSWORD="REDACTED@SECRET_001"` and then it basically outbound filters network requests to hydrate that with `s/REDACTED@SECRET_001/$REAL_SECRET/g`, and I _really_ like that mechanism b/c it defers secret usage to runtime and keeps it physically "out" of the application itself.

Basically, having `with $SECRETS -- some_app.sh --some params ...` wouldn't be terrible (conceptually).

theozero

Over at varlock (https://varlock.dev -- also free, open source), we agree that .env as we know it is full of problems. But instead of abandoning it, we evolved it. We replace your .env.example with a .env.schema - using decorator style comments to add schema info, and functions to load and compose values.

A big difference between our tool and many other similar tools is that we combine the schema and value setting into one surface, with a way of merging many definitions together, much like cuelang - but in a way that feels more intuitive. It's extremely flexible, and can even do credential brokering for untrusted workloads.

I've been enjoying the secretspec content lately, and watching it evolve :)

show comments
wonger_

I'm working with .env and secrets and teamwide configs for the first time, trying to understand solutions from first principles.

Are these valid observations so far?

- .env is the simplest and oldest / most boring solution. Secrets are passed to teammates by DM?

- .env.example seems like a nice home for documentation about secrets

- but I'd rather avoid writing sensitive credentials at all -- instead, teams can use a cloud-based secrets manager and the project fetches secrets at runtime. Is this ever a hassle / any downsides? How standard is this practice these days? I hear this also helps updating secrets so you don't have to e.g. tell every teammate when you rotate a secret. Generally not a fan of introducing a network call and a saas dependency, tho

- non-sensitive environment variables also need a home. I don't like the idea of cluttering the project with a .env.local or worse like .env.development.local

- I hear lots of secrets and config management happens in container orchestration, but what if I barely have a container in the first place? I guess docker compose is one of the simplest tools at this layer?

I guess I'm just looking for a safe, simple solution for a small team, and maybe the problem is that every team does things differently, and that many tools are marketed towards huge enterprise teams.

show comments
weinzierl

I see many .env files with unquoted values, like the first example in OP

   REDIS_URL=redis://localhost:6379
and it triggers me big time. Always quoting shell variable values is so deeply ingrained from very painful experiences decades ago that it still causes a somatic flashback whenever I see it.

This is what went wrong with .env

Yes, I know

    docker --env-file 
doesn't work with quotes but that's just docker being broken and why they fixed it in docker compose.

Also see jt2290's comment:

https://news.ycombinator.com/item?id=49171684

chis

92% of this text is detected as AI.

It may be time for Hackernews to integrate a Pangram detector into the UI, similar to what substack is doing :)

show comments
prologic

This is why I love direnv (the tool) and just a simple .envrc (not .env). It has none of the problems posited in the OP's article. It assumes nothing, only that direnv manges loading up the .envrc into your shell.

show comments
stilwe

Hey cool, a relevant place to mention my project: https://dotprot.dev/

Type "dotprot" to copy your .env file into 1Password using the 1P CLI, it verifies it's there, then deletes the local copy of .env. When you're ready to work again, type "dotprot" again to restore the .env file from 1P. There is a .prot file in the directory that you can declare other files to store in 1P as well.

Varlock (https://varlock.dev ) is also good for this and more mature. I use it on some work projects, but for local dev, I wanted something to allow me to be even lazier than that.

show comments
throw-the-towel

I can't understand for the love of me why our industry settled on env as The Way to store configuration. It's poorly discoverable, it's a shared namespace polluted by everyone and their `libdog`, it's stringly typed, etc. All of that just flies in the face of otherwise accepted dev wisdom.

I love Django's approach to configs: your config is just a plain old Python file with some constants. As simple as possible, and works great.

show comments
qntty

I use Pydantic Settings for this, would be nice to see a comparison to that. I do like the idea of keeping this in a configuration file.

show comments
KevinMS

> Where .env Went Wrong

start with making it a dot file. Why would you want to hide the fact your app was using loaded environment variables?

show comments
Brainspackle

You can easily use comments in the file for guidance, eliminating half your complaints

Natfan

Claude, find a key underpinning of most development workflows, then make a product to disrupt (and eventually SaaSify/enshitify) it. Finally, write a convincing ad disguised blog post, and purchase a fancy domain. Here's my credit card.

show comments
jonny_eh
gaigalas

Likely a 12 factor legacy issue.

https://12factor.net/

This was really popular, and lots of people bought the factor III of only configuring via environment variables.

The whole point of it was: keep config simple. Env vars are austere and will keep you from complexity.

But people found a way to complicate them, so they could have both the 12 factor literal pass and the complexity they wanted.

That resulted in the mess we are today.

TZubiri

.env files are fine, as long as you don't import a random dependency to open and read a file. If you think about it for ten seconds, it's unjustifiable, and it's only done because others do it.

tosti

One could also... Not jump on the bandwagon?

(Am I getting thrown out the window now? :)

show comments
0xbadcafebee

It's interesting how different roles come up with different solutions based on their own experience.

Developers came up with .env because they would run apps on their own machine and wanted to pass variables to their application without having to set them in a command-line or environment-variable every single time they ran their app. And it works great for that. If you're one developer, running something locally, sure, just read some lines from a file. And if you're a team of developers, and most of you have the same lines you want to use, but maybe just a few of them you want to change per developer machine, fine, either keep .env out of Git, or keep a .env.local for the non-Git stuff. Again, simple, works fine.

Then you want to run your apps in production, with different lines. And then you want to run it on a test/stage/qa machine too - again, different lines. And maybe you hard-code those into separate files (.env.prod, .env.staging, .env.test) and load them depending on which of the 2 or 3 machines you have running your app. Again, simple, works fine.

Until the problems start.

Secrets in the .env? Now those are in the code, which gets cloned everywhere, and can be stolen. "Encrypted" secrets in .env? Now you have to manage a secret key out of the file, in addition to the encrypted secret in the file. Everyone has access to the secret? Now people can use those secrets, or access different machines, potentially creating problems or exceeding their authority, and there is no way to know who did what because it's one shared secret. Somebody leaves the company? Now you need to rotate the secret (which nobody does).

Ephemeral containers/deployments? Now the .env entries don't match the new hostname. Want to support multiple hosts? One hostname in the .env line doesn't work anymore. Want to scale horizontally? Now your "prod/test/dev" files are more of an environment type than a specific host. Your RDS database's hostname, or an old IP address, has changed? App is broken, time to update all the .env lines referring to it and re-deploy the app.

These are all problems that you might or might not run into. But they are problems that do exist in the world; we know they happen, and we know how to avoid them. You decide whether you're going to wait for them to bite you, or avoid them altogether from the very start. If you do the former, you're acting like a Systems Engineer, designing a system to be resistant to known failures. If you do the latter, you're not.

You may not want to do extra work you feel is unnecessary just to avoid a possible problem. But other professions do this anyway, by regulation, because (for example) as a society we we don't want to allow houses to burn down from a preventable problem. Example: If you run wire in a conduit, the conduit must be a minimum size depending on the number and type of wires you run in that conduit. You may think it's annoying that you have plenty of space left in your conduit; why should I have to change my wire size or conduit size? But if the wires take too much current, and don't have enough airflow/space between them, they can heat up and start a fire, or a short, which could cause a larger problem somewhere else (like taking out a hospital's ventilators, as one example). Doing the extra "unnecessary" work prevents fires. That's why Systems Engineers don't use .env files. Their job is to build reliable systems, not just roll out a feature and cross their fingers.