Good to see you here @webmozart :)
We already have this distinction in core: parameters are for static values, and env vars for runtime-configurable values. I know ppl are abusing env vars, but adding a third concept won't help I fear.

@nicolas-grekas I think this confusion comes from the fact that the vcs-ignored parameters.yml file has been removed from the flex skeleton (compared to the standard-edition skeleton). this means that by default, you have 2 choices: env variables or vcs-managed parameters. But there needs to be a room for per-setup static values (where various deployments might need a different value for the parameters).

@webmozart my solution to this problem is actually to reintroduce the parameters.yml file in my project for static values, managed with https://packagist.org/packages/incenteev/composer-parameter-handler (disclaimer: I'm the maintainer of that package), as done in the old symfony standard-edition skeleton.

It is officially recommended to dump the environment variables to a PHP file in production. Hence my assumption would be that the majority of Symfony users is using mostly static variables.

Overall, this approach is not endorsable, and goes against https://12factor.net/codebase (one source, multiple deployments), which is sensible when you have one application to be re-configured differently depending on, as it may be surprising, the environment.

I only use ENV vars, and also removed bindings for symfony/dotenv from bin/console and public/index.php in each system I maintain, precisely to avoid this confusion (as @nicolas-grekas just stated, it is an additional concept).

What remains is the annoyance of running tasks like docker build or composer install within scopes where the environment is not fully bootstrapped, highlighting that sometimes the environment variables are being read even when they should only be referenced (too eager container build process - this should be treated as a bug).

The workaround I found to be working for this is to define the environment variables with defaults that don't affect the system too much:

# put this anywhere you want in your configs
parameters:
    # empty on purpose - hides absence of env variable
    env(DATABASE_URL): ''
    # or put a fake value in it - I'm a github comment, not a cop
    #env(DATABASE_URL): 'mysql://localhost'

doctrine:
    dbal:
        # use your env vars as usual
        url: '%env(my:preprocessors:here:DATABASE_URL)%'

Alternatively, declare ENV with neutral values within your Dockerfile, in the build process.

This requires a lot of dancing around changes in upstream bundles, but seems to work quite reliably for the build process. Ideally, environment variables should not be touched during compilation steps, but mistakes in upstream are not always avoidable.

@Ocramius but then, this highlights the need for a way to specify build-specific parameters in a different way than env variables, for settings that must affect the way the container is built. Bundle configurations are not always passed as is to service arguments to be used at runtime. They can also affect the way services are wired (and in such case, env placeholders don't work)

They can also affect the way services are wired

Generally best to use a runtime factory, if such a runtime decision is needed 🤔

What sometimes happens is:

• build process attempts to access a service (logger, for example)
• build process attempts to read database schema

These are supposed to not happen during build time at all. Perhaps what's needed is preventing usage of %env()% when a parameter is to be used in an extension, by having a bundle/container extension declare when a value is accepted as dynamic, and otherwise they should all be considered static by default? (BC break, obviously - lots of reasoning to be done there).

In practice, if I do following, and the bundle did NOT explicitly declare use_profiler as dynamic:

• use_profiler: %env(USE_THE_PROFILER)%

Then what I could get as useful feedback is a DynamicValueUsedInStaticExpression exception (thrown by the DIC build process).

This could potentially allow for more aggressive inlining too.

@webmozart

Invalid type for path "web_profiler.toolbar". Expected boolean, but got string.

Got the same problem here when using PrependExtension: #40198. I could only solve a part of it here, but not the usage with env variables yet.

I understand the point with parameters vs env vars, but could not also bool values be this type of env vars, because for me currently most of the time it looks like only string value can be env vars and the dependency injection can not work if it is bool, int, float or other typed values.

@Ocramius requiring bundles to declare explicitly in their configuration which settings support env placeholders and which don't support it is a separate topic (it would indeed provide better feedfack when using an env placeholder in an unsupported place, but it is hard to introduce in a BC way as we would have to support an intermediate state where bundles have not updated their config definition yet). But that's not solving this issue if we don't provide an official way to configure static values

@alexander-schranz if you need a boolean value for which the value is only needed at runtime, you can do it with env placeholders, by using %env(bool:MY_ENV_VAR)%

@stof but this doesn't work when the value is given from a bundle as value to a PrependExtension which only expects a boolValue when the extension is parsed with:

        $resolvingBag = $container->getParameterBag();
        $configs = $resolvingBag->resolveValue($configs);

it fails with Expected "bool", but got "string". See also here.

Overall, this approach is not endorsable, and goes against https://12factor.net/codebase (one source, multiple deployments),

I don't think that's true. The .env.local file should not be included in VCS and dumping should happen in the deployment environment, not before committing to VCS.

For other readers: @nicolas-grekas pointed to his presentation on Twitter that I find quite informative: https://speakerdeck.com/nicolasgrekas/configuring-symfony-from-localhost-to-high-availability

The gist is:

• Use (DI) parameters for business requirements
• Use env vars for infra-related configuration
• Use .env files first, increase complexity later
• Use bin/console secrets:*

There are various problems with this approach:

• Bundle authors need to explicitly support env vars in configuration that you consider infrastructure-related, even if that configuration does not depend on the runtime
• Some business requirements settings may need to be protected, but secrets are only supported for env vars
• Some business requirements vary from deploy host to host (same app installed for different clients)

Especially for the last reason, the parameters.yml needs to be reintroduced (as recommended by @stof) and integrated in the deploy process, i.e. during deploy, we need to build a parameters.yml with business settings and a .env.local with infrastructure settings. This isn't very simple, let alone for new users.

Using static env vars by default would flatten the learning curve:

1. Use .env and .env.local for all configuration that varies from deploy host to deploy host (both infrastructure and business)
2. Run composer dump-env for performance optimization
3. Once you scale out and need dynamic information from the runtime, change %env(XXX)% to %env(runtime:XXX)% only where you need it

Much simpler IMO.

@webmozart Should not the bundle define which env variables would be able to be changed at runtime and which are directly resolved when the container is build e.g.:

                        ->booleanNode('enabled') // config option which does not support runtime change of env
                        ->end()
                        
                        ->booleanNode('someValue')
                            ->lazyEnv() // or ->runtimeVariable()
                        ->end()

Because I think if the bundle does something like the lazy: can not be used:

if ($config['enabled'])) { // when env is used the resolved env variable is given here so this will always be bool with true or false and no call for resolving itself is needed
     $loader->load('some-services.xml');
}

// for the config lazyEnv the env placeholder is given here so its changeable on runtime
$container->setParameter('some_env', $config['someValue']);

At this case the bundle will always get the resolved values and only defines the config values which are changeable at runtime be given to the bundle as env placeholders.

@alexander-schranz Yes, although I would probably take the opposite approach and only mark those configuration options that explicitly don't support runtime variables. The status quo is that bundles will fail anyway if you pass runtime variables in those options. Adding validate()->noRuntime()->end() or similar will simply provide a better error message.

Requiring all bundles that already support runtime variables today to add runtimeVariable() flags is a BC break.

@webmozart the problem I see there is that if we automatical say all config options are possible as env vars something like this will fail:

if ($config['enabled'])) { 
     $loader->load('some-services.xml');
}

The $config['enabled'] will return by symfony in the following format like __env1413512124 (not remember the format) also when set to false when e.g.: the config and env var is:

MY_ENV_VAR=false

my_config:
     enabled: "%env(bool:MY_ENV_VAR)%"

So I think it would better to define for which $config variable I expect that symfony does not resolve the env variables instead. I think this would make it a lot better for bundle developers as they will not longer get an unexpected value for some config parameter.

Requiring all bundles that already support runtime variables today to add runtimeVariable() flags is a BC break.

Sure this is a BC break and could only be done in the next release or can be enabled over a flag in the config rootTree.

The main design issue is Symfony doesn't have a distinct build vs run phase, even though it obviously has one implicitly.

Having a build phase would mean requiring a reference to a env var to be fully resolved would lead to an exception because env variables are not available during the build phase.

We discussed this in some detail in Symfony Slack already, it was dismissed then too.

see #28896, i've re-opened it :)

I'm can't say for sure that I grasp fully the scope of this, nor I am fully up to date with what the best practices are or the most commonly used patterns, but here's my feedback:

• the situation atm with Symfony config is imho quite complicated, possibly too much. Flexible, for sure, but I find myself spending hours hunting down the exact file / env var / build script which is the origin of a specific value (I mostly develop on ezplatform, which abuses symfony config a lot, but I am sure it is not the only case)
• having env vars which can be declared/overridden in .env/.env.local files adds a certain amount mental burden, as env vars can be injected via other means, and then the problem of priority pops up (real env vars vs .env permutations)
• adding a lot of mental burden is: trying to figure out which vars/settings get resolved at container compile time, and which ones at runtime. If I make all of the settings depend on parameters, and I set the value of those parameters using env vars, will the app become dead slow?

The proposal from @webmozart seems to me to add one more layer of complexity adding a concept which is quite unusual "static env vars" as it clashes with the way the rest of the OS works.
Maybe it could be rewritten in a way to make more sense to the laymen by distinguishing compile-time parameters from runtime parameters ? (*)

More specifically:

Bundle authors need to explicitly support env vars in configuration that you consider infrastructure-related, even if that configuration does not depend on the runtime

I thought that bundle authors would only have to ever support parameters to drive the config of their bundles, and the app developers would be left with the task of having config that sets the values of those from env var values.

(*) = a similar scenario: in my docker-compose builds I have come to use a single .env file to declare env vars which are used to configure everything, from mount volumes paths, to versions of php in use, to ports to expose, etc... All of the vars are used in docker-compose.yaml, some being passed on to the containers as env vars, some as variables to the dockerfiles, and hence only usable at image build time. The simplicity of having a single config file is unrivaled. However, when changing one var value in the .env file, sometimes a container restart is enough, sometimes a rebuild is needed. I found no better way to surface this requirement than making the .env file heavily documented...

adding a lot of mental burden is: trying to figure out which vars/settings get resolved at container compile time, and which ones at runtime

IIUC that be

debug:container --env-vars from the container POV
debug:config --resolve-env (#40582) from the config POV

@ro0NL thx, but those are not available in all Smfony versions I am using (currently migrating one app from 2.8 to 3.4...)

@gggeek 2.8 does not support env variables at all...

@stof yep. Otoh 3 does, and that's where the story about parameters started to get very complicated (imho of course)