Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Console] Bash completion integration #42251

Merged
merged 2 commits into from Oct 19, 2021

Conversation

@wouterj
Copy link
Member

@wouterj wouterj commented Jul 24, 2021

Q A
Branch? 5.4
Bug fix? no
New feature? yes
Deprecations? no
Tickets Fix #38275
License MIT
Doc PR -

This is a first shot at implementing interactive bash completion support in the Console component. Besides completing option and command names, commands can override Command::complete() to implement completing values.

I've added an example code to the secrets:remove command, which now supports autocompletion quite nicely: render1630315116886

And support for other applications using symfony/console is automatically included (if the autocompletion script is installed for that specific application):
render1630314403752

This PR only implements Bash completion. Zsh and Fish have much more sophisticated completion systems, but I propose to keep those for a future PR if there is a need for this.

How it works

  1. A bash completion function (_console) is defined by bin/console completion bash for the console command (name of the "binary" file)
  2. This completion function calls the local bin/console _complete command to come up with suggestions
  3. Bash parses these suggestions and shows them to the user.

This has one drawback: the _console function is defined globally only once. This means we cannot easily change it, as it would break if you run different Symfony versions. We should probably add versioning (e.g. bin/console _complete --version=1.0) and don't suggest anything if the version doesn't match.

Maybe it would be safer to mark this feature as experimental and target 6.0, to allow us to fine tune the shell related sides over the lifespan of 6.x? #42251 (comment)

Steps to test yourself

Load this PR in your project, open a bash shell and run this command to "install" completion for this project:

bin/console completion bash > /etc/bash_completion.d/console

Then reload the bash shell and enjoy autocompletion.

TODO

  • Autocompleting in the middle of the input doesn't work yet (i.e. bin/console --en<TAB> cache:clear)
  • Better error handling
  • Add a bin/console completion command to dump the _console file, so users can install this locally
  • Add some versioning, to allow us to change the _console file in the future
  • See how we can better support standalone usage (e.g. Composer) Tested on Laravel's artisan, works flawlessly
@wouterj wouterj requested a review from chalasr as a code owner Jul 24, 2021
@carsonbot carsonbot changed the title [WIP] [Console] Bash completion integration [Console] [WIP] Bash completion integration Jul 24, 2021
@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch 2 times, most recently from 47efd2b to 0623991 Jul 24, 2021
@Nyholm
Copy link
Member

@Nyholm Nyholm commented Jul 25, 2021

Thank you.

This looks super cool. =)

@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from 0623991 to 5986f27 Jul 25, 2021
@wouterj wouterj changed the title [Console] [WIP] Bash completion integration [Console] Bash completion integration Jul 25, 2021
@wouterj
Copy link
Member Author

@wouterj wouterj commented Jul 25, 2021

Thanks :)

I've updated this PR to complete most of the important TODO items (and also updated the usage in the PR description). It still needs more unit tests (I used a local shell script to test things, I need to convert it to unit tests) and I need to add a line to the CHANGELOG. Other than that, I think this in a final state, so ready for reviews :)

@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from 5986f27 to 8873ba0 Jul 25, 2021
@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from 8873ba0 to cff89a6 Jul 27, 2021
@derrabus derrabus added this to the 5.4 milestone Jul 29, 2021
@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from cff89a6 to ca0b1db Jul 30, 2021
@OskarStark
Copy link
Contributor

@OskarStark OskarStark commented Aug 1, 2021

Should we target 6.0 and mark this big new feature as @experimental?

@Nyholm
Copy link
Member

@Nyholm Nyholm commented Aug 1, 2021

My general opinion is that we mark too many classes experimental.

But I let @wouterj decide in this specific case.

@wouterj
Copy link
Member Author

@wouterj wouterj commented Aug 2, 2021

The public API side of this feature is quite minimal (one method in Console which needs full BC and 2 classes that we should mark as @final). The feature itself is purely DX (i.e. it's not that big of a deal if it stops working), so I agree with not marking this as experimental.

Adding support for ZSH might break things, but a community member already stepped up to provide a PR for this if this is merged, so this can be caught before the 5.4 release. Other than that, all complexity of this feature is in the CompletionInput::bind() method, which will all be normal "bug fixes".


I guess the lack of reviewers in this PR indicates that it's probably a bit difficult to review. Some hints:

  • The main logic is in CompleteCommand and CompletionInput. The command receives an array of input "words" (similar to $argv) and the index in that array of the current cursor position (to know where completion is requested). Based on this, CompletionInput has to provide the correct type, value, option name, etc. That is the core logic of this PR (see also its tests).
  • From a DX side, see SecretsRemoveCommand::complete(), Command::complete() and Application::complete() for PHP usage of this feature.
  • If you're into bash, consider reviewing completion.bash and BashCompletionOutput. That latter is not doing much (only providing completion options as space-separated values). The first one is a bit more complex than the basic bash completion function, in order to simplify working with colons in the command names and = signs in between option name and value.

@dkarlovi
Copy link
Contributor

@dkarlovi dkarlovi commented Aug 3, 2021

@wouterj I'm looking for a simple way to test this in my actual app, IMO it would be most beneficial to figure it all out. I'll provide my findings if I find a way to do that.

Other than that, a cool sample might be to make a fork of Composer with this enabled. It might make it simple to visualize once you can test it for yourself easily.

@dkarlovi
Copy link
Contributor

@dkarlovi dkarlovi commented Aug 3, 2021

Also, this is an absurdly valuable feature, thank you a lot for working on this! 😍

@wouterj
Copy link
Member Author

@wouterj wouterj commented Aug 3, 2021

@dkarlovi if you have an app running Symfony 5.3, you can try to clone my fork locally and use the link utility:

git clone https://github.com/wouterj/symfony -b issue-38275/shell-autocomplete
cd symfony
./link /path/to/project

And then use the eval $(bin/console completion bash) in your app.

@dkarlovi

This comment has been hidden.

@dkarlovi
Copy link
Contributor

@dkarlovi dkarlovi commented Aug 5, 2021

@wouterj which Bash did you use to test this? For me, if I test with bin/console, it doesn't work, but ./console works, but need to change CWD obviously.

To get it to work, I needed to change it like this:

complete -F _sf_console bin/console

Also, the eval trick you gave didn't register the completion for me (checked with complete -p console), had to pipe it to a file and source it

bin/console completion bash > foo.sh
source foo.sh

@dkarlovi
Copy link
Contributor

@dkarlovi dkarlovi commented Aug 5, 2021

One thing which might be neat to have is to provide a flag for the dumper, something like

bin/console completion bash --verbose-completion

or similar, where it would create additional code in there to help you debug it if you're working on it. Might be too niche, maybe wait for use cases, but as I was testing this, I needed to edit the completion function to add them myself.

@wouterj
Copy link
Member Author

@wouterj wouterj commented Oct 11, 2021

Would Composer (or any other standalone tool) simply integrate the complete command to generate their own completion shell script and that's it?

Aside from whether this is the wanted behavior: this is currently possible (the Laravel demo in the PR description was made just by adding a complete() method to one Laravel command).

I guess from a package maintainer perspective, we would need to support a partially static completion script for this to be considered (e.g. generate a script based on the InputDefinition and only when things are unknown invoke the _complete command).

@dkarlovi
Copy link
Contributor

@dkarlovi dkarlovi commented Oct 11, 2021

@wouterj assuming

  1. Composer integrates this current work as-is
  2. this means the composer app provides the completion command, ie. composer completion bash will work
  3. the packager runs it, places the output script in the proper place

It should already work, no? The script will target the composer "binary", invoke the _complete command properly, etc. I don't see what special casing would actually be needed.

@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch 2 times, most recently from d27db4c to f071e5f Oct 12, 2021
Copy link
Contributor

@theofidry theofidry left a comment

A few minor comments/nitpicks. A big 👍 for this feature

src/Symfony/Component/Console/Completion/Completion.php Outdated Show resolved Hide resolved
src/Symfony/Component/Console/Completion/Completion.php Outdated Show resolved Hide resolved

$option = $this->getOptionFromToken($optionToken);
if (null === $option && !$this->isCursorFree()) {
$this->completionType = self::TYPE_OPTION_NAME;
Copy link
Contributor

@theofidry theofidry Oct 15, 2021

I tend to find this sort of API a bit brittle: it's easy to miss that all properties need to be set together. I am also not sure if this class is supposed to be extendable or not (it is not final at least, didn't check how hard/easy it would be to extend).

WDYT of adding a private accessor setCompletionValues(string $type, string $anme, string $value)?

Copy link
Member Author

@wouterj wouterj Oct 18, 2021

I do agree with you that this a bit brittle, but this is indeed not meant to be extended (I've made the class final now - thanks to your comment). The same comment as #42251 (comment) applies here imho.
Besides, we don't always set all 3 properties at the same time, sometimes only 1 or 2.


$relevantToken = $this->getRelevantToken();
if ('-' === $relevantToken[0]) {
// the current token is an input option: complete either option name or option value
Copy link
Contributor

@theofidry theofidry Oct 15, 2021

what about moving the different blocks into dedicated methods? e.g. addCompletionForInputOption()

Copy link
Member Author

@wouterj wouterj Oct 18, 2021

As with all parsing, it's really horrible code (lots of nested if statements) but not so easy to clean-up.

In this case, $relevantToken[0] === '-' can have multiple meanings: Either it's just - or --, indicating the next tokens are all argument values. Or it's the start of an option (short or long). And things get even more messy with things like -eprod, -vvv and -ev (indicating respectively an option with its value, an option without value, and 2 options).

I also don't think it's worth cleaning this code up. All "messiness" is contained in a single method, which will not be touched regularly (how we parse options, arguments and values is pretty static).

@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from f071e5f to 7810d19 Oct 18, 2021
@wouterj
Copy link
Member Author

@wouterj wouterj commented Oct 18, 2021

Thanks for the review @theofidry!

PR updated and rebased. We have a new failure (7.2) but I can't find any error logs whatsoever about this failure.

@wouterj wouterj force-pushed the issue-38275/shell-autocomplete branch from 7810d19 to e0a174f Oct 19, 2021
@wouterj
Copy link
Member Author

@wouterj wouterj commented Oct 19, 2021

Thanks, PR updated.

fabpot
fabpot approved these changes Oct 19, 2021
@fabpot
Copy link
Member

@fabpot fabpot commented Oct 19, 2021

Thank you @wouterj.

@fabpot fabpot merged commit cf8a997 into symfony:5.4 Oct 19, 2021
8 of 11 checks passed
symfony-splitter pushed a commit to symfony/framework-bundle that referenced this issue Oct 19, 2021
This PR was squashed before being merged into the 5.4 branch.

Discussion
----------

[Console] Bash completion integration

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Tickets       | Fix #38275
| License       | MIT
| Doc PR        | -

This is a first shot at implementing interactive bash completion support in the Console component. Besides completing option and command names, commands can override `Command::complete()` to implement completing values.

I've added an example code to the `secrets:remove` command, which now supports autocompletion quite nicely: ![render1630315116886](https://user-images.githubusercontent.com/749025/136708284-bf2e4c12-7cb7-4d5e-9c8d-68bcdca6fd7c.gif)

And support for other applications using `symfony/console` is automatically included (if the autocompletion script is installed for that specific application):
![render1630314403752](https://user-images.githubusercontent.com/749025/136708323-dfbccb77-dcbd-4d1e-8bb5-85b88f0b358b.gif)

This PR only implements Bash completion. Zsh and Fish have much more sophisticated completion systems, but I propose to keep those for a future PR if there is a need for this.

### How it works

1. A bash completion function (`_console`) is defined by `bin/console completion bash` for the `console` command (name of the "binary" file)
2. This completion function calls the local `bin/console _complete` command to come up with suggestions
3. Bash parses these suggestions and shows them to the user.

This has one drawback: the `_console` function is defined globally only once. This means we cannot easily change it, as it would break if you run different Symfony versions. We should probably add versioning (e.g. `bin/console _complete --version=1.0`) and don't suggest anything if the version doesn't match.

<s> **Maybe it would be safer to mark this feature as experimental and target 6.0, to allow us to fine tune the shell related sides over the lifespan of 6.x?** </s> symfony/symfony#42251 (comment)

### Steps to test yourself

Load this PR in your project, open a bash shell and run this command to "install" completion for this project:

```
bin/console completion bash > /etc/bash_completion.d/console
````

Then reload the bash shell and enjoy autocompletion.

### TODO

* [x] Autocompleting in the middle of the input doesn't work yet (i.e. `bin/console --en<TAB> cache:clear`)
* [x] Better error handling
* [x] Add a `bin/console completion` command to dump the `_console` file, so users can install this locally
* [x] Add some versioning, to allow us to change the `_console` file in the future
* [x] <s>See how we can better support standalone usage (e.g. Composer)</s> Tested on Laravel's artisan, works flawlessly

Commits
-------

e0a174f877 [FrameworkBundle] Add CLI completion to secrets:remove
82ef399de3 [Console] Bash completion integration
@wouterj wouterj deleted the issue-38275/shell-autocomplete branch Oct 19, 2021
if (!file_exists($debugFile)) {
touch($debugFile);
}
$process = new Process(['tail', '-f', $debugFile], null, null, null, 0);
Copy link
Member

@stof stof Oct 19, 2021

what happens on windows (and yes, bash can be used on windows, which does not imply that tail is available in the PATH AFAICT) ?

Copy link
Contributor

@dkarlovi dkarlovi Oct 20, 2021

This needs some more special casing. For example, you might not have symfony/process installed.

return $parseOptions;
}

private function getOptionValueFromToken(string $optionToken): string
Copy link
Contributor

@GromNaN GromNaN Oct 19, 2021

Is this function actually used?


private $tokens;
private $currentIndex;
private $indexToArgumentIndex = [];
Copy link
Contributor

@GromNaN GromNaN Oct 19, 2021

This property is not used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment