[Yaml] fix lexing nested sequences/mappings #33763

xabbuh · 2019-09-30T14:44:48Z

Q	A
Branch?	4.4
Bug fix?	yes
New feature?	no
Deprecations?	no
Tickets	#34805, #37788, #37876, #39011, #39013, #39064
License	MIT
Doc PR

src/Symfony/Component/Yaml/Parser.php

fabpot · 2020-08-11T10:19:58Z

@xabbuh Is it something you're still willing to work on?

simonberger

Nicely done.

I just missed a test with strings in mappings/sequences containing brackets which has been fixed in this PR as well.
As a possible test the following part of inlineNotationSpanningMultipleLinesProvider passed successfully.

            'nested collections containing strings with bracket chars' => [
                [['ba[r'], ['[ba]r'], ['bar]'], ['foo' => 'bar{'], ['foo' => 'b{ar}'], ['foo' => 'bar}']],
                <<<YAML
[
    [
        "ba[r"
    ],
    [
        '[ba]r'
    ],
    [
        "bar]"
    ],
    {
        foo: "bar{"
    },
    {
        foo: "b{ar}"
    },
    {
        foo: 'bar}'
    }
]

src/Symfony/Component/Yaml/Parser.php

xabbuh · 2020-11-15T17:15:09Z

Many thanks to @RevZer0, @simonberger and @Matth--. I have added the tests you added in #37876, #39013, and #39064 here.

xabbuh · 2020-11-15T18:46:18Z

This is ready to be reviewed.

src/Symfony/Component/Yaml/Parser.php

simonberger

Escaped quotes are not yet skipped in lexInlineQuotedString and are leading to problems for example in this yaml test:

[
    ["te\"st"],["test"]
]

Another simple error case is:

[
    ["te\"st]"]
]

simonberger · 2020-11-17T16:42:18Z

There is (now) another small issue with escaped backslashes in quotes treated as the escape for the quote.

'escaped backslash in mapping' => [
                [['"']],
                <<<YAML
[
    ["\\"]
]
YAML
            ]

I got a segmentation error writing the yam like this: [["\\"]]. l didn't look further in. Should be related to the problem but maybe a boundary check isn't robust enough.

xabbuh · 2020-11-17T16:46:32Z

Did you mean [["\\\\"]]? [["\\"]] isn't valid YAML and runs into a parsing error for me locally.

simonberger · 2020-11-17T17:00:27Z

Depends how it is declared. In HEREDOC [["\\"]]in quotes '[["\\\\"]]'.
There is also for me a parser error when using new lines, but there shouldn't.

Malformed inline YAML string: ""\"] ]]]" at line 3 (near "]")

xabbuh · 2020-11-17T18:08:36Z

@simonberger I don't see a difference here (see also https://3v4l.org/iOVjl). Could you please send a PR with a test case that you have in mind to my branch if you still think that something isn't working as expected?

simonberger · 2020-11-17T23:09:01Z

@xabbuh I am sorry. This was indeed my wrong test.

src/Symfony/Component/Yaml/Parser.php

simonberger

My pleasure. LGTM now.

fabpot · 2020-11-19T05:58:55Z

Thank you @xabbuh.

derrabus · 2020-11-21T00:44:49Z

@xabbuh Can I ask you to merge 4.4 into 5.1? This PR gives me a headache.

xabbuh · 2020-11-21T09:38:00Z

@derrabus done 👍

xabbuh added this to the next milestone Sep 30, 2019

carsonbot added Status: Needs Review Yaml Bug labels Sep 30, 2019

stof reviewed Sep 30, 2019

View changes

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

nicolas-grekas modified the milestones: next, 4.4 Oct 27, 2019

xabbuh mentioned this pull request Aug 18, 2020

[YAML] Unexpected characters using YAML compact syntax #37876

Closed

simonberger mentioned this pull request Nov 11, 2020

[Yaml] Fix parsing multidimensional inline collections #39013

Closed

xabbuh force-pushed the pr-33658 branch 6 times, most recently from 4279223 to 800a703 Compare Nov 14, 2020

simonberger reviewed Nov 14, 2020

View changes

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

xabbuh force-pushed the pr-33658 branch from 800a703 to c034cbb Compare Nov 15, 2020

xabbuh force-pushed the pr-33658 branch 3 times, most recently from 728cba9 to f21bd46 Compare Nov 15, 2020

xabbuh mentioned this pull request Nov 15, 2020

[Yaml] Fix sequence with following mapping #39064

Closed

This was linked to issues Nov 15, 2020

YAML parsing error on perfectly valid JSON #39011

Closed

Unexpected characters using YAML compact syntax #37788

Closed

jderusse reviewed Nov 15, 2020

View changes

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

simonberger approved these changes Nov 15, 2020

View changes

carsonbot added Status: Reviewed and removed Status: Needs Review labels Nov 15, 2020

simonberger suggested changes Nov 15, 2020

View changes

carsonbot added Status: Needs Work and removed Status: Reviewed labels Nov 15, 2020

xabbuh linked an issue Nov 16, 2020 that may be closed by this pull request

[YAML] Regression in parsing inline objects since version 4.4 #34805

Closed

xabbuh mentioned this pull request Nov 16, 2020

[YAML] Regression in parsing inline objects since version 4.4 #34805

Closed

xabbuh force-pushed the pr-33658 branch 2 times, most recently from d418d1d to 0e38c33 Compare Nov 17, 2020

simonberger reviewed Nov 17, 2020

View changes

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

simonberger reviewed Nov 17, 2020

View changes

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

fix lexing nested sequences/mappings

4c15f80

xabbuh force-pushed the pr-33658 branch from 0e38c33 to 4c15f80 Compare Nov 18, 2020

simonberger approved these changes Nov 18, 2020

View changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Nov 18, 2020

fabpot merged commit 308231a into symfony:4.4 Nov 19, 2020
4 of 5 checks passed

xabbuh deleted the pr-33658 branch Nov 19, 2020

This was referenced Nov 29, 2020

Release v4.4.17 #39215

Merged

Release v5.1.9 #39216

Merged

Release v5.2.0 #39224

Merged

Nyholm mentioned this pull request Nov 30, 2020

[Yaml] Unexpected character when whitespace ends the line #39234

Closed

cebe mentioned this pull request Dec 15, 2020

[Yaml] Regression 4.4.17 error loading YAML files with strings that have trailing whitespace #39521

Closed

[Yaml] fix lexing nested sequences/mappings #33763

[Yaml] fix lexing nested sequences/mappings #33763

xabbuh commented Sep 30, 2019 •

edited

fabpot commented Aug 11, 2020

simonberger left a comment

xabbuh commented Nov 15, 2020 •

edited

xabbuh commented Nov 15, 2020

simonberger left a comment •

edited

simonberger commented Nov 17, 2020 •

edited

xabbuh commented Nov 17, 2020 •

edited

simonberger commented Nov 17, 2020

xabbuh commented Nov 17, 2020

simonberger commented Nov 17, 2020

simonberger left a comment

fabpot commented Nov 19, 2020

derrabus commented Nov 21, 2020

xabbuh commented Nov 21, 2020

[Yaml] fix lexing nested sequences/mappings #33763

[Yaml] fix lexing nested sequences/mappings #33763

Conversation

xabbuh commented Sep 30, 2019 • edited

fabpot commented Aug 11, 2020

simonberger left a comment

xabbuh commented Nov 15, 2020 • edited

xabbuh commented Nov 15, 2020

simonberger left a comment • edited

simonberger commented Nov 17, 2020 • edited

xabbuh commented Nov 17, 2020 • edited

simonberger commented Nov 17, 2020

xabbuh commented Nov 17, 2020

simonberger commented Nov 17, 2020

simonberger left a comment

fabpot commented Nov 19, 2020

derrabus commented Nov 21, 2020

xabbuh commented Nov 21, 2020

xabbuh commented Sep 30, 2019 •

edited

xabbuh commented Nov 15, 2020 •

edited

simonberger left a comment •

edited

simonberger commented Nov 17, 2020 •

edited

xabbuh commented Nov 17, 2020 •

edited