Skip to content

More generic source location parsing + more errors #11913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

cameel
Copy link
Member

@cameel cameel commented Sep 8, 2021

Depends on #11915. Merged.

This is the refactor requested in #11867 (comment) and a bunch of other small tweaks that apply generally to @src tags even without code snippets.

  • Parser tests were not checking for errors reported during parsing.
  • A few more tests for corner cases related to whitespace.
  • More general parsing. Tags are found first and then for each tag parameters are parsed separately.
    • This allows us to report an error in cases where the invalid tags were just being ignored until now. Note that this part is not strictly a refactor since it changes error behavior.

@cameel cameel self-assigned this Sep 8, 2021
@cameel cameel requested a review from chriseth September 8, 2021 19:53
solAssert(sourceName, "");
m_debugDataOverride = DebugData::create(SourceLocation{*start, *end, move(sourceName)});
smatch srcTagArgsMatch;
if (!regex_search(position, commentLiteral.end(), srcTagArgsMatch, srcTagArgsRegex))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing stopping us from using Scanner for parsing tag args is that there's no way to check where the scanner has stopped and use that to update position.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think I merged this too early: Shouldn't this be regex_match instead? Otherwise it would match @src something 1:1:1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But maybe the ^ already does it...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, both would work (regex_match() with .* added at the end). It starts with ^ exactly for this purpose.

Comment on lines +520 to +522
BOOST_REQUIRE(errorList.size() == 1);
BOOST_TEST(errorList[0]->type() == Error::Type::SyntaxError);
BOOST_TEST(errorList[0]->errorId() == 8387_error);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// @src 0:420:680foo used to result in the tag being ignored. Now it's a syntax error.

Comment on lines +47 to +48
BOOST_TEST_DONT_PRINT_LOG_VALUE(ErrorId)
BOOST_TEST_DONT_PRINT_LOG_VALUE(Error::Type)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have << for these defined. I can either define it or tell boost not to try to print it. For now I did the latter.

@cameel cameel force-pushed the source-location-parsing-extra-tests-and-regex-refactor branch from a9d56a3 to dc44dc7 Compare September 8, 2021 20:25
@cameel cameel force-pushed the source-location-parsing-extra-tests-and-regex-refactor branch from dc44dc7 to eeb1946 Compare September 9, 2021 12:52
@cameel cameel changed the base branch from develop to fix-out-of-sync-gas-costs September 9, 2021 12:52
@cameel
Copy link
Member Author

cameel commented Sep 9, 2021

Rebased on top of #11915.

Base automatically changed from fix-out-of-sync-gas-costs to develop September 9, 2021 13:42
@cameel cameel force-pushed the source-location-parsing-extra-tests-and-regex-refactor branch from eeb1946 to 14396c2 Compare September 9, 2021 15:13
Comment on lines 133 to 136
static regex const lineRE = std::regex(
R"~~~((^|\s*)@src\s+(-1|\d+):(-1|\d+):(-1|\d+)(\s+|$))~~~",
std::regex_constants::ECMAScript | std::regex_constants::optimize
static regex const tagRegex = regex(
R"~~(\s*(@[a-zA-Z0-9\-_]+)(?:\s+|$))~~", // tag, e.g: @src
regex_constants::ECMAScript | regex_constants::optimize
);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added one more change to the regex here. \s+ after the tag was replaced with (?:\s+|$). This way in the example below @tag2 will not be ignored:

/// @src 1:3:4 some text, more text @tag1 @tag2

It does not matter all that match currently because there are no tags without args but it might become an unpleasant surprise if someone wanted to add them at some point.

@chriseth chriseth merged commit f957820 into develop Sep 13, 2021
@chriseth chriseth deleted the source-location-parsing-extra-tests-and-regex-refactor branch September 13, 2021 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants