Skip to content

[HttpKernel] Strip exception file paths from log messages #46828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 1, 2023

Conversation

ostrolucky
Copy link
Contributor

Q A
Branch? 6.2
Bug fix? no
New feature? no
Deprecations? no
Tickets
License MIT
Doc PR

In log messages, I would like to strip full file path to a file where error occurred. Instead, I suggest to use name of the file only. There are 2 reasons:

  1. Whole $exception is supplied to logger anyways, so filepath is not lost from logs. I am only removing it from log message, not log entry. There isn't really a need to have filepath both in log message and log context.
  2. We, similarly as lot of people use symlinks for deployments. This means real path after every release changes (/app/releases/385/ -> /app/releases/386/ and so on). Since Exception::getFile() contains real path and not symlink path (/app/current/), every release log messages for exact same errors change. This causes error deduplication of 3rd parties like NewRelic and other APM solutions to not trigger, which means we are getting alerts for same old errors after every release.

Copy link
Contributor

@ro0NL ro0NL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe only strip the project dir ...

@ostrolucky
Copy link
Contributor Author

ostrolucky commented Jul 4, 2022

That's a bit more complicated solution, but sure we could that too if you think it's better. Any other opinions?

edit: Also I think that's a bit more error prone too, because I don't think there is a guarantee kernel.project_dir will always resolve to a real path and not a symlink.

@alexislefebvre
Copy link
Contributor

Could you please provide an example before and after the file path has been cleaned?

What if users have classes with the same name (src/GitHub/Client.php and src/GitLab/Client.php) and an error is reported in Client.php? Will they have to dig more in the logs to see the path of the class?

This causes error deduplication of 3rd parties like NewRelic and other APM solutions to not trigger, which means we are getting alerts for same old errors after every release.

We use Sentry and it doesn't have this issue, it looks like it groups errors by the relative path, e.g. src/Form/Upload/Uploader.php. We can still see the real path of the file with the numbered release /releases/20220630…/ in the stacktrace.

Maybe this can be configured on the side of NewRelic?

@ostrolucky
Copy link
Contributor Author

ostrolucky commented Jul 7, 2022

Could you please provide an example before and after the file path has been cleaned?

Before: Uncaught PHP Exception Exception: "foo" at /home/application/deployer/releases/123/src/ErrorListenerTest.php line 123
After: Uncaught PHP Exception Exception: "foo" at ErrorListenerTest.php line 123

What if users have classes with the same name (src/GitHub/Client.php and src/GitLab/Client.php) and an error is reported in Client.php?

This can happen, but is very unlikely, especially at the same line

Will they have to dig more in the logs to see the path of the class?

What does this mean? Same log entry that's having that message also has full context and trace. No log digging is necessary, you just expand current log.

And for that matter, I don't see why is symfony even putting file and line in log message at all. Other logs that don't go through this listener don't have this info and I haven't seen anyone complaining. If people think this is good idea in general, monolog should do this itself via processor.

Maybe this can be configured on the side of NewRelic?

No, their support told me this

The grouping indeed is going to be taking the error message into consideration and when the message has a changing attribute (like your release number) the error groups might appear as a completely different error.

This behaviour is however expected from our product and we are currently not in a position to offer a pattern detection such that we could replace your deployment ID with a placeholder for all the similar errors.

In relation to your suggestion on using the symlink path instead of the physical path I believe this isn't something that we can control in this particular use case. The path appears to be a part of the message, that is passed to our 'newrelic_notice_error()' function using the ekino bundle. The message, I believe, is constructed by PHP and Symfony so you would have to adjust the path resolution there in order for the message string to include symlink path. This in turn would cause our error recording function to receive same message allowing the Error Inbox to correctly group them together.

We use Sentry and it doesn't have this issue

You probably use Sentry bundle which uses error listener. In our case, we hook into Monolog handler and send all error logs to NR, not only errors caught by error listener.

@nicolas-grekas nicolas-grekas modified the milestones: 6.2, 6.3 Nov 5, 2022
@nicolas-grekas nicolas-grekas modified the milestones: 6.3, 6.4 May 23, 2023
@ostrolucky
Copy link
Contributor Author

Any update? Or shall we simply stop adding file path and line in the log message?

@nicolas-grekas
Copy link
Member

Works for me, but please keep the CS: we prefer one liners (they help with grep)

@ostrolucky ostrolucky force-pushed the strip-exception-file-paths branch from 6a0b0c9 to 272c13e Compare August 1, 2023 15:38
@ostrolucky
Copy link
Contributor Author

CS adjusted, rebased

@fabpot
Copy link
Member

fabpot commented Aug 1, 2023

Thank you @ostrolucky.

@fabpot fabpot merged commit db1d6eb into symfony:6.4 Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants