Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MemoryHandler can't be setup through a DictConfig if any other logger is setup after it #123239

Open
TheSylex opened this issue Aug 22, 2024 · 1 comment
Open

MemoryHandler can't be setup through a DictConfig if any other logger is setup after it #123239

TheSylex opened this issue Aug 22, 2024 · 1 comment
Labels
pending The issue will be closed if no feedback is provided type-bug An unexpected behavior, bug, or error

Comments

@TheSylex
Copy link

TheSylex commented Aug 22, 2024 •
edited

Bug report

Bug description:

This is the logging that's failing.

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        "fine-logger1": {
            'level': logging.INFO,
            'formatter': 'verbose-console',
            'class': 'logging.StreamHandler',
            'stream': 'ext://sys.stdout',
        },
        "fine-logger2": {
            "level": logging.INFO,
            "class": "logging.FileHandler",
            "filename": Path("logs/somefilename.log"),
            "mode": "a",
            "formatter": "verbose",
        },
        "fine-logger3": {
            "level": logging.DEBUG,
            "class": "logging.FileHandler",
            "filename": Path("logs/errors/somefilename.log"),
            "delay": True,
            "formatter": "verbose",
        },
        "bugged-logger3": {
            "level": logging.DEBUG,
            "class": "logging.handlers.MemoryHandler",
            "capacity": 9999999,
            "flushLevel": logging.ERROR,
            "target": "fine-logger3",
            "flushOnClose": False,
        },
    },
    "loggers": {
        "bug-logger": {
            "handlers": ["fine-logger1", "fine-logger2", "bugged-logger3"],
            "level": logging.DEBUG,
            "propagate": True
        },
    },
}

After it's been setup, if any other handler is setup afterwards (in my case it's the copernicusmarine logger) the handler bugged-logger3 will have a None inside its target field, as every handler that's not incremental will delete existing unused handlers.

I've dug into the source code of logging and I think I've tracked down where this is happpening, but I'm not really sure on how to fix it.
In the config file python3.11/logging/config.py in the configure() function there's a call to _clearExistingHandlers():

            else:
                disable_existing = config.pop('disable_existing_loggers', True)

                _clearExistingHandlers() <-----

                # Do formatters first - they don't refer to anything else
                formatters = config.get('formatters', EMPTY_DICT)
                for name in formatters:
                    try:
                        formatters[name] = self.configure_formatter(
                                                            formatters[name])

If we comment this line, it works completely fine. But I guess this just keeps handlers loaded in memory and that's not a proper solution.

CPython versions tested on:

3.11

Operating systems tested on:

Linux
@TheSylex TheSylex added the type-bug An unexpected behavior, bug, or error label Aug 22, 2024
@picnixz
Copy link
Contributor

picnixz commented Aug 23, 2024

Can you test with 3.12+? 3.11 only receives security fixes so we wouldn't correc this bug in 3.11 if there is one.

@picnixz picnixz added the pending The issue will be closed if no feedback is provided label Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending The issue will be closed if no feedback is provided type-bug An unexpected behavior, bug, or error
Projects
Logging issues 🪵
Status: No status
Milestone
No milestone
Development

No branches or pull requests
2 participants
@picnixz @TheSylex