Django logging configuration

I wanted to configure logging in my django project. I had the following requirements:

- All logs will be written to a file named app.log and the file will be rotated on each day
- All error logs will be written to a file named app.error.log and the file will be rotated on each day
- Logs will be written to console if DEBUG=True is set


After experimenting with different configurations, I finally was able to achieve my goal using the following configuration.

DJANGO_LOG_LEVEL = 'DEBUG' # need to change this value to enable/disable debug logs

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '%(levelname)s %(asctime)s %(module)s %(funcName)s:%(lineno)d %(message)s'        },
    },
    'filters': {
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'file': {
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': '/path/to/app.log',
            'formatter': 'simple',
            'when': 'midnight',
            'interval': 1,
            'level': 'DEBUG'        },
        'error_file': {
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': '/path/to/app.error.log',
            'formatter': 'simple',
            'when': 'midnight',
            'interval': 1,
            'level': 'ERROR',
        },
        'console': {
            'class': 'logging.StreamHandler',
            'filters': ['require_debug_true'],
            'formatter': 'simple'        }
    },
    'loggers': {
        'django': {
            'handlers': ['error_file', 'console'],
            'level': 'ERROR',
            'propagate': True,
        },
        'myapp': {
            'handlers': ['file', 'error_file', 'console'],
            'level': DJANGO_LOG_LEVEL,
            'propagate': True,
        },
    },
}

Let me explain what is going on here:

We've defined a filter require_debug_true. The filter, whenever applied, will make sure we've DEBUG=True set in Django settings

We've defined 3 handlers. The first two are file handlers. They'll write logs to specified files. 

They'll also write the logs if the level is above the defined level for that logger. 

For example, the file handler will write logs to app.log file if log levels are equal to or above DEBUG. 

the error_file handler will write logs to app.error.log file if log levels are equal to or above ERROR. 

Both of the file logger will rotate the log file each day at midnight.

Finally, we're defining our loggers. 

We've defined two loggers: django to handle the django logs and myapp to handle logs from myapp. 

The name myapp is just an example. You can give it any name you like. 

You only have to make sure you get the logger instance using this same 

name or a dotted extension e.g. myapp.mymodule

We've added all 3 handlers in our myapp logger. We also setting the level of our myapp logger 

to a variable we defined earlier. 

How it works? 

Suppose we're writing log using the following code.

import logging

logger = logging.getLogger(name='myapp') # myapp is the name of the logger

logger.debug('message: %s', message)
logger.info('message: %s', message)
logger.error('message: %s', message)


Case 1: We've set DEBUG=True and DJANGO_LOG_LEVEL=DEBUG


As DJANGO_LOG_LEVEL is DEBUG so myapp logger level is also debug. So log message will be 
received by myapp logger. 
myapp logger will send the log message to 3 handlers.
file handler has level=DEBUG so it'll write all 3 messages to app.log file
error_file handler has level=ERROR so it'll only write the error log to app.error.log
console has level=DEBUG and require_debug_true filter is applied. 
So all 3 messages will be written to console.




Case 2: We've set DEBUG=False and DJANGO_LOG_LEVEL=DEBUG
It'll be same as Case 1 except console handler will discard the logs and will write nothing 
because of the require_debug_true filter and DEBUG=False





Case 3: We've set DEBUG=True and DJANGO_LOG_LEVEL=INFO
Same as case 1 except the debug log message will not be processed file the file handler.


Case 4: We've set DEBUG=False and DJANGO_LOG_LEVEL=ERROR
In this case, myapp logger will not send the debug and info logs to file handler because 
this handler has level=DEBUG. It'll send the log to error_file handler. It'll also send the 
error log to console handler because the console handler has no level defined and we've set 
DEBUG=True




I'm sure this post will help me in future. If you are reading, I hope it'll help you too.


Happy coding!