This class wraps the functionality in the logging module to provide an easier to use API for logging while providing advanced features including a independent namespace. Each application or library wishing to have it's own logging namespace should instantiate exactly one instance of this class and use it to manage all it's logging.

Traditionally (or simplistically) logging was set up with a single global root logger with output handlers bound to it. The global root logger (whose name is the empty string) was shared by all code in a loaded process. The only the global unamed root logger had a level set on it, all other loggers created inherited this global level. This can cause conflicts in more complex scenarios where loaded code wants to maintain it's own logging configuration independent of whomever loaded it's code. By using only a single logger level set on the global root logger it was not possible to have fine grained control over individual logger output. The pattern seen with this simplistic setup has been frequently copied despite being clumsy and awkward. The logging module has the tools available to support a more sophisitcated and useful model, but it requires an overarching framework to manage. This class provides such a framework.

An independent namespace is established by creating a independent root logger for this manager (root_logger_name). This root logger is a direct child of the global unamed root logger. All loggers created by this manager will be descendants of this managers root logger. The managers root logger has it's propagate flag set to False which means all loggers and handlers created by this manager will be isolated in the global logging tree.

Log level management:

Traditionally loggers inherited their logging level from the root logger. This was simple but made it impossible to independently control logging output from different loggers. If you set the root level to DEBUG you got DEBUG output from every logger in the system, often overwhelming in it's voluminous output. Many times you want to turn on debug for just one class (a common idom is to have one logger per class). To achieve the fine grained control you can either use filters or set a logging level on every logger (see the module documentation for the pros and cons). This manager sets a log level on every logger instead of using level inheritence because it's more efficient at run time.

Global levels are supported via the verbose and debug flags setting every logger level to INFO and DEBUG respectively. Fine grained level control is provided via regular expression matching on logger names (see configure() for the details. For example if you want to set a debug level for the foo.bar logger set a regular expression to match it and bind it to the debug level. Note, the global verbose and debug flags always override the regular expression level configuration. Do not set these global flags if you want fine grained control.

The manager maintains the minimum level for all loggers under it's control and the minimum level for all handlers under it's control. The reason it does this is because there is no point in generating debug messages on a logger if there is no handler defined which will output a debug message. Thus when the level is set on a logger it takes into consideration the set of handlers that logger can emit to.

IMPORTANT: Because the manager maintains knowledge about all the loggers and handlers under it's control it is essential you use only the managers interface to modify a logger or handler and not set levels on the objects directly, otherwise the manger will not know to visit every object under it's control when a configuraiton changes (see 'LogManager.apply_configuration()).

Example Usage:

# Create a log managers for use by 'my_app'
log_mgr = LogManager('my_app')

 # Create a handler to send error messages to stderr
log_mgr.create_log_handlers([dict(stream=sys.stdout,
                                  level=logging.ERROR)])

 # Create logger for a class
class Foo(object):
    def __init__(self):
        self.log = log_mgr.get_logger(self)

configure(self, config, configure_state=None)

source code

The log manager is initialized from key,value pairs in the config dict. This may be called any time to modify the logging configuration at run time.

The supported entries in the config dict are:

default_level

The default level applied to a logger when not indivdually configured. The verbose and debug config items override the default level. See log_manager.parse_log_level() for details on how the level can be specified.

verbose

Boolean, if True sets default_level to INFO.

debug

Boolean, if True sets default_level to DEBUG.

logger_regexps

List of (regexp, level) tuples. This is a an ordered list regular expressions used to match against a logger name to configure the logger's level. The first regexp in the sequence which matches the logger name will use the the level bound to that regexp to set the logger's level. If no regexp matches the logger name then the logger will be assigned the default_level.

The regular expression comparision is performed with the re.search() function which means the match can be located anywhere in the name string (as opposed to the start of the the string). Do not forget to escape regular expression metacharacters when appropriate. For example dot ('.') is used to seperate loggers in a logging hierarchy path (e.g. a.b.c)

Examples:

# To match exactly the logger a.b.c and set it to DEBUG:
    logger_regexps = [(r'^a\.b\.c$', 'debug')]

# To match any child of a.b and set it to INFO:
    logger_regexps = [(r'^a\.b\..*', 'info')]

# To match any leaf logger with the name c and set it to level 5:
    logger_regexps = [(r'\.c$', 5)]

handlers

List of handler config dicts or (config, logger) tuples. See create_log_handlers() for details of a hanlder config.

The simple form where handlers is a list of dicts each handler is bound to the log mangers root logger (see create_log_handlers() optional logger parameter). If you want to bind each handler to a specific logger other then root handler then group the handler config with a logger in a (config, logger) tuple. The logger may be either a logger name or a logger instance. The following are all valid methods of passing handler configuration.:

# List of 2 config dicts; both handlers bound to root logger
[{}, {}]

# List of 2 tuples; first handler bound to logger_name1
# by name, second bound to logger2 by object.
[({}, 'logger_name1'), ({}, logger2']

# List of 1 dict, 1 tuple; first bound to root logger,
# second bound to logger_name by name
[{}, ({}, 'logger_name']

Parameters:

config - Dict of <key,value> pairs describing the configuration.
configure_state - If other than None update the log manger's configure_state variable to this object. Clients of the log manager can use configure_state to track the state of the log manager.

create_log_handlers(self, configs, logger=None, configure_state=None)

source code

Create new handlers and attach them to a logger (log mangers root logger by default).

Note, you may also pass the handler configs to `LogManager.configure()`.

configs is an iterable yielding a dict. Each dict configures a handler. Currently two types of handlers are supported:

stream
file

Which type of handler is created is determined by the presence of the stream or filename in the dict.

Configuration keys:

Handler type keys:

Exactly of the following must present in the config dict:

stream: Use the specified stream to initialize the StreamHandler.
filename: Specifies that a FileHandler be created, using the specified filename.

Common keys:

name

Set the name of the handler. This is optional but can be useful when examining the logging configuration. For files defaults to 'file:absolute_path' and for streams it defaults to 'stream:stream_name'

format

Use the specified format string for the handler.

time_zone_converter

Log record timestamps are seconds since the epoch in the UTC time zone stored as floating point values. When the formatter inserts a timestamp via the %(asctime)s format substitution it calls a time zone converter on the timestamp which returns a time.struct_time value to pass to the time.strftime function along with the datefmt format conversion string. The time module provides two functions with this signature, time.localtime and time.gmtime which performs a conversion to local time and UTC respectively. time.localtime is the default converter. Setting the time zone converter to time.gmtime is appropriate for date/time strings in UTC. The time_zone_converter attribute may be any function with the correct signature. Or as a convenience you may also pass a string which will select either the time.localtime or the time.gmtime converter. The case insenstive string mappings are:

'local'     => time.localtime
'localtime' => time.localtime
'gmt'       => time.gmtime
'gmtime'    => time.gmtime
'utc'       => time.gmtime

datefmt

Use the specified time.strftime date/time format when formatting a timestamp via the %(asctime)s format substitution. The timestamp is first converted using the time_zone_converter to either local or UTC

level

Set the handler logger level to the specified level. May be one of the following strings: 'debug', 'info', 'warn', 'warning', 'error', 'critical' or any of the logging level constants. Thus level='debug' is equivalent to level=logging.DEBUG. Defaults to self.default_level.

File handler keys:

filemode: Specifies the mode to open the file. Defaults to 'a' for append, use 'w' for write.
permission: Set the permission bits on the file (i.e. chmod). Must be a valid integer (e.g. 0660 for rw-rw----)
user: Set the user owning the file. May be either a numeric uid or a basestring with a user name in the passwd file.
group: Set the group associated with the file, May be either a numeric gid or a basestring with a group name in the groups file.

Examples:

The following shows how to set two handlers, one for a file (ipa.log) at the debug log level and a second handler set to stdout (e.g. console) at the info log level. (One handler sets it level with a simple name, the other with a logging constant just to illustrate the flexibility)

# Get a root logger
log_mgr = LogManger('my_app')

# Create the handlers
log_mgr.create_log_handlers([dict(filename='my_app.log',
                                  level='info',
                                  user='root',
                                  group='root',
                                  permission=0600,
                                  time_zone_converter='utc',
                                  datefmt='%Y-%m-%dT%H:%M:%SZ', # ISO 8601
                                  format='<%(levelname)s> [%(asctime)s] module=%(name)s "%(message)s"'),
                             dict(stream=sys.stdout,
                                  level=logging.ERROR,
                                  format='%(levelname)s: %(message)s')])

# Create a logger for my_app.foo.bar
foo_bar_log = log_mgr.get_logger('foo.bar')

root_logger.info("Ready to process requests")
foo_bar_log.error("something went boom")

In the file my_app.log you would see:

<INFO> [2011-10-26T01:39:00Z] module=my_app "Ready to process requests"
<ERROR> [2011-10-26T01:39:00Z] module=may_app.foo.bar "something went boom"

On the console you would see:

ERROR: something went boom

Parameters:

configs - Sequence of dicts (any iterable yielding a dict). Each dict creates one handler and contains the configuration parameters used to create that handler.
logger - If unspecified the handlers will be attached to the LogManager.root_logger, otherwise the handlers will be attached to the specified logger.
configure_state - If other than None update the log manger's configure_state variable to this object. Clients of the log manager can use configure_state to track the state of the log manager.

Returns:

The list of created handers.

get_logger(self, who, bind_logger_names=False)

source code

Return the logger for an object or a name. If the logger already exists return the existing instance otherwise create the logger.

The who parameter may be either a name or an object. Loggers are identified by a name but because loggers are usually bound to a class this method is optimized to handle that case. If who is an object:

The name object's module name (dot seperated) and the object's class name.
Optionally the logging output methods can be bound to the object if bind_logger_names is True.

Otherwise if who is a basestring it is used as the logger name.

In all instances the root_logger_name is prefixed to every logger created by the manager.

Parameters:

who - If a basestring then use this as the logger name, prefixed with the root_logger_name. Otherwise who is treated as a class instance. The logger name is formed by prepending the root_logger_name to the module name and then appending the class name. All name components are dot seperated. Thus if the root_logger_name is 'my_app', the class is ParseFileConfig living in the config.parsers module the logger name will be: my_app.config.parsers.ParseFileConfig.
bind_logger_names - If true the class instance will have the following bound to it: log, debug(), info(), warning(), error(), exception(), critical(). Where log is the logger object and the others are the loggers output methods. This is a convenience which allows you emit logging messages directly, for example:
```
self.debug('%d names defined', self.num_names).
```

Returns:

The logger matching the name indicated by who. If the logger pre-existed return that instance otherwise create the named logger return it.

Class LogManager

Log level management:

init(self, root_logger_name=`''`, configure_state=None)
(Constructor)

str(self)
(Informal representation operator)

configure(self, config, configure_state=None)

create_log_handlers(self, configs, logger=None, configure_state=None)

Configuration keys:

Handler type keys:

Common keys:

File handler keys:

Examples:

get_handler(self, handler_name)

set_handler_level(self, handler_name, level, configure_state=None)

get_loggers_with_handler(self, handler)

remove_handler(self, handler, logger=None, configure_state=None)

apply_configuration(self, configure_state=None)

get_configured_logger_level(self, name)

get_logger_handlers(self, logger)

get_minimum_handler_level_for_logger(self, logger)

_set_configured_logger_level(self, logger)

get_logger(self, who, bind_logger_names=False)

set_default_level(self, level, configure_state=None)

default_level

Class LogManager

Log level management:

__init__(self, root_logger_name='', configure_state=None) (Constructor)

__str__(self) (Informal representation operator)

configure(self, config, configure_state=None)

create_log_handlers(self, configs, logger=None, configure_state=None)

Configuration keys:

Handler type keys:

Common keys:

File handler keys:

Examples:

get_handler(self, handler_name)

set_handler_level(self, handler_name, level, configure_state=None)

get_loggers_with_handler(self, handler)

remove_handler(self, handler, logger=None, configure_state=None)

apply_configuration(self, configure_state=None)

get_configured_logger_level(self, name)

get_logger_handlers(self, logger)

get_minimum_handler_level_for_logger(self, logger)

_set_configured_logger_level(self, logger)

get_logger(self, who, bind_logger_names=False)

set_default_level(self, level, configure_state=None)

default_level

init(self, root_logger_name=`''`, configure_state=None)
(Constructor)

str(self)
(Informal representation operator)