爱站程序员基地日志处理
1.日志的作用
- 1.记录程序运行状态1.线上环境所有程序以deamon形式运行在后台,无法使用Print输出程序状态
- 2.线上程序无人值守全天执行,需要有一种能持续记录程序运行状态的机制,以便遇到问题之后分析处理
2.基本用法
#!/usr/bin/env python3import loggingfrom logging.handlers import TimedRotatingFileHandler# 设置日志格式fmt = \'%(asctime)s %(levelname)7.7s %(funcName)s: %(message)s\'formatter = logging.Formatter(fmt,datefmt=\'%Y-%m-%d %H:%M:%S\')# 设置 handlerhandler = logging.handlers.TimedRotatingFileHandler(\'myapp.log\',when=\'D\',backupCount = 30)handler.setFormatter(formatter)# 定义 logger 对象logger = logging.getLogger(\'MyApp\')logger.addHandler(handler)logger.setLevel(logging.INFO)
# *******************在ipython中测试*********************In [1]: import loggingIn [2]: from logging.handlers import TimedRotatingFileHandlerIn [3]: from logging.handlers import TimedRotatingFileHandlerIn [4]: # 设置日志格式...: fmt = \'%(asctime)s %(levelname)7.7s %(funcName)s: %(message)s\'...: formatter = logging.Formatter(fmt,datefmt=\'%Y-%m-%d %H:%M:%S\')...:...: # 设置 handler...: handler = logging.handlers.TimedRotatingFileHandler(\'myapp.log\',when=\'D\'...: ,...: backupCount = 30)...: handler.setFormatter(formatter)...:...: # 定义 logger 对象...: logger = logging.getLogger(\'MyApp\')...: logger.addHandler(handler)...: logger.setLevel(logging.INFO)In [5]: logger.error(\'error\')
**以TimedRotatingFileHandler方式在每天系统都会在项目目录中创建一个新的日志文件(日志滚动)
3.日志的等级
- DEBUG:调试信息
- INFO:普通信息
- WARNING:警告
- ERROR:错误
- FATAL:致命错误
4.对应的函数
-
logger.debug(msg)
-
logger.info(msg)
-
logger.warning(msg)
-
logger.error(msg)
-
logger.fatal(msg)
5.日志格式允许字段
-
%(name)s
: Logger的名字
-
%(levelno)s
: 数字形式的日志级别
-
%(levelname)s
: 文本形式的日志级别
-
%(pathname)s
: 调用日志输出函数的模块的完整路径名,可能没有
-
%(filename)s
: 调用日志输出函数的模块的文件名
-
%(module)s
: 调用日志输出函数的模块名
-
%(funcName)s
: 调用日志输出函数的函数名
-
%(lineno)d
: 调用日志输出函数的语句所在代码行
-
%(created)f
: 当前时间,用UNIX标准的表示时间的浮点数表示
-
%(relativeCreated)d
: 输出日志信息自Logger创建以来的毫秒数
-
%(asctime)s
: 字符串形式的当前时间.默认格式是\”2003-07-08 16:49:45,896\”,(逗号后面是毫秒)
-
%(thread)d
: 线程ID,可能没有
-
%(threadName)s
: 线程名字,可能没有
-
%(process)d
: 进程ID,可能没有
-
%(message)s
: 用户输出的消息
6.Django中的日志配置
#!/usr/bin/env python3LOGGING = {\'version\':1,\'disable_existing_loggers\':True,# 格式配置\'formatters\':{\'simple\':{\'format\':\'%(asctime)s %(module)s.%(funcName)s:%(message)s\',\'datefmt\':\'%Y-%m-%d %H:%M:%S\',},\'verbose\':{\'format\':(\'%(asctime)s %(levelname)s [%(process)d-%(threadName)s]\'\'%(module)s.%(funcName)s line %(lineno)d:%(message)s\'),\'datefmt\':\'%Y-%m-%d %H:%M%S\',}},#Handler 配置\'handlers\':{\'console\':{\'class\':\'logging.StreamHandler\',\'level\':\'DEBUG\' if DEBUG else \'WARNING\'},\'info\':{\'class\':\'logging.handlers.TimedRotatingFileHandler\',\'filename\': \'{}/logs/info.log\'.format(BASE_DIR),# 日志保存路径\'when\':\'D\', #每天切割日志\'backupCount\':30,#日志保留30天\'formatter\':\'simple\',\'level\':\'INFO\',},\'error\':{\'class\':\'logging.handlers.TimedRotatingFileHandler\',\'filename\': \'{}/logs/error.log\'.format(BASE_DIR),\'when\':\'W0\',\'backupCount\':4,\'formatter\':\'verbose\',\'level\':\'WARNING\',}},# logger配置\'loggers\':{\'django\':{\'handlers\':[\'console\'],},\'inf\':{\'handlers\':[\'info\'],\'propagate\':True,\'level\':\'INFO\',},\'err\':{\'handlers\':[\'error\'],\'propagate\':True,\'level\':\'WARNING\',}}}
终端指令
-
tail -f myapp.log
: 持续跟踪myapp.log日志文件
-
tail -f logs/error.log|grep WARN
:只跟踪日志等级为WARNING的logs/error.log文件日志
-
grep \'16:54\' error.log | grep WARNING | awk \'{print $8}\':找到过滤之后的error.log日志（awk是一种语言,有print打印,$8代表第八列）
-
grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort找到过滤之后的error.log日志
-
grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort | uniq -
error.log | grep WARNING | awk \'{print $6}\' | sort | uniq -c -
grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort | uniq -c|sort -r
测试
ipython:In [1]: import loggingIn [2]: err = logging.getLogger(\'err\')In [3]: err.warning(\'1.1.1.1\')In [4]: err.warning(\'1.1.2.2\')In [5]: err.warning(\'4.4.2.2\')In [6]: err.warning(\'4.4.2.2\')In [7]: err.warning(\'4.4.2.3\')In [8]: err.warning(\'1.1.1.1\')In [9]: err.warning(\'1.1.1.1\')In [10]: err.warning(\'1.1.1.1\')另一个shellluciano@luciano:~/Desktops/fuck/logs$ grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\'1:4.4.2.21:4.4.2.31:1.1.1.11:1.1.1.11:1.1.1.1luciano@luciano:~/Desktops/fuck/logs$ grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort1:1.1.1.11:1.1.1.11:1.1.1.11:4.4.2.21:4.4.2.3luciano@luciano:~/Desktops/fuck/logs$ grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort | uniq1:1.1.1.11:4.4.2.21:4.4.2.3luciano@luciano:~/Desktops/fuck/logs$ grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort | uniq -c3 1:1.1.1.11 1:4.4.2.21 1:4.4.2.3luciano@luciano:~/Desktops/fuck/logs$ grep \'18:24\' error.log | grep WARNING | awk \'{print $6}\' | sort | uniq -c|sort -r3 1:1.1.1.11 1:4.4.2.31 1:4.4.2.2
