How to find all traceback from log file by using python regex?

问题

There is a log file from jenkins. and I want to extract all the traceback from the log file. (many different kinds of tracebacks may be appear in the log). How to write the python regex? I have tried below method, but it works not well.

trace_regex = r'^Traceback[\s\S]*Error.*|^Traceback[\s\S]*timeout.*|^Traceback[\s\S]*HunterDeviceOffline.*|Traceback[\s\S]*Exception.*|Traceback[\s\S]*BadStatusLine.*|Traceback[\s\S]*NoSuchDevice.*|Traceback[\s\S]*AuthenticationsFail.*'
trace = re.findall(trace_regex, log, re.M | re.I)

the log file content is like blow:

Started by upstream project "run_script" build number 435
originally caused by:
 Started by user test
[Pipeline] {
[Pipeline] stage (Svnup)
Entering stage Svnup
+ python -m test.cli install 479CI9SOZ android://0.0.0.0:10000/479CI9SOZ /root/workspace/group/apk/com.data.app.apk - - false false
[03:21:33][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ wait-for-device
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/cli/parser.py", line 55, in main
    run_script(args)
  File "/root/env/common/test/test/core/helper.py", line 224, in wrapper
    return f(pictarget, *args[1:], **opargs)
  File "/root/env/common/test/test/core/main.py", line 408, in assert_exists
    raise AssertionTimeout("%s does not exist in screen" % v)
AssertionTimeout: MoaPic(group_login.owl/tpl1487844291185.png) does not exist in screen
EndOfStream
[03:23:18][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ forward --remove tcp:12671
+ python -m test.cli run 479CI9SOZ android://0.0.0.0:10000/479CI9SOZ /root/workspace/group/util/scripts/group_login.owl 
rpc_uri: http://0.0.0.0:12708/jsonrpc/0
rpc_uri: http://0.0.0.0:12708/jsonrpc/0
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/__main__.py", line 5, in <module>
    main()
  File "/root/env/common/test/test/cli/parser.py", line 55, in main
    run_script(args)
  File "/root/env/common/test/test/cli/runner.py", line 124, in run_script
    exec_script(args.script, scope=globals(), root=True)
  File "/root/env/common/test/test/cli/runner.py", line 186, in exec_script
    exec(compile(code, scriptpath, 'exec')) in scope
  File "/root/env/common/mator/mator/mator.py", line 520, in start
    raise IOError("RPC server not started!")
IOError: RPC server not started!
[03:24:40][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ forward --remove tcp:12708
[03:26:41][DEBUG]<main>  ->sift result: None
[03:26:41][DEBUG]<main> match result: None
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/core/main.py", line 195, in touch
    pos = loop_find(v, timeout=timeout)
  File "/root/env/common/test/test/core/utils/logwraper.py", line 70, in wrapper
    res = f(*args, **kwargs)
  File "/root/env/common/test/test/core/cv.py", line 83, in loop_find
    raise NotFoundException('Picture %s not found in screen' % query)
test.core.error.NotFoundException: 'Picture MoaPic(group_login.owl/tpl1482311722698.png) not found in screen'
+ python -m test.cli stop_screen_record android://0.0.0.0:10000/479CI9SOZ 479CI9SOZ.mp4
ERROR: script returned exit code 1
Finished: FAILURE

any idea? thanks everyone

回答1:

You could possibly get along with

^Traceback  # look for Traceback right at the start of a line
[\s\S]+?    # everything lazily
(?=^\[|\Z)  # pos. lookahead, either [ at the start of a line...
            # or the very end of the string

In MULTILINE and VERBOSE mode, see a demo on regex101.com.

In Python, this would be:

import re

rx = re.compile(r'''
    ^Traceback
    [\s\S]+?
    (?=^\[|\Z)
    ''', re.M | re.X)

for match in rx.finditer(your_string_here):
    print match.group(0)

Alternatively, use the DOTALL modifier as well (instead of the [\s\S] construct, that is):

rx = re.compile(r'''
    ^Traceback
    .+?
    (?=^\[|\Z)
    ''', re.M | re.X | re.S)

来源：https://stackoverflow.com/questions/45852700/how-to-find-all-traceback-from-log-file-by-using-python-regex

标签

python

regex

traceback