Monday, November 26, 2018

python - Matching multiple regex groups and removing them




I have been given a file that I would like to extract the useful data from. The format of the file goes something like this:



LINE: 1
TOKENKIND: somedata
TOKENKIND: somedata
LINE: 2
TOKENKIND: somedata
LINE: 3



etc...



What I would like to do is remove LINE: and the line number as well as TOKENKIND: so I am just left with a string that consists of 'somedata somedate somedata...'



I'm using Python to do this, using regular expressions (that I'm not sure are correct) to match the bits of the file I'd like removing.



My question is, how can I get Python to match multiple regex groups and ignore them, adding anything that isn't matched by my regex to my output string? My current code looks like this:



import re
import sys


ignoredTokens = re.compile('''
(?P \s+ ) |
(?P LINE:\s[0-9]+ ) |
(?P [A-Z]+: )
''', re.VERBOSE)

tokenList = open(sys.argv[1], 'r').read()
cleanedList = ''


scanner = ignoredTokens.scanner(tokenList)

for line in tokenList:
match = scanner.match()

if match.lastgroup not in ('WHITESPACE', 'LINE', 'TOKEN'):
cleanedList = cleanedList + match.group(match.lastindex) + ' '

print cleanedList


Answer



import re

x = '''LINE: 1
TOKENKIND: somedata
TOKENKIND: somedata
LINE: 2
TOKENKIND: somedata
LINE: 3'''


junkre = re.compile(r'(\s*LINE:\s*\d*\s*)|(\s*TOKENKIND:)', re.DOTALL)

print junkre.sub('', x)

No comments:

Post a Comment

plot explanation - Why did Peaches' mom hang on the tree? - Movies & TV

In the middle of the movie Ice Age: Continental Drift Peaches' mom asked Peaches to go to sleep. Then, she hung on the tree. This parti...