ANTLR What is simpliest way to realize python like indent-depending grammar?

后端 未结 4 1000
自闭症患者
自闭症患者 2020-12-01 06:24

I am trying realize python like indent-depending grammar.

Source example:

ABC QWE
  CDE EFG
  EFG CDE
    ABC 
  QWE ZXC

As i see,

4条回答
  •  -上瘾入骨i
    2020-12-01 07:30

    Have you looked at the Python ANTLR grammar?

    Edit: Added psuedo Python code for creating INDENT/DEDENT tokens

    UNKNOWN_TOKEN = 0
    INDENT_TOKEN = 1
    DEDENT_TOKEN = 2
    
    # filestream has already been processed so that each character is a newline and
    # every tab outside of quotations is converted to 8 spaces.
    def GetIndentationTokens(filestream):
        # Stores (indentation_token, line, character_index)
        indentation_record = list()
        line = 0
        character_index = 0
        column = 0
        counting_whitespace = true
        indentations = list()
        for c in filestream:
            if IsNewLine(c):
                character_index = 0
                column = 0
                line += 1
                counting_whitespace = true
            elif c != ' ' and counting_whitespace:
                counting_whitespace = false
                if(len(indentations) == 0):
                    indentation_record.append((token, line, character_index))
                else:
                    while(len(indentations) > 0 and indentations[-1] != column:
                        if(column < indentations[-1]):
                            indentations.pop()
                            indentation_record.append((
                                DEDENT, line, character_index))
                        elif(column > indentations[-1]):
                            indentations.append(column)
                            indentation_record.append((
                                INDENT, line, character_index))
    
            if not IsNewLine(c):
                column += 1
    
            character_index += 1
        while(len(indentations) > 0):
            indentations.pop()
            indentation_record.append((DEDENT_TOKEN, line, character_index))
        return indentation_record
    

提交回复
热议问题