Catching (and keeping) all comments with ANTLR

前端 未结 6 1155
抹茶落季
抹茶落季 2020-12-10 19:49

I\'m writing a grammar in ANTLR that parses Java source files into ASTs for later analysis. Unlike other parsers (like JavaDoc) I\'m trying to keep all of the comments. This

6条回答
  •  小蘑菇
    小蘑菇 (楼主)
    2020-12-10 20:09

    first: direct all comments to a certain channel (only comments)

    COMMENT
        : '/*' .*? '*/' -> channel(2)
        ;
    
    LINE_COMMENT
        : '//' ~[\r\n]* -> channel(2)
        ;
    

    second: print out all comments

          CommonTokenStream tokens = new CommonTokenStream(lexer);
          tokens.fill();
          for (int index = 0; index < tokens.size(); index++)
          {
             Token token = tokens.get(index);
             // substitute whatever parser you have
             if (token.getType() != Parser.WS) 
             {
                String out = "";
                // Comments will be printed as channel 2 (configured in .g4 grammar file)
                out += "Channel: " + token.getChannel();
                out += " Type: " + token.getType();
                out += " Hidden: ";
                List hiddenTokensToLeft = tokens.getHiddenTokensToLeft(index);
                for (int i = 0; hiddenTokensToLeft != null && i < hiddenTokensToLeft.size(); i++)
                {
                   if (hiddenTokensToLeft.get(i).getType() != IDLParser.WS)
                   {
                      out += "\n\t" + i + ":";
                      out += "\n\tChannel: " + hiddenTokensToLeft.get(i).getChannel() + "  Type: " + hiddenTokensToLeft.get(i).getType();
                      out += hiddenTokensToLeft.get(i).getText().replaceAll("\\s", "");
                   }
                }
                out += token.getText().replaceAll("\\s", "");
                System.out.println(out);
             }
          }
    

提交回复
热议问题