I saw in the html_document help page an easy-way to customize the anchor style by adapting the anchor-section class. This works well for individual Rmd document
anchor-section