Did you try the includes: option in your YAML header?
https://rmarkdown.rstudio.com/html_document_format.html#includes
But maybe you'll have the same problem I have: I'd like to include the HTML file in a specific section in my RMarkdown document, not in the header or before/after body.