Parse SGML with Open Arbitrary Tags in Python 3

Question

I am trying to parse a file such as: http://www.sec.gov/Archives/edgar/data/1409896/000118143112051484/0001181431-12-051484.hdr.sgml

I am using Python 3 and have bee

Answer 1

If you can find an SGML DTD for the documents that you work with, a solution could be to use the osx SGML to XML converter from the OpenSP SGML toolkit to turn the documents into XML.

Here is a simple example. Let's say that we have the following SGML document (company.sgml; with a root element):

Awesome Corp
 24-7

101 PARSNIP LN
31337

The DTD (company.dtd) looks like this:

The - o bit means that the end tag can be omitted.

The SGML document can be parsed with osx, and the output can be formatted with xmllint, as follows:

osx company.sgml | xmllint --format -

Output from the above command:

  Awesome Corp
   24-7
  

    101 PARSNIP LN
    31337
  

Now we have well-formed XML that can be processed with lxml or other XML tools.

I don't know if there is a complete DTD for the document that you link to. The following PDF file contains related information about EDGAR, including a DTD that might be useful: http://www.sec.gov/info/edgar/pdsdissemspec910.pdf (I found it via this answer). But the linked SGML document contains elements (SEC-HEADER, for example) that are not mentioned in the PDF file.