How to use Bert for long text classification?

Question

We know that bert has a max length limit of tokens = 512, So if an acticle has a length of much bigger than 512, such as 10000 tokens in text How can bert be used?

Answer 1

There are two main methods:

• Concatenating 'short' BERT altogether (which consists of 512 characters max)
• Constructing a real long BERT (CogLTX, Blockwise BERT, Longformer, Big Bird)

I resumed some typical papers of BERT for long text in this post : https://lethienhoablog.wordpress.com/2020/11/19/paper-dissected-and-recap-4-which-bert-for-long-text/

You can have an overview of all methods there.