Is there a way to create out of DOM elements in Web Worker?

Question

Context: I have a web application that processes and shows huge log files. They\'re usually only about 100k lines long, but it can be up to 4 million lines

Answer 1

You have a couple of anti-patterns in your design:

1. Creating a DOM object has considerable overhead, and you are creating potentially millions of them at once.
2. Trying to get a web worker to manage the DOM is exactly what web workers are not for. They do everything else so the DOM event loop stays responsive.

You can use a cursor pattern to scroll through arbitrarily large sets of data.

1. DOM posts a message to worker with start position and number of lines requested (cursor).
2. Web worker random accesses logs, posts back the fetched lines (cursor data).
3. DOM updates an element with the async cursor response event.

This way, the heavy lifting is done by the worker, whose event loop is blocked during the fetch instead of the DOM, resulting in happy non-blocked users marvelling at how smooth all your animations are.