Generating large pdf without having content in main memory

Question

I am using iText to generate very large tables in pdf format. What is the best way to generate these tables instead of having the entire content in memory? If I just increase the below size in the for loop to a million I run out of memory, is there a better way to stream it than having the entire content in memory

I saw this post How to directly stream large content to PDF with minimal memory footprint? But I want to know what iText api to use.

Example Code:

import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Date;

import com.itextpdf.text.Document;
import com.itextpdf.text.DocumentException;
import com.itextpdf.text.Element;
import com.itextpdf.text.Font;
import com.itextpdf.text.Font.FontFamily;
import com.itextpdf.text.Phrase;
import com.itextpdf.text.pdf.PdfPCell;
import com.itextpdf.text.pdf.PdfPTable;
import com.itextpdf.text.pdf.PdfWriter;

public class SimpleTable11 {
public final static String DEST = "/Users/.../Documents/test23.pdf";

public static void main(String[] args) throws IOException, DocumentException {
    new SimpleTable11().createPdf(DEST);
}

public void createPdf(String dest) throws IOException, DocumentException {

    System.out.println(new Date());
    Document document = new Document();
    PdfWriter.getInstance(document, new FileOutputStream(DEST));
    document.open();
    PdfPTable table = new PdfPTable(23);
    table.setWidths(new int[] { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 });
    table.setWidthPercentage(100);
    table.addCell(createCell("Account", 2, 1, Element.ALIGN_JUSTIFIED));
    table.addCell(createCell("Org Id", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Contract Number", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Transaction Type", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Transaction Number", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Transaction Date", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Start Date", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("End Date", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Billing Reference", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Line Description", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Product Name", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Related Invoices", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Monthly Unit Price", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("quantity", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Total Line Tax", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Total Price", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Exchange Rate", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Taxable", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Vat Rate", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("VAT", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Taxable", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("VAT", 2, 1, Element.ALIGN_LEFT));
    table.addCell(createCell("Total Price", 2, 1, Element.ALIGN_LEFT));

    String[] data = { "44445555", "123456", "0105567", "INV", "123456", "10/10/2018", "11/15/2018", "11/20/2050",
            "SO-0000000000-Mento", "Marketing Product", "Marketing Product ", "Marketing Product",
            "Marketing Product", "0.00", "12.56", "300.00", "0.566667345", "12.54", "10.00%", "12.56", "7.58",
            "7.27", "176.67" };
    for (int i = 0; i < 20000; i++) {
        for (int j = 0; j < data.length; j++) {
            table.addCell(createCell(data[j], 1, 1, Element.ALIGN_LEFT));
        }
    }
    document.add(table);
    document.close();
    System.out.println(new Date());
}

public PdfPCell createCell(String content, float borderWidth, int colspan, int alignment) {
    Font font = new Font(FontFamily.HELVETICA, 4, Font.NORMAL);
    PdfPCell cell = new PdfPCell(new Phrase(content, font));
    cell.setBorderWidth(borderWidth);
    cell.setColspan(colspan);
    cell.setHorizontalAlignment(alignment);
    return cell;
}

}

Answer 1

When using PdfPTable objects with very many cells you should make use of that class implementing LargeElement which is documented as

/**
 * Interface implemented by Element objects that can potentially consume
 * a lot of memory. Objects implementing the LargeElement interface can
 * be added to a Document more than once. If you have invoked setComplete(false),
 * they will be added partially and the content that was added will be
 * removed until you've invoked setComplete(true);
 * @since   iText 2.0.8
 */
public interface LargeElement extends Element

I.e. you should first use

setComplete(false),

then add some content (e.g. 20 rows) to the table, then add the table to the document, add some more content, add the table to the document again, etc..., and when all is added, use

setComplete(true)

and add the table once more. That way the table data does not remain on the heap but gets bit by bit serialized and written to the writer.

---

As an aside, there are other iText classes, too, which implement LargeElement. If you have issues with a huge memory consumption with iText, you should always check the objects you add to your Document: If they implement LargeElement, first try to forward them to the Document piece by piece as explained above.