Is it possible to increase the maximum number of characters that ksh variable accepts?

Question

This is a follow up question to

What is the maximum number of characters that the ksh variable accepts?

I checked my environment and it's allowing only

#include <sys/limits.h>
$ cpp <<  HERE | tail -1
> #include <limits.h>
> ARG_MAX
> HERE
1048576

Is there a way to increase this? Or any alternatives for

 while read line;
   do
      #parse logic
   done < $filename

To handle really long lines? Based from the records I'm parsing it will not stop at 2M character lines.

Environment Details :

 AIX $ KSH Version M-11/16/88f 

Answer 1

You could compile a Linux 3.7.x kernel, and edit its include/uapi/linux/limits.h file to increase the ARG_MAX argument (to some bigger power of two, e.g. 2097152). But you should rather have a lot of RAM (e.g. 8GBytes) if you want to increase it more.

The actual limit is related to execve(2). That man page has a paragraph on it.

But you could probably avoid having huge shell variables (in the Unix environment). Did you consider using some other tool (awk, python, perl ....) to read your file? Their variable environment is not the shell environment transmitted to forked programs, so they can have variables with very long values. Maybe ksh has some builtin (unexport) to avoid exporting some variable into the Unix environment.