Can pandas handle variable-length whitespace as column delimiters [duplicate]

Answer 1

I think there's just a missing \ in the docs (maybe because it was interpreted as an escape marker at some point?) It's a regexp, after all:

In [68]: data = read_table('sample.txt', skiprows=3, header=None, sep=r"\s*")

In [69]: data
Out[69]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 7 entries, 0 to 6
Data columns:
X.1     7  non-null values
X.2     7  non-null values
X.3     7  non-null values
X.4     7  non-null values
X.5     7  non-null values
X.6     7  non-null values
[...]
X.23    7  non-null values
X.24    7  non-null values
X.25    5  non-null values
X.26    3  non-null values
dtypes: float64(8), int64(10), object(8)

Because of the delimiter problem noted by @MRAB, it has some trouble with the last few columns:

In [73]: data.ix[:,20:]
Out[73]: 
   X.21  X.22           X.23                   X.24            X.25    X.26
0   315  0.95            ABC            transporter   transmembrane  region
1   527  0.93            ABC            transporter            None    None
2   408  0.86  RecF/RecN/SMC                      N        terminal  domain
3   575  0.85  RecF/RecN/SMC                      N        terminal  domain
4   556  0.72            AAA                 ATPase          domain    None
5   275  0.85      YceG-like                 family            None    None
6   200  0.85       Pyridine  nucleotide-disulphide  oxidoreductase    None

but that can be patched up at the end.

Answer 2

You should be able to just do this, which @DSM just taught me in another thread:

data = read_table('sample.txt', skiprows=3, header=None, delim_whitespace=True)

Documentation

Answer 3

None of the given answers works in a case like this:

Block..Col.name.with.spaces..col3
...1..6.141754e+003..2.998903e+000
2048..6.154461e+003..6.010216e+000

that is, two or more spaces are used as separators, but the column names can themselves contain one space.

In such a case, we need a regular expression for two or more spaces. This will work:

sep=r"[ ]{2,}"

But again, the drawback is that it triggers the python parser.