Suppressing treatment of string as iterable

Question

UPDATE:

An idea to make built-in strings non-iterable was proposed on python.org in 2006. My question differs in that I'm trying to only suppress this features once in a while; still this whole thread is quite relevant.

Here are the critical comments by Guido who implemented non-iterable str on a trial basis:

[...] I implemented this (it was really simple to do) but then found I had to fix tons of places that iterate over strings. For example:

• The sre parser and compiler use things like set("0123456789") and also iterate over the characters of the input regexp to parse it.

• difflib has an API defined for either two lists of strings (a typical line-by-line diff of a file), or two strings (a typical intra-line diff), or even two lists of anything (for a generalized sequence diff).

• small changes in optparse.py, textwrap.py, string.py.

And I'm not even at the point where the regrtest.py framework even works (due to the difflib problem).

I'm abandoning this project; the patch is SF patch 1471291. I'm no longer in favor of this idea; it's just not practical, and the premise that there are few good reasons to iterate over a string has been refuted by the use cases I found in both sre and difflib.

ORIGINAL QUESTION:

While it's a neat feature of the language that a string is an iterable, when combined with the duck typing, it may lead to disaster:

# record has to support [] operation to set/retrieve values
# fields has to be an iterable that contains the fields to be set
def set_fields(record, fields, value):
  for f in fields:
    record[f] = value

set_fields(weapon1, ('Name', 'ShortName'), 'Dagger')
set_fields(weapon2, ('Name',), 'Katana')
set_fields(weapon3, 'Name', 'Wand') # I was tired and forgot to put parentheses

No exception will be raised, and there's no easy way to catch this except by testing for isinstance(fields, str) in a myriad places. In some circumstances, this bug will take a very long time to find.

I want to disable strings from being treated as an iterable entirely in my project. Is it a good idea? Can it be done easily and safely?

Perhaps I could subclass built-in str such that I would need to explicitly call get_iter() if I wanted its object to be treated as an iterable. Then whenever I need a string literal, I would instead create an object of this class.

Here are some tangentially related questions:

How can I tell if a python variable is a string or a list?

how to tell a variable is iterable but not a string

Answer 1

There aren't any ways to do this automatically, unfortunately. The solution you propose (a str subclass that isn't iterable) suffers from the same problem as isinstance() ... namely, you have to remember to use it everywhere you use a string, because there's no way to make Python use it in place of the native class. And of course you can't monkey-patch the built-in objects.

I might suggest that if you find yourself writing a function that takes either an iterable container or a string, maybe there's something wrong with your design. Sometimes you can't avoid it, though.

In my mind, the least intrusive thing to do is to put the check into a function and call that when you get into a loop. This at least puts the behavior change where you are most likely to see it: in the for statement, not buried away somewhere in a class.

def iterate_no_strings(item):
    if issubclass(item, str):   # issubclass(item, basestring) for Py 2.x
        return iter([item])
    else:
        return iter(item)

for thing in iterate_no_strings(things):
    # do something...

Answer 2

To expand, and make an answer out of it:

No, you shouldn't do this.

1. It changes the functionality people expect from strings.
2. It means extra overhead throughout your program.
3. It's largely unnecessary.
4. Checking types is very unpythonic.

You can do it, and the methods you have given are probably the best ways (for the record, I think sub-classing is the better option If you have to do it, see @kindall's method) but it's simply not worth doing, and it's not very pythonic. Avoid the bugs in the first place. In your example, you might want to ask yourself if that's more an issue with clarity in your arguments, and whether named arguments or the splat might be a better solution.

E.g: Change the ordering.

def set_fields(record, value, *fields):
  for f in fields:
    record[f] = value

set_fields(weapon1, 'Dagger', *('Name', 'ShortName')) #If you had a tuple you wanted to use.
set_fields(weapon2, 'Katana', 'Name')
set_fields(weapon3, 'Wand', 'Name')

E.g: Named arguments.

def set_fields(record, fields, value):
  for f in fields:
    record[f] = value

set_fields(record=weapon1, fields=('Name', 'ShortName'), value='Dagger')
set_fields(record=weapon2, fields=('Name'), value='Katana')
set_fields(record=weapon3, fields='Name', value='Wand') #I find this easier to spot.

If you really want the order the same, but don't think the named arguments idea is clear enough, then what about making each record a dict-like item instead of a dict (if it isn't already) and having:

class Record:
    ...
    def set_fields(self, *fields, value):
        for f in fileds:
            self[f] = value

weapon1.set_fields("Name", "ShortName", value="Dagger")

The only issue here is the introduced class and the fact that value parameter has to be done with a keyword, although it keeps it clear.

Alternatively, if you are using Python 3, you always have the option of using extended tuple unpacking:

def set_fields(*args):
      record, *fields, value = args
      for f in fields:
        record[f] = value

set_fields(weapon1, 'Name', 'ShortName', 'Dagger')
set_fields(weapon2, 'Name', 'Katana')
set_fields(weapon3, 'Name', 'Wand')

Or, for my last example:

class Record:
    ...
    def set_fields(self, *args):
        *fields, value = args
        for f in fileds:
            self[f] = value

weapon1.set_fields("Name", "ShortName", "Dagger")

However, these do leave some weirdness when reading the function calls, due to the fact one usually assumes that arguments would not be handled this way.

Answer 3

Type checking in this case is not unpythonic or bad. Just do a:

if isinstance(var, (str, bytes)):
    var = [var]

In the beginning of the call. Or, if you want to educate the caller:

if isinstance(var, (str, bytes)):
    raise TypeError("Var should be an iterable, not str or bytes")

Answer 4

What do you think about creating a non-iterable string?

class non_iter_str(str):
    def __iter__(self):
        yield self

>>> my_str = non_iter_str('stackoverflow')
>>> my_str
'stackoverflow'
>>> my_str[5:]
'overflow'
>>> for s in my_str:
...   print s
... 
stackoverflow

Answer 5

Instead of trying to make your strings non-iterable, switch the way you are looking at the problem: One of your parameters is either an iterable, or a ...

• string
• int
• custom class
• etc.

When you write your function, the first thing you do is validate your parameters, right?

def set_fields(record, fields, value):
    if isinstance(fields, str):
        fields = (fields, )  # tuple-ize it!
    for f in fields:
        record[f] = value

This will serve you well as you deal with other functions and parameters that can be either singular, or pluralized.