Optimal amount of testing for Selenium

问题

This question arises in the context of Selenium, but it's really about internal testing in "production quality" software in general.

Basically, there are two things in Selenium that I am debating whether an internal test is needed and if so, how best to implement the test. (By internal test, I mean a test that is implemented within the script or library itself; not a dedicated unittest.)

Navigating through webpages in Selenium should always (IMHO) have internal tests to make sure that the webdriver has reached the correct target or the page is laid out correctly. These tests usually go something like this:


try:
  my_element = driver.find_element_by_xpath('some_xpath')
except NoSuchElementException:
  pass

OR alternatively, like this (see e.g., selenium testing w/find_elements):

try:
  my_multiple_elements = driver.find_elements_by_xpath('some_xpath')
  if (my_multiple_elements > 0):
    # success
  else:
    raise Exception()
except NoSuchElementException:
  raise Exception()

If we assume we're only looking for a single occurrence, which is the better internal test? It's possible that if the target site is changed, a previously unique xpath (returning only a single occurrence) may now return multiple occurrences making find_elements the more robust test. But what about time & space efficiency in running find_elements? (Assuming that find_elements must parse the whole document and therefore take longer.) Time efficiency may not be that bad for one Selenium test but what about 1,000?

Checking whether the webdriver has actually closed (see, e.g., checking for webdriver quit) or whether a [webdriver] object is actually created. Both these events have very low failure rates. In fact, neither of these actions have EVER failed, at least as far as I know. Additionally, I rarely--if ever--see projects where people test object instantiation. I.e.,


class MyObject(object):
  def __init__(self, x):
    self.x = x

def main():
  x = 5
  my_obj = MyObject(5)

  # TESTING
  if (my_obj is None):
    raise Exception()

My question for this is: why? Isn't it possible, however remote, that a constructor will fail to produce an object when called? Is the counter argument just that there is a slippery slope from checking object instantiation to checking variable instantiation to checking everything? Or is it true that due to space/time constraints, internal testing to that degree is simply unnecessary?

Ultimately, the general question is: When writing production quality software, some stuff has to be tested (see scenario 1). In that case, how thorough should the test be? OR if some stuff is likely not going to fail, should I test it at all (see scenario 2).

EDIT: Made some changes to the code example to reflect more robust testing.

回答1:

What you describe is probably best called "defensive programming": assuming that inputs to your code will break any assumptions you make about them.

So in your scenario 1, you'll never know whether a given web page at a given point in time will conform to the structure you assume, or whether it will actually be accessible in the first place. These are good reasons to perform checks.

On the other hand, checking object instantiation is out of scope of that kind of checking, provided that it doesn't depend on outside conditions. What's considered "outside" is up for debate, of course: If you consider the selenium server and browser to be part of your system and assume it's completely under control, you don't need to check successful instantiation of the webdriver object. From the point of view of only your Python process, selenium and the browser may, however, also be considered "outside", so checks may be in order. In mym opinion, the latter attitude makes sense for a library that may be used under unforeseen conditions.

To wrap it up, at least scenario 2 is very much a matter of finding a good balance, and the question you need to ask when considering a check within your production code is whether the particular piece of code depends on some kind of input you make assumptions about.

来源：https://stackoverflow.com/questions/35426824/optimal-amount-of-testing-for-selenium

标签

python

selenium

testing