Tracking is a classical computer vision problem to which research is still devoted in computer science; you can quickly get a sense of the state-of-the-art in this field by checking the list of accepted papers in CVPR 2010 (which is an annual top computer vision conference) and you'll see that there is still active work being published on the topic (search for the word "tracking" within the list).
The standard processing pipeline of a solution for a tracking problem works as follows: The image is first parsed to extract meaningful descriptors that capture relevant corners and other salient features of the image. These descriptors are later fed to an on-line classifier that is trained to detect likely instances of your particular object of interest in each frame. The descriptor of your object may be known a priori, (i.e. computed off-line) from previous examples of what the object looks like, but it is usually updated in every frame by what the the system sees over time, to make the detection adaptive to the dynamic object appearance. Finally, in order to choose from a pool of possible candidates in each frame (from those that were detected), parameters such as the position and velocity of your objects are estimated with respect to previous frames using a sequential statistical model.
There is a vast computer vision literature on good image descriptors, but some of the most popular ones are SIFT, SURF, or HOG. For classification, two of the most successful methods are support vector machines or classification ensembles (e.g. boosting or random forests), and for the estimation part, most people still use Kalman filters (which is a type of sequential Markov model), particle filters or more generally density estimation models.
The specific case you described is a bit easier than the more general and difficult object-tracking problem with arbitrary camera and object motion in natural outdoor scenes, so you might be able to find some code online that could work right away in your setting, but I doubt it. As others pointed out, (and to the best of my knowledge), there is no off-the-shelf library that works right away for all sorts of objects, backgrounds and motion spaces. That said, you can probably find code for the individual components of the standard general pipeline I described above (classifiers, banks of filters/features, Markov estimation models) online.
My suggestion is, if you are interested in building a good system (i.e. one that actually works), then look at the websites of the authors of most recent papers in top annual computer vision conferences, such as CVPR, ICCV, ECCV and SIGGRAPH. They tend to have code online for their most recent work with some video examples, and this might help you get a sense of how their methods work in a real setting.
