If you want to override a template coming with an app in django (in app/templates/app/) you create a template of the same name in another directory, which the template loade
In the Django wiki, Simon Willison presents a trick to achieve the "self-extending template" effect. It isn't directly applicable if you're using the app_directories template loader, though.
Symlinking apps' templates directories inside a new directory might do a trick.