Should you? Yes.
Why? Log4J has essentially been deprecated by Logback.
Is it urgent? Maybe not.
Is it painless? Probably, but it may depend on your logging statements.
Note that if you really want to take full advantage of LogBack (or SLF4J), then you really need to write proper logging statements. This will yield advantages like faster code because of the lazy evaluation, and less lines of code because you can avoid guards.
Finally, I highly recommend SLF4J. (Why recreate the wheel with your own facade?)