If using mentioned jquery mousewheel plugin, then what about to use the 2nd argument of event handler function - delta
:
$('#my-element').on('mousewheel', function(event, delta) {
if(delta > 0) {
console.log('scroll up');
}
else {
console.log('scroll down');
}
});