Turn on grep
's line buffering mode when using BSD grep (FreeBSD, Mac OS X etc.)
tail -f file | grep --line-buffered my_pattern
You don't need to do this for GNU grep (used on pretty much any Linux) as it will flush by default (YMMV for other Unix-likes such as SmartOS, AIX or QNX).