Found the solution from here: http://social.msdn.microsoft.com/Forums/en-US/wpf/thread/3d0eaa54-09a9-4c51-8677-8e90577e7bac/
XAML:
C#:
protected void HandleDoubleClick(object sender, MouseButtonEventArgs e)
{
var track = ((ListViewItem) sender).Content as Track; //Casting back to the binded Track
}