I used Neil-N's class but changed the if statement to test the Type property instead of the Tag property. I set each column to Type Number (instead of Text) that had an integer value in it. Sort works great.
if (l1.ListView.Columns[Column].Type.ToString() == "Number")