I want to track users that are online at the moment.
The definition of being online is when they are on the index page of the website which has the chat function.
Well, how does the chat function work? Is it an ajax-based chat system?
Ajax-based chat systems work by the clients consistently hitting the chat server to see if there are any new messages in queue. If this is the case, you can update the user's online status either in a cookie or a PHP Session (assuming you are using PHP, of course). Then you can set the online timeout to be something slightly longer than the update frequency.
That is, if your chat system typically requests new messages from the server every 5 seconds, then you can assume that any user who hasn't sent a request for 10-15 seconds is no longer on the chat page.
If you are not using an ajax-based chat system (maybe Java or something), then you can still accomplish the same thing by adding an ajax request that goes out to the server periodically to establish whether or not the user is online.
I would not suggest storing this online status information in a database. Querying the database every couple of seconds to see who is online and who isn't is very resource intensive, especially if this is a large site. You should cache this information and operate on the cache (very fast) vs. the database (very slow by comparison).