You don't need jQuery for this. You can use JavaScript's .childNodes.length.
Just make sure to subtract 1 if you don't want to include the default text node (which is empty by default). Thus, you'd use the following:
var count = elem.childNodes.length - 1;