if this is a bad idea, how to allocate memory in the function?
It's not a "bad idea", but rather "sometimes a bad idea", which can be said about many ideas in programming.
Allocating memory inside a function and releasing it outside may be a common design pattern, by the way. Consider:
// hashtable is a typedef-ed pointer type
hashtable ht = hashtable_new();
// .. do something with hashtable
hashtable_free(ht);
ht
was allocated in a function hashtable_new
and released outside it, yet you will see this pattern over and over in lots of good C code.
What it does show, however, is how the same logical unit (the hash-table ADT) takes care of allocation and de-allocation. This makes lots of sense - because the one who knows how to allocate, knows best how to deallocate. Allocating and releasing in different logical units is more often a bad idea.