It's not yet possible with the main sigma.js library. However, I just forked the repository and added the capability on github.com here: https://github.com/sam2themax/sigma.js
To enable this new feature, set edgeLabels to true in your drawingProperties.