I want to add an icon to my child context menus. But right now, the chrome extension options for context menu doesnt provide any option to add icon while creating a child me
serg answer is no longer true. Add this snippet into your manifest:
"icons": {
   "16": "icon16.png", --> this icons will be used for you menu item
   "32": "icon32.png"
}
                                                                        A possible workaround is using Unicode Symbols:
  
Adapted code from this Google sample:
// Create a parent item and two children.
chrome.contextMenus.create({"title": "⛔ Parent", "id": "parent"});
chrome.contextMenus.create(
  {"title": "♣ Child 1", "parentId": "parent", "id": "child1"});
chrome.contextMenus.create(
  {"title": "⚑ Child 2", "parentId": "parent", "id": "child2"});
The submenu "Radio 1" behaves as a radio button, and it's defined with chrome.contextMenus.create({"title": "Radio 1", "type": "radio", "id": "radio1"});
This is just one of many pages of symbols:

Sorry, it is currently not possible. All you got is that one parent icon set through manifest.