How can Office Add-in (Task-pane) not display by default

问题

I have an Task-pane Office Add-in (https://msdn.microsoft.com/en-us/library/office/fp123523.aspx) on registered Office 365 Site. When the user opens a spreadsheet from a document library on that site, then in edit mode, can use the "Office Add-ins" from the Insert menu to insert my add-in to the Spreadsheet. My add-in helps them population and update data on the spreadsheet. They then save the spreadsheet back with the updated data.

Now, the next time they open that spreadsheet, my task-pane shows up again. Is there any the task pane can not start opened? I would like the user to explicitly open from the ribbon bar.

Furthermore, when they download and give the spreadsheet to someone else, and that person opens the spreadsheet using Excel 2013, the add-in shows up again (they are asked if they want to run the add-in)

In my case, I never want the user to have access to this task-pane add-in from Excel desktop. So I'm thinking blanking out the page when I can determine it's running in the desktop. I would, however, would like the add-in window closed. Is there a way to do that?

The best way to understand the functionality of my add-in is that it's kind of like the TFS add-in in Excel. It's purpose is to populate and pull data. After you populate the data in the spreadsheet, you don't want the Add-in still there open. In fact, someone without the TFS Add-in can open the spreadsheet and it all works. Unfortunately, we cannot create ribbon add-ins like TFS has using the Office Online technology. The Task-pane will suffice, but it seems to be too "sticky".