I want to have docker CLI to connect to remote daemon but do I need to install the whole engine including daemon on the local machine?
On Windows, you can install the CLI by itself using chocolatey package manager.
Once you have chocolatey loaded you can run this from an admin command prompt:
choco install /y docker-cli
This seems to be much more up-to-date than the Windows link provided by Aaron, for some reason. (v19 instead of v17, as of Jan 2020)