I\'d like to have \"Download Latest Version\" button on my website which would represent the link to the latest release (stored at GitHub Releases>
As noted by @Dan Dascalescu in a comment to accepted answer, there are some projects (roughly 30%) which do not bother to file formal releases, so neither "Latest release" button nor /releases/latest API call would return useful data.
To reliably fetch the latest release for a GitHub project, you can use lastversion.