Can't boot to windows, when Ubuntu drive is not plugged in

Question

So I installed Ubuntu on my portable HDD, and when that HDD is plugged in to my PC, I get the boot menu asking me whether I want to boot windows or Ubuntu. However, when I r