You should set some specific options on your FKey, such as ON DELETE {CASCADE, SET NULL, SET DEFAULT}
Instead you'll not be able to delete referenced row, because it is prohibited by sql server due to referrential integrity.
So, the option is to set referencing table's value to NULL or any other DEFAULT value.
Or delete it too
Or, if your referencing row has some meaning without parent row - then something is wrong with your DB design - either you do not need the FKey or schema is not normalized.