Your code works OK, the problem is that you're trying to overwrite the file you've read the data from.
See this answer C# : the close method of Xml.Load(file)
This should works:
XDocument xdoc = XDocument.Load(filename);
xdoc.Element("WorkTable").Element("SpecialDays").Elements("Day")
.Where(x => (string)x.Attribute("date") == "24.07.2015")
.Remove();
xdoc.Save(filename);