If you read the complete error, it should say this:
"&" may only be used at the beginning of a compound selector.
However, .dropdown-toggle.btn-primary is the same as .btn-primary.dropdown-toggle. So place the parent selector at the beginning:
.btn-primary {
.open &.dropdown-toggle { color: blue; }
}
As of Sass 3.4, you can do that like this:
.btn-primary {
.open .dropdown-toggle#{&} { color: blue; }
}