How to return a newly created struct as a reference?
You can't. No way around this; it's simply impossible. As you said, if it's declared on the stack then the value will be dropped and any references would be invalidated.
So what makes Vec different?
A Vec is the owned counterpart of a slice (&[T]). While a Vec has a pointer to the beginning of data, a count, and a capacity, a slice only has the pointer and a count. Both guarantee that all the data is contiguous. In pseudo-Rust, they look like this:
struct Vec {
    data: *mut T,
    size: usize,
    capacity: usize,
}
struct Slice<'a, T> {
    data: *mut T,
    size: usize,
}
Vec::split_at can return slices because it essentially contains a slice. It's not creating something and returning a reference to it, it's just a copy of the pointer and the count.
If you create a borrowed counterpart to your owned datatype, then you could return that. Something like
struct BitVector {
    data: Vec,
    capacity: usize,
    storage_size: usize
}
struct BitSlice<'a> {
    data: &'a [u8],
    storage_size: usize,
}
impl BitVector {
    fn with_capacity(capacity: usize) -> BitVector {
        let storage_size = std::mem::size_of::() * 8;
        let len = (capacity / storage_size) + 1;
        BitVector { 
            data: vec![0; len],
            capacity: capacity,
            storage_size: storage_size
        }
    }
    fn split_at<'a>(&'a self) -> (BitSlice<'a>, BitSlice<'a>) {
        let (data_left, data_right) = self.data.split_at(0);
        let left = BitSlice {
            data: data_left,
            storage_size: self.storage_size
        };
        let right = BitSlice {
            data: data_right,
            storage_size: self.storage_size
        };
        (left, right)
    }
}
fn main() {}
To follow the theme of Vec, you would want to probably Deref and DerefMut to the BitSlice and then implement all the non-capacity-changing methods on the BitSlice.
  I suppose this is done for end user convenience as they rather deal with references than with boxes.
References and boxes should be mostly transparent at the use-site. The main reason is performance. A Box is heap-allocated.
  I think I could fix my errors by putting the returned bit vectors into a Box<_>
This would not be a good idea. You already have a heap allocation via the Vec, and boxing it would introduce another indirection and extra heap usage.
  It does work if I return (BitVector, BitVector), what are the downsides to doing this? Why does the SliceExt trait not do this?
Yes, here you are returning the heap-allocated structures. There's no downside to returning these, there's just the downside of performing the allocations. That's why SliceExt doesn't do it.
  Does this also directly translate to the split_at_mut variant? 
Yes.
struct BitSliceMut<'a> {
    data: &'a mut [u8],
    storage_size: usize,
}
fn split_at_mut<'a>(&'a mut self) -> (BitSliceMut<'a>, BitSliceMut<'a>) {
    let (data_left, data_right) = self.data.split_at_mut (0);
    let left = BitSliceMut {
        data: data_left,
        storage_size: self.storage_size
    };
    let right = BitSliceMut {
        data: data_right,
        storage_size: self.storage_size
    };
    (left, right)
}
This helps point out that &T and &mut T are different types and behave in different ways.
  it's not allowed to give (mut BitSlice<'a>, mut BitSlice<'a> as return type.
It doesn't make sense to return a mut T: What's the difference in `mut` before a variable name and after the `:`?. With a BitSliceMut, the mutability is an aspect of the contained type (&mut [u8]).