All shells understand these commands:
$ cd .
$ cd ..
And zsh will also understand:
$ cd ...
$ cd ....
Provide
You have to use compinit and use _expand_alias as completer. Here is an example:
zstyle ':completion:*' completer _complete _ignored _expand_alias
autoload -Uz compinit
compinit
_complete _ignored is the default setting for completer, you could set it to only _expand_alias but then completion would only work for aliases.
If compinit is already configured in your ~/.zshrc, then you just need to add _expand_alias into the list for completer, for example:
zstyle ':completion:*' completer _expand _complete _ignored _approximate _expand_alias
By default _expand_alias expands global and and regular aliases, if you do not want to expand regular aliases, set:
zstyle ':completion:*' regular false
Note: This of course works only, where global aliases would work. So they would not be expanded as part of an entire path like a/b/..../c/d