Indicator functions with by variables in mgcv or gamm4

Question

I am interested in fitting a generalized additive mixed model with multivariate response. A simple example that explains the problem is a model which has the form y_{i} =