Build a multimodal LSTM

Question

I have the following LSTM network. I want to add the red line in this figure to the model. Here is my model:

import numpy as np
import tensorflow as tf
from