What are the best practices, for building a multi-tenant app in the context of Kafka and storm?
For example: creating topic for each tenant and consume multi-topics spout (using wildcard).
I think that topic for each tenant is the right choice.
The naming convention might be something like this: topic_base_name_tenant_id.
The reasons are:
- It allows flexible configuration for each tenant (like @Sebastian mentioned earlier).
- Clearer logical separation.
Now let's say that we will use different approach. For example, partition for each tenant. It might be problematic, since:
- You are limiting the parallelism level to the number of tenants.
- Adding new tenants, results adding new partition → republish old messages (The default partitioning algorithm is: message_key % partition_size).
Sometimes you may need multiple topics per application. In that case you could follow the following topic naming convention: topic_base_name_tenant_id. So for a given base topic, you will have as many topics as the number of tenants
Don't consider my answer as the "best practice" because I'm not Kafka expert, but in case it helps: we faced a similar design scenario (one Kafka endpoint available to multiple teams that acts as a central point for event creation and routing). After pondering different alternatives, we decided what you suggest: to have one topic per application (you could say tenant), customizing the topic (partition, limits, etc.) according to the application needs.
Hope this helps.
来源：https://stackoverflow.com/questions/23516427/how-to-achieve-multi-tenancy-in-the-context-of-kafka-and-storm