Creating Indices name Dynamically in Elasticsearch using Spring-Data Elasticsearch

Question

I have a use case where in need to create the indices per month in Elasticsearch. The idea is to create indices on the monthly bases so that they are easy to maintain and can be deleted when expired.For this to achieve i have used the spring-batch and have a monthly job which will create the indices on monthly bases For Elasticsearch-Java integration I have used the Spring-Data Elasticsearch implementation. The problem which i am facing now is, I am not able to figure out how to provide the dynamic-name to the index and mapping using the Entity object. My Current implementation is done keeping the Single Index in mind. Please find the below code which i am using to create indices

elasticsearchTemplate.createIndex(SingleChat.class);
elasticsearchTemplate.putMapping(SingleChat.class);
elasticsearchTemplate.refresh(SingleChat.class, true);

And SingleChat is my Entity Class

@Document(indexName="singlemsgtemp_#{jobParameters['MONTH']}",type="singlechat")
public class SingleChat {
    @org.springframework.data.annotation.Id
    String Id;
    @Field(type = FieldType.String)
    String conservationId;
    @Field(type = FieldType.String)
    String from;
    @Field(type = FieldType.String)
    String to;
    @Field(type = FieldType.String)
    String msgContent; 
    @Field(type = FieldType.String)
    String sessionId;
    @Field(type = FieldType.Date, index = FieldIndex.not_analyzed, store = true, format = DateFormat.date_hour_minute_second_millis)
    Date postedDate;
    @Field(type = FieldType.Date, index = FieldIndex.not_analyzed, store = true, format = DateFormat.date_hour_minute_second_millis)
    Date expireDate;
}   

But this is not working as expected. I am trying out some other things also. But i am open for suggestion. Also feel free to comment on my current approach, will it work or not. Please let me know if any more details are required.

Answer 1

What I am doing on my application is that I use ElasticSearchTemplate to create my dynamic index name and then I point an alias to the new index I have created and then drop the old one.

esTemplate.createIndex(newIndexName, loadfromFromFile(settingsFileName));
esTemplate.putMapping(newIndexName, "MYTYPE", loadfromFromFile(mappingFileName));

I'm not using the mapping and settings from my class since I need it to be dynamic.

    protected String loadFromFile(String fileName) throws IllegalStateException {
       StringBuilder buffer = new StringBuilder(2048);
       try {
           InputStream is = getClass().getResourceAsStream(fileName);
           LineNumberReader reader = new LineNumberReader(new InputStreamReader(is));
           while (reader.ready()) {
               buffer.append(reader.readLine());
               buffer.append(' ');
           }
       } catch (Exception e) {
           throw new IllegalStateException("couldn't load file " + fileName, e);
       }
       return buffer.toString();
   }

Answer 2

What I am doing in my project , we are storing index name and type name in DB whenever it gets changed .

Now i m getting index and type Dynamically in this manner:

First Solution

• 1)In configuration file ,create a Bean returning a value for someProperty . Here I injected the somePropertyValue with @Value annotation from DB or property file :-

  @Value("${config.somePropertyValue}")
  private String somePropertyValue;
  
  @Bean
  public String somePropertyValue(){
      return somePropertyValue;
  }
  

• 2)After this , it is possible to inject the somePropertyValue into the @Document annotation like this :-

  @Document(index = "#{@somePropertyValue}")
  public class Foobar {
      //...
  }
  

Second Solution

• 1) create getter setter in bean :-

  @Component
  public class config{
       @Value("${config.somePropertyValue}")
       private String somePropertyValue;
  
       public String getSomePropertyValue() {
         return somePropertyValue;
       }
      public void setSomePropertyValue(String somePropertyValue) {
         this.somePropertyValue = somePropertyValue;
      }
  }
  

• 2)After this , it is possible to inject the somePropertyValue into the @Document annotation like this :-

  @Document(index = "#{config.somePropertyValue}")
  public class Foobar {
      //...
  }