How to get ISO string in Nifi getMongo Query Field

问题:

I'm trying to use expression languge to generate ISO string in Nifi getMongo Query field using following query,

{ "remindmeDate": { "$gte": "${now():format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",'GMT')}", "$lte": "${now():toNumber():plus(359999):format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",'GMT')}" } } 

But i'm getting invalid JSON error error as double quotes are not escaped. When we try to escape it using \ operator, nifi is not evaluating the expression language. Is there any method or workaround to get this working ?

Thanks in advance

回答1:

GetMongo processor of nifi requires your query to be in extended json format of mongo.So you can use query of below format to query mongo based on datetime:

{"bday":{"$gt":{"$date":"2014-01-01T05:00:00.000Z"}, "$lt" :{"$date":"2019-01-  01T05:00:00.000Z"}}} 

回答2:

I think you may be able to use the unescapeJson expression language function to handle this. You have to provide valid JSON (escaped quotes) for the field level (PropertyDescriptor in NiFi parlance) validation, but the expression language string expects unescaped JSON during expression parsing, so the unescapeJson function removes the escapes first and then format receives a properly quoted string.

{ "remindmeDate": { "$gte": "${now():format(\"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\":unescapeJson(),'GMT')}", "$lte": "${now():toNumber():plus(359999):format(\"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'\":unescapeJson(),'GMT')}" } } 

回答3:

I used your not changed expression in UpdateAttribute processor to evaluate new flowFile attribute.

your expression:

{ "remindmeDate": { "$gte": "${now():format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",'GMT')}", "$lte": "${now():toNumber():plus(359999):format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",'GMT')}" } } 

the result:

{ "remindmeDate": { "$gte": "2017-06-16T07:38:04.811Z", "$lte": "2017-06-16T07:44:04.810Z" } } 

and this is a correct json object.

Finally I found that GetMongo.Query property does not support nifi expression language (nifi 1.2.0 and 1.3.0). Just hover the question mark near parameter.

It means no way to build dynamic query (

Seems need to register an issue... https://issues.apache.org/jira/browse/NIFI-4082

But it's possible to specify current and relative date in mongo query language. something like this:

{     "remindmeDate": {         "$gte": new Date(),         "$lte": new Date(ISODate().getTime() + 359999)     } } 

回答4:

Nifi's getMongo Query field doesnt support EL. So i created a stored function in MongoDB for my dynamic query and called it from Nifi.

{     "_id" : "reminderDateGMT",     "value" : function (reminderDateGMT) {             var reminder = new Date(reminderDateGMT)             var fromDate = new Date();             var toDate = new Date(new Date().getTime()+(1000 * 60 * 60));             if ((reminder >= fromDate) && (reminder <=toDate )) {                 return true;             } else {                 return false;             }          } } 

In nifi GetMongo Query,

{   "$where": "reminderDateGMT(this.reminderDateGMT)" } 

回答5:

I had a similar discussion on the mailing list, and here is the solution I found that works:

Mongo console:

db.system.js.save({     "_id": "lastFiveMinutes",     "value": function() {         return new Date(ISODate().getTime() - (1000 * 60 * 5));     } }); db.loadServerScripts(); 

Query field:

{     "$where": "obj.ts >= lastFiveMinutes()" } 

Note: you probably want to set this on a timer in the scheduling property.

文章来源: How to get ISO string in Nifi getMongo Query Field