Error: TABLE_QUERY expressions cannot query BigQuery tables

Question

This s a followup question regarding Jordans answer here: Weird error in BigQuery

I was using to query reference table within \"Table_Query\" for quit some time. Now

Answer 1

This is not ideal solution. But it seems to do the job.

In my previous query I passed the IDs List as a parameter in an external process that constructed the query. I wanted this process to be unaware to any logic implemented in the query.

Eventually we came up with this solution:

Instead of passing a list of IDs, we pass a JSON that contains the relevant meta data for each ID. We parse this JSON within the Table_Query() function. So instead of querying a physical reference table, we query some sort of a "table variable" that we have put in a JSON.
Below is a sample query that runs on the public dataset that demonstrates this solution.

    SELECT
  YEAR,
  COUNT (*) CNT
FROM
  TABLE_QUERY([fh-bigquery:weather_gsod], 'table_id in
(Select table_id
From
(Select table_id,concat(Right(table_id,4),"0101") as TBL_Date from [fh-bigquery:weather_gsod.__TABLES_SUMMARY__]
where table_id Contains "gsod"
)TBLs
CROSS JOIN 
(select 
Regexp_Replace(Regexp_extract(SPLIT(DatesInput,"},{"),r"\"fromDate\":\"(\d\d\d\d-\d\d-\d\d)\""),"-","") as fromDate,
Regexp_Replace(Regexp_extract(SPLIT(DatesInput,"},{"),r"\"toDate\":\"(\d\d\d\d-\d\d-\d\d)\""),"-","") as toDate,
FROM
(Select
"[  
      {  
         \"CycleID\":\"123456\",
         \"fromDate\":\"1929-01-01\",
         \"toDate\":\"1950-01-10\"
      },{  
         \"CycleID\":\"123456\",
         \"fromDate\":\"1970-02-01\",
         \"toDate\":\"2000-02-10\"
      }
   ]"
   as DatesInput)) RefDates
   WHERE TBLs.TBL_Date>=RefDates.fromDate
   AND TBLs.TBL_Date<=RefDates.toDate
)')
GROUP BY
  YEAR
ORDER BY
  YEAR

This solution is not ideal as it requires an external process to be aware of the data stored in the reference tables. Ideally the BigQuery team will re-enable this very useful functionality.