Extract value from ValueProvider in Apache Beam

左心房为你撑大大i 提交于 2019-12-12 04:37:59

问题


I have a runtime value that I'm getting in my Apache Beam program. I need to access that value but Beam does not allow me to read that value unless I'm reading it from within a transfrom like ParDo.

If I try to access that value outside any transform, it gives me an error saying: "Not called from a runtime context".

How to read such values?

P.S. I'm using a template of the program.


回答1:


A program (such as a template) is executed in two stages. In the first, the main method is evaluated to produce a pipeline structure. This represents the template. In the second stage, that pipeline structure is executed.

Accessing a ValueProvider during evaluation (outside of a user-defined function like a DoFn) is imposible, because the runtime values aren't provided until the second stage.

Could you elaborate on what you are trying to do? Generally, the solution is to make the ValueProvider available to the DoFn, and then have the DoFn evaluate the ValueProvider at runtime.



来源:https://stackoverflow.com/questions/45818059/extract-value-from-valueprovider-in-apache-beam

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!