AWS Glue jobs log output and errors to two different CloudWatch logs, /aws-glue/jobs/error
and /aws-glue/jobs/output
by default. When I include p
I know the article is not new but maybe it could be helpful for someone: For me logging in glue works with the following lines of code:
# create glue context
glueContext = GlueContext(sc)
# set custom logging on
logger = glueContext.get_logger()
...
#write into the log file with:
logger.info("s3_key:" + your_value)