问题
I see that the DPU is made of 4 vCPUs & 16 GB memory. Is it possible to change this settings for vCPU, memory, so that I don't run out of DPUs or exceed the DPU limit.
I think there is a maximum limit of 5 DPUs for a Dev Endpoint, and a maximum of 2 DEV Endpoints for an account?
Regards
Yuva
回答1:
Right now there is no way to configure DPU memory, but you can request a limit increase on your account to be able to use more DPUs.
回答2:
As of April 2019, there are two new types of workers:
You can now specify a worker type for Apache Spark jobs in AWS Glue for memory intensive workloads.
Previously, all Apache Spark jobs in AWS Glue ran with a standard configuration of 1 Data Processing Unit (DPU) per worker node and 2 Apache Spark executors per node. You can now pick from two new configurations, G.1X and G.2X, that provide more memory per executor.
https://aws.amazon.com/about-aws/whats-new/2019/04/aws-glue-now-supports-additional-configuration-options-for-memory-intensive-jobs/
来源：https://stackoverflow.com/questions/48684663/aws-glue-dpu-configurations