Azure Data Factory - MS Access as Source Database - Error

有些话、适合烂在心里 提交于 2021-02-10 20:31:03

问题


My source is 'Access Database'

Dynamically generating Source query as 'Select * from <tableName>'

But I got field names with spaces in source table, and destination is of type .parquet, Data Factory pipeline is failing with below error

Example if Table Employee got a column 'First Name'

 {
    "errorCode": "2200",
    "message": "Failure happened on 'Sink' side. ErrorCode=UserErrorJavaInvocationException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking java,
     message: java.lang.IllegalArgumentException:field ended by ';': expected ';' but got 'Area' at line 0: message adms_schema { optional binary RAM Area\ntotal entry:10\r
     \norg.apache.parquet.schema.MessageTypeParser.check(MessageTypeParser.java:215)\r\norg.apache.parquet.schema.MessageTypeParser.addPrimitiveType(MessageTypeParser.java:188)\r
     \norg.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:112)\r\norg.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:100)\r
     \norg.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:93)\r\norg.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:83)\r
     \ncom.microsoft.datatransfer.bridge.parquet.ParquetWriterBuilderBridge.getSchema(ParquetWriterBuilderBridge.java:187)\r\ncom.microsoft.datatransfer.bridge.parquet.ParquetWriterBuilderBridge.build
     (ParquetWriterBuilderBridge.java:159)\r\ncom.microsoft.datatransfer.bridge.parquet.ParquetWriterBridge.open(ParquetWriterBridge.java:13)\r
     \ncom.microsoft.datatransfer.bridge.parquet.ParquetFileBridge.createWriter(ParquetFileBridge.java:27)\r
     \n,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.DataTransfer.Richfile.JniExt.JavaBridgeException,Message=,Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'",
    "failureType": "UserError",
    "target": "Copy Table Data to Sink",
    "details": []
 }

if i change query to SELECT [First Name] as FirstName from Employee, it works fine.

As am generating query dynamically, i was using '*'

Is there some setting on Sink (.parquet) to ignore spaces in column names?

EDIT some info here https://issues.apache.org/jira/browse/SPARK-4521, not sure how to deal in ADF.

And this link: https://github.com/MicrosoftDocs/azure-docs/issues/28320

来源:https://stackoverflow.com/questions/59044405/azure-data-factory-ms-access-as-source-database-error

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!