Why am I getting a line Feed Error when loading data via Powershell into clickhouse in Docker on Windows?

问题

I am attempting to load data into clickhouse in a docker container built in windows docker desktop. I have my mock data prepared in R, written as a csv and my table created in clickhouse (i'm ommiting the connections):

library(dplyr)
library(data.table)
library(clickhouse)
setwd("C:/Users/xxxx/Documents/testing_load")
my_df = data.table(datetime = as.character(c("2018-01-01 11:21:00", "2019-01-01 11:45:00")))
c(2018, 2019) %>%
  lapply(function(y) {
    print(y)
    fwrite(my_df[substr(datetime,1,4) == y],
              paste("test_",y,".csv"),
              row.names = F,
              col.names = F
           )
  })


dbSendQuery(con,
            paste(
              "CREATE TABLE test(
              datetime DateTime

              ) ENGINE = Log;"
            )
)

The data I am trying to load is huge, so this is just a sample to show the setup and why I am getting an error on the first line. I want to load the data using clickhouse client in powershell accessing the docker container as such:

#loop through files and load
$files = Get-ChildItem "C:\Users\xxxx\Documents\testing_load"

foreach ($f in $files){
    $outfile = $f.FullName | Write-Host
    Import-Csv –Delimiter "," $f.FullName | Write-Host
    Get-Date | Write-Host    
    "Start loading" + $f.FullName | Write-Host
    docker run -it --rm --link chanalytics:clickhouse-server yandex/clickhouse-client --host clickhouse-server clickhouse-client --query="INSERT INTO test FORMAT CSV"
     Get-Date | Write-Host 
    "End loading" + $f.FullName | Write-Host
}

I added reading the data as well in case but I'm getting an error from clickhouse regarding the datetime time. I have tried switching to positxc in R and it makes no difference. I can't tell but feel that this is a very simple thing I am not understanding. Below is the error I get:

Opening excel I can see that the datetime is fine in date format. Any suggestions on what might be going on?

回答1:

I was able to solve the issue:

#loop through files
$files = Get-ChildItem "C:\Users\xxxx\Documents\testing_load"

foreach ($f in $files){
    $outfile = $f.FullName | Write-Host
    Get-Date | Write-Host    
    "Start loading" + $f.FullName | Write-Host
    cat $f.FullName | docker run -i --rm --link chanalytics:clickhouse-client yandex/clickhouse-client -m --host chanalytics --query="INSERT INTO tpep FORMAT CSV"
     Get-Date | Write-Host 
    "End loading" + $f.FullName | Write-Host
}

来源：https://stackoverflow.com/questions/57349761/why-am-i-getting-a-line-feed-error-when-loading-data-via-powershell-into-clickho

标签

powershell

Docker

ClickHouse