as.numeric in sparkSQL

[wush978]

Under dplyr v 0.4.3, the as.numeric might fail.

After executing the following R script:

dplyr::group_by(df, day) %>%
  dplyr::summarise(imp = count(adid), clk = mean(as.numeric(is_click))) %>%
  dplyr::collect()

R will raise an error from spark: (org.apache.spark.sql.AnalysisException: cannot recognize input near 'numeric' ')' ')' in primitive type specification;

I fixed this issue according to https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-NumericTypes in bridgewell/dplyrSparkSQL@781fba1 . It seems that this package does not add these customized translator (

dplyr-spark/pkg/R/src-sparkSQL.R

Line 114 in e073c60

src_translate_env.src_SparkSQL =

)

Hope this helps.