none
Issue Installing RMySQL package on spark cluster (Azure Data Bricks)

All replies

  • Hi Venkatesh,

    Are you following a tutorial by chance: Tutorial: Extract, transform, and load data using Azure Databricks

    Would it be possible for you to include some repro steps, or provide screen capture(s) and log files showing the issue with greater clarity? Thank you!

    Monday, August 6, 2018 6:49 PM
    Moderator
  • Hello Mike,Thank you for the response.
    I am following the tutorial, here is the Error log screenshot. 
    I tried in different ways(creating library in work space, installing using R) but it was not working in both way's. 

    Monday, August 6, 2018 7:22 PM
  • I couldn't send the screenshots. This page not letting me send them.
    Let me send you the error details here,

    1. Error log of creating RMySQL library in work-space.
    Error: java.lang.RuntimeException: Installation failed with message: Error installing R package: Could not install package with error: installation of package ‘RMySQL’ had non-zero exit status Full error log available at /databricks/driver/library-install-logs/r-package-installation-RMySQL-2018-08-06T14:42:30Z-jeXxn9.log Error installing R package: Could not install package with error: installation of package ‘RMySQL’ had non-zero exit status Full error log available at /databricks/driver/library-install-logs/r-package-installation-RMySQL-2018-08-06T14:42:30Z-smSU18.log
    2.Error log of using install.packages("RMySQL") in notebook.

    Installing package into ‘/databricks/spark/R/lib’
    (as ‘lib’ is unspecified)
    trying URL 'https://cloud.r-project.org/src/contrib/RMySQL_0.10.15.tar.gz'
    Content type 'application/x-gzip' length 52728 bytes (51 KB)
    ==================================================
    downloaded 51 KB
    
    * installing *source* package ‘RMySQL’ ...
    ** package ‘RMySQL’ successfully unpacked and MD5 sums checked
    Using PKG_CFLAGS=
    Using PKG_LIBS=-lmysqlclient
    ------------------------- ANTICONF ERROR ---------------------------
    Configuration failed because no mysql client library was found. Try installing:
     * deb: libmariadbclient-dev | libmariadb-client-lgpl-dev (Debian, Ubuntu)
     * rpm: mariadb-connector-c-devel | mariadb-devel | mysql-devel (Fedora, CentOS, RHEL)
     * csw: mysql56_dev (Solaris)
     * brew: mariadb-connector-c (OSX)
    If you already have a mysql client library installed, verify that either
    mariadb_config or mysql_config is on your PATH. If these are unavailable
    you can also set INCLUDE_DIR and LIB_DIR manually via:
    R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
    --------------------------------------------------------------------
    ERROR: configuration failed for package ‘RMySQL’
    * removing ‘/databricks/spark/R/lib/RMySQL’
    
    The downloaded source packages are in
    	‘/tmp/RtmpyMbK8t/downloaded_packages’

    Monday, August 6, 2018 7:29 PM
  • Although MySQL related, I am going to move this to another forum with the hope Azure Data Lake / Data Bricks will be more applicable. 
    Monday, August 6, 2018 7:49 PM
    Moderator
  • I will wait for the response, Thank you.
    Monday, August 6, 2018 8:41 PM