| sparkR.session {SparkR} | R Documentation |
SparkSession is the entry point into SparkR. sparkR.session gets the existing
SparkSession or initializes a new SparkSession.
Additional Spark properties can be set in ..., and these named parameters take priority
over values in master, appName, named lists of sparkConfig.
When called in an interactive session, this checks for the Spark installation, and, if not
found, it will be downloaded and cached automatically. Alternatively, install.spark can
be called manually.
sparkR.session(master = "", appName = "SparkR",
sparkHome = Sys.getenv("SPARK_HOME"), sparkConfig = list(),
sparkJars = "", sparkPackages = "", enableHiveSupport = TRUE, ...)
master |
the Spark master URL. |
appName |
application name to register with cluster manager. |
sparkHome |
Spark Home directory. |
sparkConfig |
named list of Spark configuration to set on worker nodes. |
sparkJars |
character vector of jar files to pass to the worker nodes. |
sparkPackages |
character vector of package coordinates |
enableHiveSupport |
enable support for Hive, fallback if not built with Hive support; once set, this cannot be turned off on an existing session |
... |
named Spark properties passed to the method. |
For details on how to initialize and use SparkR, refer to SparkR programming guide at http://spark.apache.org/docs/latest/sparkr.html#starting-up-sparksession.
sparkR.session since 2.0.0
## Not run:
##D sparkR.session()
##D df <- read.json(path)
##D
##D sparkR.session("local[2]", "SparkR", "/home/spark")
##D sparkR.session("yarn-client", "SparkR", "/home/spark",
##D list(spark.executor.memory="4g"),
##D c("one.jar", "two.jar", "three.jar"),
##D c("com.databricks:spark-avro_2.10:2.0.1"))
##D sparkR.session(spark.master = "yarn-client", spark.executor.memory = "4g")
## End(Not run)