# Interaction with Pyspark[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#module-dataiku.spark "Permalink to this headline")

* 
`dataiku.spark.``start_spark_context_and_setup_sql_context`(*load\_defaults=True*, *hive\_db='dataiku'*, *conf={}*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.start_spark_context_and_setup_sql_context "Permalink to this definition"): Helper to start a Spark Context and a SQL Context “like DSS recipes do”.
This helper is mainly for information purpose and not used by default.

* 
`dataiku.spark.``setup_sql_context`(*sc*, *hive\_db='dataiku'*, *conf={}*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.setup_sql_context "Permalink to this definition"): Helper to start a SQL Context “like DSS recipes do”.
This helper is mainly for information purpose and not used by default.

* 
`dataiku.spark.``distribute_py_libs`(*sc*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.distribute_py_libs "Permalink to this definition"):

* 
`dataiku.spark.``get_dataframe`(*sqlContext*, *dataset*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.get_dataframe "Permalink to this definition"): Opens a DSS dataset as a SparkSQL dataframe. The ‘dataset’ argument must be a dataiku.Dataset object

* 
`dataiku.spark.``write_schema_from_dataframe`(*dataset*, *dataframe*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.write_schema_from_dataframe "Permalink to this definition"): Sets the schema on an existing dataset to be write-compatible with given SparkSQL dataframe

* 
`dataiku.spark.``write_dataframe`(*dataset*, *dataframe*, *delete\_first=True*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.write_dataframe "Permalink to this definition"): Saves a SparkSQL dataframe into an existing DSS dataset

* 
`dataiku.spark.``write_with_schema`(*dataset*, *dataframe*, *delete\_first=True*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.write_with_schema "Permalink to this definition"): Writes a SparkSQL dataframe into an existing DSS dataset. This first overrides
the schema of the dataset to match the schema of the dataframe

* 
`dataiku.spark.``apply_prepare_recipe`(*df*, *recipe\_name*, *project\_key=None*)[¶](https://doc.dataiku.com/dss/latest/api/python/pyspark.html#dataiku.spark.apply_prepare_recipe "Permalink to this definition"):
