📄️ How to define an UDFs in Spark with scala
Code examples on how to define an UDF (User Defined Function) in Spark with Scala and include unit tests.
📄️ Exploding arrays in Spark and keeping the index position of each element
Check how to explode arrays in Spark and how to keep the index position of each element in SQL and Scala with examples.
📄️ Group output files from a Spark dataframe
How to group output files from a Spark Dataframe into a single file so you can use it in other applications.
📄️ How to read from Databases in Apache Spark
How to read from a Database with Spark (scala) using jdbc connections.
📄️ How to use Case-When expressions in Spark
Learn Spark basics - How to use the Case-When syntax in your spark queries. Spark SQL, Scala API and Pyspark with examples.
📄️ Writing Spark dataframes to file with a size limit
How to write a spark dataframe in partitions with a maximum limit in the file size.
📄️ How to write to databases in Apache Spark
How to write a dataframe to databases with Apache Spark using jdbc connections.