📄️ How to archive processed data files in Spark Structured Streaming
Learn how to move already processed data automatically when using Spark Structured Streaming (also in Databricks)
📄️ Generating a Spark StructType from a Scala Case Class
Learn how to generate a Spark Struct Type from a Scala case class using the Scala Reflection system.
📄️ Reading incremental files from directory without Spark Streaming
Learn how to read incremental files (deltas) from a folder without using Apache Spark Streaming or Databricks in scala.
📄️ Reading SAS Files with Spark
How to read SAS7BDAT files with Spark in scala and some common issues.
📄️ Introduction to working and defining with Spark Encoders
How to use Encoders in Spark. Solving issues with Encoders in scala.