Load JSON data into a Spark Dataframe using Scala

In this article we will load JSON data into a Spark Dataframe using Scala.

The delimited file is provided below for reference.



The SBT library dependencies are shown below for reference.

scalaVersion := "2.11.12"

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"

libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.0"

The Scala program is provided below.

import org.apache.spark.sql.SparkSession

object JsonReader extends App {

  val spark = SparkSession.builder()
    .master("local")
    .appName("JsonFileReader")
    .getOrCreate()

  import spark.implicits._

  val df = spark    .read
    .format("json")
    .option("multiline", "true")
    .load("C:\\data\\data.json")

  df.show()
}

Here is the output after running the program.

+---+---+------+
|Age| Id|  Name|
+---+---+------+
| 53|  1|Name-1|
| 42|  2|Name-2|
| 37|  3|Name-3|
+---+---+------+

Thanks. That is all for now!