scala - What causes a NullPointerException when a SharedSparkContext (sc) is used outside a test function in FunSuite? -

the following scala code works fine, , test runs:

import org.scalatest._ import com.holdenkarau.spark.testing._  class dummytest extends funsuite sharedsparkcontext {    test("shared context works inside test functions.") {      val myrdd = sc.parallelize(list(1,2,3,4))    } } 

however, following scala code results in java.lang.nullpointerexception on line sc.parallelize:

import org.scalatest._ import com.holdenkarau.spark.testing._  class dummytest extends funsuite sharedsparkcontext {    val myrdd = sc.parallelize(list(1,2,3,4))    test("shared context works inside test functions.") {       assert(true)    } } 

what causes nullpointerexception when sparkcontext used outside of test function?

the sparkcontext declared within sharedsparkcontext not initialized part of trait's initialization. rather initialized in trait's beforeall() method, called test framework after suite has been instantiated. source here: https://github.com/holdenk/spark-testing-base/blob/master/src/main/pre-2.0/scala/com/holdenkarau/spark/testing/sharedsparkcontext.scala. if use while initializing class, beforeall() has not yet been called, still null.

so summarize, order is:

1. super-class initialization (code in trait body)
2. sub-class initialization (code in class's body)
3. beforeall() called
4. tests run

so can use sc in step 4 not in step 2.