Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

6.  def compareAppleTables(left: AppleTable, right: AppleTable): DiffResult

   Performs schema based comparison irrespective of source data types

   Performs schema based comparison irrespective of source data types

   left

   Custom table for source1

   right

   Custom table for source2

   returns

   a pair of DataFrames, the left parameter has values in DF1 and not in DF2, the right parameter has values in DF2 but not in DF1

7.  def compareAppleTablesCount(left: AppleTable, right: AppleTable): CountResult

   Performs schema based comparison irrespective of source data types

   Performs schema based comparison irrespective of source data types

   left

   Custom table for LeftSource

   right

   Custom table for RightSource

   returns

   a pair containing the count in left and right

8.  def compareAppleTablesSaveResults(left: AppleTable, right: AppleTable, outputDirectory: String, singleFileOutput: Boolean, delimiter: String): Unit

   Performs comparison between two custom source data types that were created from the actual source data details

   Performs comparison between two custom source data types that were created from the actual source data details

   left

   Custom table for source1

   right

   Custom table for source2

   outputDirectory

   path where the comparison results have to be redirected

   singleFileOutput

   a boolean variable to denote the number of output files to be one or more than one

   delimiter

   character to use as delimiter in the files

9.  def compareAppleTablesSaveResultsWithManipulation(left: AppleTable, right: AppleTable, outputDirectory: String, singleFileOutput: Boolean, delimiter: String, excludeCols: Option[Array[String]], orderByCols: Option[Array[String]], ascOrder: Boolean): Boolean
10.  def compareFileSaveResults(file1Location: String, file2Location: String, outputDirectory: String, singleFileOutput: Boolean, delimiter: String): Unit

    Compares two data sources and stores results locally

    Compares two data sources and stores results locally

    file1Location

    path to source1 data

    file2Location

    path to source2 data

    outputDirectory

    path where the comparison results have to be redirected

    singleFileOutput

    a boolean variable to denote the number of output files to be one or more than one

    delimiter

    character to use as delimiter in the files

11.  def compareFiles(file1Location: String, file2Location: String): DiffResult

    Can compare two files whether S3 , HDFS , local file system, For example, for HDFS, "hdfs://nn1home:8020/input/war-and-peace.txt" For S3 location, "s3n://myBucket/myFile1.log"

    Can compare two files whether S3 , HDFS , local file system, For example, for HDFS, "hdfs://nn1home:8020/input/war-and-peace.txt" For S3 location, "s3n://myBucket/myFile1.log"

    file1Location

    location of file1 representing Left values

    file2Location

    location of file2 representing Right values

    returns

    a Pair of RDDs the Left parameter has values in RDD1 and not in RDD2, the Right parameter has values in RDD2 but not in RDD1

12.  def compareSchemaDataFrames(left: DataFrame, right: DataFrame): DiffResult
    left

    dataframe containing source1 data

    right

    dataframe containing source2 data

    returns

    a pair of RDDs, the left parameter has values in RDD1 and not in RDD2, the right parameter has values in RDD2 but not in RDD1

13.  final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

14.  def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

15.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

16.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

17.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

18.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

19.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

20.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

21.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

22.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

23.  def toString(): String

    Definition Classes

    AnyRef → Any

24.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

25.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

26.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()