Packages

  • package root

    Snowpark by itself is a powerful library, but still some utility functions can always help.

    Snowpark by itself is a powerful library, but still some utility functions can always help.

    snowpark-extensions

    Snowpark by itself is a powerful library, but still some utility functions can always help.

    The source code for this library is available here

    Installation

    With Maven you can add something like this to your POM:

    <dependency>
    <groupId>net.mobilize.snowpark-extensions</groupId>
    <artifactId>snowparkextensions</artifactId>
    <version>0.0.9</version>
</dependency>

    or with sbt use

    libraryDependencies += "net.mobilize.snowpark-extensions" % "snowparkextensions" % "0.0.16"

    Usage

    just import it at the top of your file and it will automatically extend your snowpark package.

    For example: 

    


    import com.snowflake.snowpark_extensions.Extensions._
import com.snowflake.snowpark.Session

val new_session = Session.builder.from_snowsql().appName("app1").create()

    Extensions

    See Session Extensions
    See Session Builder Extensions
    See DataFrame Extensions
    See Column Extensions
    See Function Extensions

    Definition Classes
    root
  • package com
    Definition Classes
    root
  • package snowflake
    Definition Classes
    com
  • package snowpark_extensions
    Definition Classes
    snowflake
  • package helpers
    Definition Classes
    snowpark_extensions
  • BuiltinOverrides
  • ColumnsSimplifier

class ColumnsSimplifier extends AnyRef

Column simplifier helper class. Abstracts some column operations for a dataframe. Applied automatically to a dataframe when using the DataframeExtensions implicit class.

Its purpose is to provide a helper to increase performance when performing multiple withColumn operations. This because Snowpark performs these operations immediately when executing, contrary to Spark that performs the operations when a .show(), .collect() or other functions are called.

What this class do is collect a series of withColumn operations that should be applied to a DataFrame, and groups them to execute them in bulk when the .endCols function is called.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ColumnsSimplifier
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ColumnsSimplifier(df: DataFrame)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def cacheAsParquet(str: String, str1: String): Nothing

    Dummy function.

    Dummy function. No current functionality has been implemented.

    returns

    Nothing

  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  7. def drop(colNames: String*): ColumnsSimplifier

    Adds columns to drop.

    Adds columns to drop.

    colNames

    Columns to drop.

    returns

    The current ColumnsSimplifier instance.

  8. def endCols: DataFrame

    Applies the expected operations to the DataFrame.

    Applies the expected operations to the DataFrame.

    returns

    DataFrame with the column operations applied.

  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  13. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  21. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. def withColumn(colName: String, colValue: Column): ColumnsSimplifier

    Adds a column to be added into the DataFrame.

    Adds a column to be added into the DataFrame.

    colName

    Name of the new column.

    colValue

    Column object for the new column.

    returns

    The current ColumnsSimplifier instance.

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from AnyRef

Inherited from Any

Ungrouped