close

How do I use multiple conditions with pyspark.sql.functions.when()?

Hello Guys, How are you all? Hope You all Are Fine. Today We Are Going To learn about How do I use multiple conditions with pyspark.sql.functions.when() in Python. So Here I am Explain to you all the possible Methods here.

Without wasting your time, Let’s start This Article.

Table of Contents

How do I use multiple conditions with pyspark.sql.functions.when()?

  1. How do I use multiple conditions with pyspark.sql.functions.when()?

    Use parentheses to enforce the desired operator precedence:
    F.when( (df["col-1"]>0.0) & (df["col-2"]>0.0), 1).otherwise(0)

  2. use multiple conditions with pyspark.sql.functions.when()

    Use parentheses to enforce the desired operator precedence:
    F.when( (df["col-1"]>0.0) & (df["col-2"]>0.0), 1).otherwise(0)

Method 1

Use parentheses to enforce the desired operator precedence:

F.when( (df["col-1"]>0.0) & (df["col-2"]>0.0), 1).otherwise(0)

Method 2

when in pyspark multiple conditions can be built using &(for and) and | (for or), it is important to enclose every expressions within parenthesis that combine to form the condition

%pyspark
dataDF = spark.createDataFrame([(66, "a", "4"), 
                                (67, "a", "0"), 
                                (70, "b", "4"), 
                                (71, "d", "4")],
                                ("id", "code", "amt"))
dataDF.withColumn("new_column",
       when((col("code") == "a") | (col("code") == "d"), "A")
      .when((col("code") == "b") & (col("amt") == "4"), "B")
      .otherwise("A1")).show()

when in spark scala can be used with && and || operator to build multiple conditions

//Scala
val dataDF = Seq(
          (66, "a", "4"), (67, "a", "0"), (70, "b", "4"), (71, "d", "4"
          )).toDF("id", "code", "amt")
    dataDF.withColumn("new_column",
           when(col("code") === "a" || col("code") === "d", "A")
          .when(col("code") === "b" && col("amt") === "4", "B")
          .otherwise("A1"))
          .show()

Output:

+---+----+---+----------+
| id|code|amt|new_column|
+---+----+---+----------+
| 66|   a|  4|         A|
| 67|   a|  0|         A|
| 70|   b|  4|         B|
| 71|   d|  4|         A|
+---+----+---+----------+

Conclusion

It’s all About this issue. Hope all Methods helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which Method worked for you? Thank You.

Also, Read