close

[Solved] PySpark Error When running SQL Query

Hello Guys, How are you all? Hope You all Are Fine. Today I get the following error PySpark Error When running SQL Query in python. So Here I am Explain to you all the possible solutions here.

Without wasting your time, Let’s start This Article to Solve This Error.

How PySpark Error When running SQL Query Error Occurs?

Today I get the following error PySpark Error When running SQL Query in python.

How To Solve PySpark Error When running SQL Query Error ?

  1. How To Solve PySpark Error When running SQL Query Error ?

    To Solve PySpark Error When running SQL Query Error Bizarrely, this was thrown for me if I terminated my SQL statement with ;. Simply removing that fixed it.

  2. PySpark Error When running SQL Query

    To Solve PySpark Error When running SQL Query Error Bizarrely, this was thrown for me if I terminated my SQL statement with ;. Simply removing that fixed it.

Solution 1

Because you are using \ in the first one and that’s being passed as odd syntax to spark. If you want to write multi-line SQL statements, use triple quotes:

results5 = spark.sql("""SELECT
  appl_stock.Open
 ,appl_stock.Close
FROM appl_stock
WHERE appl_stock.Close < 500""")

Solution 2

Bizarrely, this was thrown for me if I terminated my SQL statement with ;. Simply removing that fixed it.

i.e.

spark.sql("""
SELECT * FROM foo WHERE bar = 3;
""")

breaks.

spark.sql("""
SELECT * FROM foo WHERE bar = 3
""")

does not.

Summery

It’s all About this issue. Hope all solution helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which solution worked for you? Thank You.

Also, Read