close

How to pass schema to create a new Dataframe from existing Dataframe?

Hello Guys, How are you all? Hope You all Are Fine. Today We Are Going To learn about How to pass schema to create a new Dataframe from existing Dataframe in Python. So Here I am Explain to you all the possible Methods here.

Without wasting your time, Let’s start This Article.

Table of Contents

How to pass schema to create a new Dataframe from existing Dataframe?

  1. How to pass schema to create a new Dataframe from existing Dataframe?

    You cannot apply a new schema to already created dataframe. However, you can change the schema of each column by casting to another datatype as below.

  2. pass schema to create a new Dataframe from existing Dataframe

    You cannot apply a new schema to already created dataframe. However, you can change the schema of each column by casting to another datatype as below.

Method 1

You cannot apply a new schema to already created dataframe. However, you can change the schema of each column by casting to another datatype as below.

df.withColumn("column_name", $"column_name".cast("new_datatype"))

If you need to apply a new schema, you need to convert to RDD and create a new dataframe again as below

df = sqlContext.sql("SELECT * FROM people_json")
val newDF = spark.createDataFrame(df.rdd, schema=schema)

Hope this helps!

Summery

It’s all About this issue. Hope all Methods helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which Method worked for you? Thank You.

Also, Read