close

[Solved] in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.]

Hello Guys, How are you all? Hope You all Are Fine. Today I get the following error in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] in python. So Here I am Explain to you all the possible solutions here.

Without wasting your time, Let’s start This Article to Solve This Error.

How in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] Error Occurs?

Today I get the following error in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] in python.

How To Solve in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] Error ?

  1. How To Solve in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] Error ?

    To Solve in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] Error As it breaks when you read it in csv module, you can at least locate the line where the error occurs:

  2. in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.]

    To Solve in Reading a csv file in pandas[CParser: tokenizing data. C : Buffer overflow caught – possible malformed input file.] Error As it breaks when you read it in csv module, you can at least locate the line where the error occurs:

Solution 1

Not an answer, but too long for a comment (not speaking of code formatting)

As it breaks when you read it in csv module, you can at least locate the line where the error occurs:

import csv
with open(r"C:\work\DATA\Raw_data\store.csv", 'rb') as f:
    reader = csv.reader(f)
    linenumber = 1
    try:
        for row in reader:
            linenumber += 1
    except Exception as e:
        print (("Error line %d: %s %s" % (linenumber, str(type(e)), e.message)))

Then look in store.csv what happens at that line.

Solution 2

I found this error, the cause was that there were some carriage returns “\r” in the data that pandas was using as a line terminator as if it was “\n”. I thought I’d post here as that might be a common reason this error might come up.

The solution I found was to add lineterminator=’\n’ into the read_csv function like this:

df_clean = pd.read_csv('test_error.csv',
                 lineterminator='\n')

Summery

It’s all About this issue. Hope all solution helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which solution worked for you? Thank You.

Also, Read