close

How to increase Jupyter notebook Memory limit?

Hello Guys, How are you all? Hope You all Are Fine. Today We Are Going To learn about How to increase Jupyter notebook Memory limit in Python. So Here I am Explain to you all the possible Methods here.

Without wasting your time, Let’s start This Article.

Table of Contents

How to increase Jupyter notebook Memory limit?

  1. How to increase Jupyter notebook Memory limit?

    max_buffer_size is a Tornado Web Server setting, corresponds to the Maximum amount of incoming data to buffer and defaults to 100MB (104857600).

  2. increase Jupyter notebook Memory limit

    max_buffer_size is a Tornado Web Server setting, corresponds to the Maximum amount of incoming data to buffer and defaults to 100MB (104857600).

Method 1

Jupyter notebook has a default memory limit size. You can try to increase the memory limit by following the steps:
1) Generate Config file using command:

jupyter notebook --generate-config

2) Open jupyter_notebook_config.py file situated inside ‘jupyter’ folder and edit the following property:

NotebookApp.max_buffer_size = your desired value

Remember to remove the ‘#’ before the property value.
3) Save and run the jupyter notebook. It should now utilize the set memory value. Also, don’t forget to run the notebook from inside the jupyter folder.


Alternatively, you can simply run the Notebook using below command:

 jupyter notebook --NotebookApp.max_buffer_size=your_value

Method 2

For Jupyter you need to consider 2 processes:

  1. The local HTTP server (which is based on Tornado)
  2. Kernel process (normally local but can be distributed and depends on your config).

max_buffer_size is a Tornado Web Server setting, corresponds to the Maximum amount of incoming data to buffer and defaults to 100MB (104857600).

Based on this PR, this value seems to have been increased to 500 MB in Notebook.

Tornado HTTP server does not allow to my knowledge to define the max memory, it runs as a Python3 process.

For the kernel, you should look at the command defined kernel spec.

Summery

It’s all About this issue. Hope all Methods helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which Method worked for you? Thank You.

Also, Read