close

[Solved] Keras: change learning rate

Hello Guys, How are you all? Hope You all Are Fine. Today I get the following error Keras: change learning rate in Python. So Here I am Explain to you all the possible solutions here.

Without wasting your time, Let’s start This Article to Solve This Error.

How Keras: change learning rate Error Occurs?

How To Solve Keras: change learning rate Error ?

  1. How To Solve Keras: change learning rate Error ?

    To Solve Keras: change learning rate Error optimizer = tf.keras.optimizers.Adam(0.001)
    optimizer.learning_rate.assign(0.01) print(optimizer.learning_rate)

Solution 1

You can change the learning rate as follows:

from keras import backend as K
K.set_value(model.optimizer.learning_rate, 0.001)

Included into your complete example it looks as follows:

from keras.models import Sequential
from keras.layers import Dense
from keras import backend as K
import keras
import numpy as np

model = Sequential()

model.add(Dense(1, input_shape=(10,)))

optimizer = keras.optimizers.Adam(lr=0.01)
model.compile(loss='mse', optimizer=optimizer)

print("Learning rate before first fit:", model.optimizer.learning_rate.numpy())

model.fit(np.random.randn(50,10), np.random.randn(50), epochs=50, verbose=0)

# Change learning rate to 0.001 and train for 50 more epochs
K.set_value(model.optimizer.learning_rate, 0.001)
print("Learning rate before second fit:", model.optimizer.learning_rate.numpy())

model.fit(np.random.randn(50,10), 
          np.random.randn(50), 
          initial_epoch=50, 
          epochs=50,
          verbose=0)

I’ve just tested this with keras 2.3.1. Not sure why the approach didn’t seem to work for you.

Solution 2

There is another way, you have to find the variable that holds the learning rate and assign it another value.

optimizer = tf.keras.optimizers.Adam(0.001)
optimizer.learning_rate.assign(0.01)
print(optimizer.learning_rate)

output:

<tf.Variable 'learning_rate:0' shape=() dtype=float32, numpy=0.01>

Summery

It’s all About this issue. Hope all solution helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which solution worked for you? Thank You.

Also Read

Leave a Comment