Pytorch對Himmelblau函數(shù)的優(yōu)化詳解
Himmelblau函數(shù)如下:
有四個全局最小解,且值都為0,這個函數(shù)常用來檢驗優(yōu)化算法的表現(xiàn)如何:
可視化函數(shù)圖像:
import numpy as np from matplotlib import pyplot as plt from mpl_toolkits.mplot3d import Axes3D def himmelblau(x): return (x[0] ** 2 + x[1] - 11) ** 2 + (x[0] + x[1] ** 2 - 7) ** 2 x = np.arange(-6, 6, 0.1) y = np.arange(-6, 6, 0.1) X, Y = np.meshgrid(x, y) Z = himmelblau([X, Y]) fig = plt.figure("himmeblau") ax = fig.gca(projection='3d') ax.plot_surface(X, Y, Z) ax.view_init(60, -30) ax.set_xlabel('x') ax.set_ylabel('y') plt.show()
結(jié)果:
使用隨機梯度下降優(yōu)化:
import torch def himmelblau(x): return (x[0] ** 2 + x[1] - 11) ** 2 + (x[0] + x[1] ** 2 - 7) ** 2 # 初始設(shè)置為0,0. x = torch.tensor([0., 0.], requires_grad=True) # 優(yōu)化目標(biāo)是找到使himmelblau函數(shù)值最小的坐標(biāo)x[0],x[1], # 也就是x, y # 這里是定義Adam優(yōu)化器,指明優(yōu)化目標(biāo)是x,學(xué)習(xí)率是1e-3 optimizer = torch.optim.Adam([x], lr=1e-3) for step in range(20000): # 每次計算出當(dāng)前的函數(shù)值 pred = himmelblau(x) # 當(dāng)網(wǎng)絡(luò)參量進行反饋時,梯度是被積累的而不是被替換掉,這里即每次將梯度設(shè)置為0 optimizer.zero_grad() # 生成當(dāng)前所在點函數(shù)值相關(guān)的梯度信息,這里即優(yōu)化目標(biāo)的梯度信息 pred.backward() # 使用梯度信息更新優(yōu)化目標(biāo)的值,即更新x[0]和x[1] optimizer.step() # 每2000次輸出一下當(dāng)前情況 if step % 2000 == 0: print("step={},x={},f(x)={}".format(step, x.tolist(), pred.item()))
輸出結(jié)果:
step=0,x=[0.0009999999310821295, 0.0009999999310821295],f(x)=170.0 step=2000,x=[2.3331806659698486, 1.9540692567825317],f(x)=13.730920791625977 step=4000,x=[2.9820079803466797, 2.0270984172821045],f(x)=0.014858869835734367 step=6000,x=[2.999983549118042, 2.0000221729278564],f(x)=1.1074007488787174e-08 step=8000,x=[2.9999938011169434, 2.0000083446502686],f(x)=1.5572823031106964e-09 step=10000,x=[2.999997854232788, 2.000002861022949],f(x)=1.8189894035458565e-10 step=12000,x=[2.9999992847442627, 2.0000009536743164],f(x)=1.6370904631912708e-11 step=14000,x=[2.999999761581421, 2.000000238418579],f(x)=1.8189894035458565e-12 step=16000,x=[3.0, 2.0],f(x)=0.0 step=18000,x=[3.0, 2.0],f(x)=0.0
從上面結(jié)果看,找到了一組最優(yōu)解[3.0, 2.0],此時極小值為0.0。如果修改Tensor變量x的初始化值,可能會找到其它的極小值,也就是說初始化值對于找到最優(yōu)解很關(guān)鍵。
補充拓展:pytorch 搭建自己的神經(jīng)網(wǎng)絡(luò)和各種優(yōu)化器
還是直接看代碼吧!
import torch import torchvision import torchvision.transforms as transform import torch.utils.data as Data import matplotlib.pyplot as plt from torch.utils.data import Dataset,DataLoader import pandas as pd import numpy as np from torch.autograd import Variable # data set train=pd.read_csv('Thirdtest.csv') #cut 0 col as label train_label=train.iloc[:,[0]] #只讀取一列 #train_label=train.iloc[:,0:3] #cut 1~16 col as data train_data=train.iloc[:,1:] #change to np train_label_np=train_label.values train_data_np=train_data.values #change to tensor train_label_ts=torch.from_numpy(train_label_np) train_data_ts=torch.from_numpy(train_data_np) train_label_ts=train_label_ts.type(torch.LongTensor) train_data_ts=train_data_ts.type(torch.FloatTensor) print(train_label_ts.shape) print(type(train_label_ts)) train_dataset=Data.TensorDataset(train_data_ts,train_label_ts) train_loader=DataLoader(dataset=train_dataset,batch_size=64,shuffle=True) #make a network import torch.nn.functional as F # 激勵函數(shù)都在這 class Net(torch.nn.Module): # 繼承 torch 的 Module def __init__(self ): super(Net, self).__init__() # 繼承 __init__ 功能 self.hidden1 = torch.nn.Linear(16, 30)# 隱藏層線性輸出 self.out = torch.nn.Linear(30, 3) # 輸出層線性輸出 def forward(self, x): # 正向傳播輸入值, 神經(jīng)網(wǎng)絡(luò)分析出輸出值 x = F.relu(self.hidden1(x)) # 激勵函數(shù)(隱藏層的線性值) x = self.out(x) # 輸出值, 但是這個不是預(yù)測值, 預(yù)測值還需要再另外計算 return x # net=Net() # optimizer = torch.optim.SGD(net.parameters(), lr=0.0001,momentum=0.001) # loss_func = torch.nn.CrossEntropyLoss() # the target label is NOT an one-hotted # loss_list=[] # for epoch in range(500): # for step ,(b_x,b_y) in enumerate (train_loader): # b_x,b_y=Variable(b_x),Variable(b_y) # b_y=b_y.squeeze(1) # output=net(b_x) # loss=loss_func(output,b_y) # optimizer.zero_grad() # loss.backward() # optimizer.step() # if epoch%1==0: # loss_list.append(float(loss)) # print( "Epoch: ", epoch, "Step ", step, "loss: ", float(loss)) # 為每個優(yōu)化器創(chuàng)建一個 net net_SGD = Net() net_Momentum = Net() net_RMSprop = Net() net_Adam = Net() nets = [net_SGD, net_Momentum, net_RMSprop, net_Adam] #定義優(yōu)化器 LR=0.0001 opt_SGD = torch.optim.SGD(net_SGD.parameters(), lr=LR,momentum=0.001) opt_Momentum = torch.optim.SGD(net_Momentum.parameters(), lr=LR, momentum=0.8) opt_RMSprop = torch.optim.RMSprop(net_RMSprop.parameters(), lr=LR, alpha=0.9) opt_Adam = torch.optim.Adam(net_Adam.parameters(), lr=LR, betas=(0.9, 0.99)) optimizers = [opt_SGD, opt_Momentum, opt_RMSprop, opt_Adam] loss_func = torch.nn.CrossEntropyLoss() losses_his = [[], [], [], []] for net, opt, l_his in zip(nets, optimizers, losses_his): for epoch in range(500): for step, (b_x, b_y) in enumerate(train_loader): b_x, b_y = Variable(b_x), Variable(b_y) b_y = b_y.squeeze(1)# 數(shù)據(jù)必須得是一維非one-hot向量 # 對每個優(yōu)化器, 優(yōu)化屬于他的神經(jīng)網(wǎng)絡(luò) output = net(b_x) # get output for every net loss = loss_func(output, b_y) # compute loss for every net opt.zero_grad() # clear gradients for next train loss.backward() # backpropagation, compute gradients opt.step() # apply gradients if epoch%1==0: l_his.append(loss.data.numpy()) # loss recoder print("optimizers: ",opt,"Epoch: ",epoch,"Step ",step,"loss: ",float(loss)) labels = ['SGD', 'Momentum', 'RMSprop', 'Adam'] for i, l_his in enumerate(losses_his): plt.plot(l_his, label=labels[i]) plt.legend(loc='best') plt.xlabel('Steps') plt.ylabel('Loss') plt.xlim((0,1000)) plt.ylim((0,4)) plt.show() # # for epoch in range(5): # for step ,(b_x,b_y) in enumerate (train_loader): # b_x,b_y=Variable(b_x),Variable(b_y) # b_y=b_y.squeeze(1) # output=net(b_x) # loss=loss_func(output,b_y) # loss.backward() # optimizer.zero_grad() # optimizer.step() # print(loss)
以上這篇Pytorch對Himmelblau函數(shù)的優(yōu)化詳解就是小編分享給大家的全部內(nèi)容了,希望能給大家一個參考,也希望大家多多支持腳本之家。
相關(guān)文章
Python中asyncore異步模塊的用法及實現(xiàn)httpclient的實例
asyncore即是一個異步的socket封裝,特別是dispatcher類中包含了很多異步調(diào)用的socket操作方法,非常犀利,下面我們就來講解Python中asyncore異步模塊的用法及實現(xiàn)httpclient的實例2016-06-06Django配置celery(非djcelery)執(zhí)行異步任務(wù)和定時任務(wù)
這篇文章主要介紹了Django配置celery(非djcelery)執(zhí)行異步任務(wù)和定時任務(wù),小編覺得挺不錯的,現(xiàn)在分享給大家,也給大家做個參考。一起跟隨小編過來看看吧2018-07-07python應(yīng)用之如何使用Python發(fā)送通知到微信
現(xiàn)在通過發(fā)微信信息來做消息通知和告警已經(jīng)很普遍了,下面這篇文章主要給大家介紹了關(guān)于python應(yīng)用之如何使用Python發(fā)送通知到微信的相關(guān)資料,文中通過實例代碼介紹的非常詳細(xì),需要的朋友可以參考下2022-03-03升級keras解決load_weights()中的未定義skip_mismatch關(guān)鍵字問題
這篇文章主要介紹了升級keras解決load_weights()中的未定義skip_mismatch關(guān)鍵字問題,具有很好的參考價值,希望對大家有所幫助。一起跟隨小編過來看看吧2020-06-06解決Numpy報錯:ImportError: numpy.core.multiarray faile
這篇文章主要介紹了解決Numpy報錯:ImportError: numpy.core.multiarray failed問題,具有很好的參考價值,希望對大家有所幫助,如有錯誤或未考慮完全的地方,望不吝賜教2024-01-01Python實現(xiàn)xml格式轉(zhuǎn)txt格式的示例代碼
VOC 的標(biāo)注是xml格式的,而YOLO是.txt格式,所以要實現(xiàn)VOC數(shù)據(jù)集轉(zhuǎn)YOLO數(shù)據(jù)集,只能利用代碼實現(xiàn)。所以本文為大家介紹了Python中xml轉(zhuǎn)txt的示例代碼,需要的可以參考一下2022-03-03Python多線程與多進程相關(guān)知識總結(jié)
進程(process)和線程(thread)是操作系統(tǒng)的基本概念,是操作系統(tǒng)程序運行的基本單元,本文簡要介紹進程和線程的概念以及Python中的多進程和多線程.需要的朋友可以參考下2021-05-05