softmax及python實(shí)現(xiàn)過程解析
相對(duì)于自適應(yīng)神經(jīng)網(wǎng)絡(luò)、感知器,softmax巧妙低使用簡(jiǎn)單的方法來實(shí)現(xiàn)多分類問題。
- 功能上,完成從N維向量到M維向量的映射
- 輸出的結(jié)果范圍是[0, 1],對(duì)于一個(gè)sample的結(jié)果所有輸出總和等于1
- 輸出結(jié)果,可以隱含地表達(dá)該類別的概率
softmax的損失函數(shù)是采用了多分類問題中常見的交叉熵,注意經(jīng)常有2個(gè)表達(dá)的形式
這兩個(gè)版本在求導(dǎo)過程有點(diǎn)不同,但是結(jié)果都是一樣的,同時(shí)損失表達(dá)的意思也是相同的,因?yàn)樵诘谝环N表達(dá)形式中,當(dāng)y不是
正確分類時(shí),y_right等于0,當(dāng)y是正確分類時(shí),y_right等于1。
下面基于mnist數(shù)據(jù)做了一個(gè)多分類的實(shí)驗(yàn),整體能達(dá)到85%的精度。
''' softmax classifier for mnist created on 2019.9.28 author: vince ''' import math import logging import numpy import random import matplotlib.pyplot as plt from tensorflow.contrib.learn.python.learn.datasets.mnist import read_data_sets from sklearn.metrics import accuracy_score def loss_max_right_class_prob(predictions, y): return -predictions[numpy.argmax(y)]; def loss_cross_entropy(predictions, y): return -numpy.dot(y, numpy.log(predictions)); ''' Softmax classifier linear classifier ''' class Softmax: def __init__(self, iter_num = 100000, batch_size = 1): self.__iter_num = iter_num; self.__batch_size = batch_size; def train(self, train_X, train_Y): X = numpy.c_[train_X, numpy.ones(train_X.shape[0])]; Y = numpy.copy(train_Y); self.L = []; #initialize parameters self.__weight = numpy.random.rand(X.shape[1], 10) * 2 - 1.0; self.__step_len = 1e-3; logging.info("weight:%s" % (self.__weight)); for iter_index in range(self.__iter_num): if iter_index % 1000 == 0: logging.info("-----iter:%s-----" % (iter_index)); if iter_index % 100 == 0: l = 0; for i in range(0, len(X), 100): predictions = self.forward_pass(X[i]); #l += loss_max_right_class_prob(predictions, Y[i]); l += loss_cross_entropy(predictions, Y[i]); l /= len(X); self.L.append(l); sample_index = random.randint(0, len(X) - 1); logging.debug("-----select sample %s-----" % (sample_index)); z = numpy.dot(X[sample_index], self.__weight); z = z - numpy.max(z); predictions = numpy.exp(z) / numpy.sum(numpy.exp(z)); dw = self.__step_len * X[sample_index].reshape(-1, 1).dot((predictions - Y[sample_index]).reshape(1, -1)); # dw = self.__step_len * X[sample_index].reshape(-1, 1).dot(predictions.reshape(1, -1)); # dw[range(X.shape[1]), numpy.argmax(Y[sample_index])] -= X[sample_index] * self.__step_len; self.__weight -= dw; logging.debug("weight:%s" % (self.__weight)); logging.debug("loss:%s" % (l)); logging.info("weight:%s" % (self.__weight)); logging.info("L:%s" % (self.L)); def forward_pass(self, x): net = numpy.dot(x, self.__weight); net = net - numpy.max(net); net = numpy.exp(net) / numpy.sum(numpy.exp(net)); return net; def predict(self, x): x = numpy.append(x, 1.0); return self.forward_pass(x); def main(): logging.basicConfig(level = logging.INFO, format = '%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s', datefmt = '%a, %d %b %Y %H:%M:%S'); logging.info("trainning begin."); mnist = read_data_sets('../data/MNIST',one_hot=True) # MNIST_data指的是存放數(shù)據(jù)的文件夾路徑,one_hot=True 為采用one_hot的編碼方式編碼標(biāo)簽 #load data train_X = mnist.train.images #訓(xùn)練集樣本 validation_X = mnist.validation.images #驗(yàn)證集樣本 test_X = mnist.test.images #測(cè)試集樣本 #labels train_Y = mnist.train.labels #訓(xùn)練集標(biāo)簽 validation_Y = mnist.validation.labels #驗(yàn)證集標(biāo)簽 test_Y = mnist.test.labels #測(cè)試集標(biāo)簽 classifier = Softmax(); classifier.train(train_X, train_Y); logging.info("trainning end. predict begin."); test_predict = numpy.array([]); test_right = numpy.array([]); for i in range(len(test_X)): predict_label = numpy.argmax(classifier.predict(test_X[i])); test_predict = numpy.append(test_predict, predict_label); right_label = numpy.argmax(test_Y[i]); test_right = numpy.append(test_right, right_label); logging.info("right:%s, predict:%s" % (test_right, test_predict)); score = accuracy_score(test_right, test_predict); logging.info("The accruacy score is: %s "% (str(score))); plt.plot(classifier.L) plt.show(); if __name__ == "__main__": main();
損失函數(shù)收斂情況
Sun, 29 Sep 2019 18:08:08 softmax.py[line:104] INFO trainning end. predict begin. Sun, 29 Sep 2019 18:08:08 softmax.py[line:114] INFO right:[7. 2. 1. ... 4. 5. 6.], predict:[7. 2. 1. ... 4. 8. 6.] Sun, 29 Sep 2019 18:08:08 softmax.py[line:116] INFO The accruacy score is: 0.8486
以上就是本文的全部?jī)?nèi)容,希望對(duì)大家的學(xué)習(xí)有所幫助,也希望大家多多支持腳本之家。
相關(guān)文章
Python實(shí)現(xiàn)的微信好友數(shù)據(jù)分析功能示例
這篇文章主要介紹了Python實(shí)現(xiàn)的微信好友數(shù)據(jù)分析功能,結(jié)合實(shí)例形式分析了Python使用itchat、pandas、pyecharts等模塊針對(duì)微信好友數(shù)據(jù)進(jìn)行統(tǒng)計(jì)與計(jì)算相關(guān)操作技巧,需要的朋友可以參考下2018-06-06時(shí)間序列預(yù)測(cè)中的數(shù)據(jù)滑窗操作實(shí)例(python實(shí)現(xiàn))
滑動(dòng)窗口操作非常普遍,非常有用,它們也很容易在Python中實(shí)現(xiàn),下面這篇文章主要給大家介紹了關(guān)于時(shí)間序列預(yù)測(cè)中的數(shù)據(jù)滑窗操作python實(shí)現(xiàn)的相關(guān)資料,需要的朋友可以參考下2022-03-03基于python select.select模塊通信的實(shí)例講解
下面小編就為大家?guī)硪黄趐ython select.select模塊通信的實(shí)例講解。小編覺得挺不錯(cuò)的,現(xiàn)在就分享給大家,也給大家做個(gè)參考。一起跟隨小編過來看看吧2017-09-09Pycharm2020.1安裝中文語言插件的詳細(xì)教程(不需要漢化)
這篇文章主要介紹了Pycharm2020.1安裝中文語言插件的詳細(xì)教程,不需要漢化,本文給大家分享三種方法,在這小編推薦使用方法二,具體內(nèi)容詳情大家跟隨小編一起看看吧2020-08-08Python實(shí)現(xiàn)隨機(jī)從圖像中獲取多個(gè)patch
經(jīng)常有一些圖像任務(wù)需要從一張大圖中截取固定大小的patch來進(jìn)行訓(xùn)練。本文就來和大家聊聊如何用Python實(shí)現(xiàn)隨機(jī)從圖像中獲取多個(gè)patch,感興趣的可以了解一下2022-08-08