softmax及python實現過程解析
更新時間:2019年09月30日 08:42:50 作者:沙克的世界
這篇文章主要介紹了softmax及python實現過程解析,文中通過示例代碼介紹的非常詳細,對大家的學習或者工作具有一定的參考學習價值,需要的朋友可以參考下
相對于自適應神經網絡、感知器,softmax巧妙低使用簡單的方法來實現多分類問題。
- 功能上,完成從N維向量到M維向量的映射
- 輸出的結果范圍是[0, 1],對于一個sample的結果所有輸出總和等于1
- 輸出結果,可以隱含地表達該類別的概率
softmax的損失函數是采用了多分類問題中常見的交叉熵,注意經常有2個表達的形式
這兩個版本在求導過程有點不同,但是結果都是一樣的,同時損失表達的意思也是相同的,因為在第一種表達形式中,當y不是
正確分類時,y_right等于0,當y是正確分類時,y_right等于1。
下面基于mnist數據做了一個多分類的實驗,整體能達到85%的精度。
''' softmax classifier for mnist created on 2019.9.28 author: vince ''' import math import logging import numpy import random import matplotlib.pyplot as plt from tensorflow.contrib.learn.python.learn.datasets.mnist import read_data_sets from sklearn.metrics import accuracy_score def loss_max_right_class_prob(predictions, y): return -predictions[numpy.argmax(y)]; def loss_cross_entropy(predictions, y): return -numpy.dot(y, numpy.log(predictions)); ''' Softmax classifier linear classifier ''' class Softmax: def __init__(self, iter_num = 100000, batch_size = 1): self.__iter_num = iter_num; self.__batch_size = batch_size; def train(self, train_X, train_Y): X = numpy.c_[train_X, numpy.ones(train_X.shape[0])]; Y = numpy.copy(train_Y); self.L = []; #initialize parameters self.__weight = numpy.random.rand(X.shape[1], 10) * 2 - 1.0; self.__step_len = 1e-3; logging.info("weight:%s" % (self.__weight)); for iter_index in range(self.__iter_num): if iter_index % 1000 == 0: logging.info("-----iter:%s-----" % (iter_index)); if iter_index % 100 == 0: l = 0; for i in range(0, len(X), 100): predictions = self.forward_pass(X[i]); #l += loss_max_right_class_prob(predictions, Y[i]); l += loss_cross_entropy(predictions, Y[i]); l /= len(X); self.L.append(l); sample_index = random.randint(0, len(X) - 1); logging.debug("-----select sample %s-----" % (sample_index)); z = numpy.dot(X[sample_index], self.__weight); z = z - numpy.max(z); predictions = numpy.exp(z) / numpy.sum(numpy.exp(z)); dw = self.__step_len * X[sample_index].reshape(-1, 1).dot((predictions - Y[sample_index]).reshape(1, -1)); # dw = self.__step_len * X[sample_index].reshape(-1, 1).dot(predictions.reshape(1, -1)); # dw[range(X.shape[1]), numpy.argmax(Y[sample_index])] -= X[sample_index] * self.__step_len; self.__weight -= dw; logging.debug("weight:%s" % (self.__weight)); logging.debug("loss:%s" % (l)); logging.info("weight:%s" % (self.__weight)); logging.info("L:%s" % (self.L)); def forward_pass(self, x): net = numpy.dot(x, self.__weight); net = net - numpy.max(net); net = numpy.exp(net) / numpy.sum(numpy.exp(net)); return net; def predict(self, x): x = numpy.append(x, 1.0); return self.forward_pass(x); def main(): logging.basicConfig(level = logging.INFO, format = '%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s', datefmt = '%a, %d %b %Y %H:%M:%S'); logging.info("trainning begin."); mnist = read_data_sets('../data/MNIST',one_hot=True) # MNIST_data指的是存放數據的文件夾路徑,one_hot=True 為采用one_hot的編碼方式編碼標簽 #load data train_X = mnist.train.images #訓練集樣本 validation_X = mnist.validation.images #驗證集樣本 test_X = mnist.test.images #測試集樣本 #labels train_Y = mnist.train.labels #訓練集標簽 validation_Y = mnist.validation.labels #驗證集標簽 test_Y = mnist.test.labels #測試集標簽 classifier = Softmax(); classifier.train(train_X, train_Y); logging.info("trainning end. predict begin."); test_predict = numpy.array([]); test_right = numpy.array([]); for i in range(len(test_X)): predict_label = numpy.argmax(classifier.predict(test_X[i])); test_predict = numpy.append(test_predict, predict_label); right_label = numpy.argmax(test_Y[i]); test_right = numpy.append(test_right, right_label); logging.info("right:%s, predict:%s" % (test_right, test_predict)); score = accuracy_score(test_right, test_predict); logging.info("The accruacy score is: %s "% (str(score))); plt.plot(classifier.L) plt.show(); if __name__ == "__main__": main();
損失函數收斂情況
Sun, 29 Sep 2019 18:08:08 softmax.py[line:104] INFO trainning end. predict begin. Sun, 29 Sep 2019 18:08:08 softmax.py[line:114] INFO right:[7. 2. 1. ... 4. 5. 6.], predict:[7. 2. 1. ... 4. 8. 6.] Sun, 29 Sep 2019 18:08:08 softmax.py[line:116] INFO The accruacy score is: 0.8486
以上就是本文的全部內容,希望對大家的學習有所幫助,也希望大家多多支持腳本之家。
相關文章
基于python select.select模塊通信的實例講解
下面小編就為大家?guī)硪黄趐ython select.select模塊通信的實例講解。小編覺得挺不錯的,現在就分享給大家,也給大家做個參考。一起跟隨小編過來看看吧2017-09-09Pycharm2020.1安裝中文語言插件的詳細教程(不需要漢化)
這篇文章主要介紹了Pycharm2020.1安裝中文語言插件的詳細教程,不需要漢化,本文給大家分享三種方法,在這小編推薦使用方法二,具體內容詳情大家跟隨小編一起看看吧2020-08-08