C++實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)框架SimpleNN的詳細(xì)過(guò)程

更新時(shí)間：2021年08月25日 10:40:40 作者：Peiy_Liu

本來(lái)自己想到用C++實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)主要是想強(qiáng)化一下編碼能力并入門(mén)深度學(xué)習(xí)，對(duì)C++實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)框架SimpleNN的詳細(xì)過(guò)程感興趣的朋友一起看看吧

Features

Construct neural networks.
Configure optimizer and loss layer for the network to train models and use models to do prediction.Save models.
Network architecture will be saved as a .netjson file,while weights will be saved as a .weightsbinary file.
Load models.Load network from an existing .netfile and load weights from an existing .weights.Load dataset (including data and labels) from a .csvfile.It is neccesary to preprocess the dataset（mnist etc.) into SimpleNN stype 2D matrix and save it into a .csvfile.All data in SimpleNN will be organized into columns and conbined into a 2D matrix.
For example,mostly a batch of C C C channels H H Hx W W W images with batch size N N N will be flatten into columns by channel and organized into an ( H ∗ W ) (H*W) (H∗W)x ( C ∗ N ) (C*N) (C∗N) matrix.構(gòu)建自定義網(wǎng)絡(luò)。
為網(wǎng)絡(luò)對(duì)象配置優(yōu)化器和損失函數(shù)層來(lái)訓(xùn)練模型，并用模型作預(yù)測(cè)。
保存模型。網(wǎng)絡(luò)結(jié)構(gòu)用json格式描述，擴(kuò)展名為 .net；權(quán)重存為二進(jìn)制文件，擴(kuò)展名為.weights。
加載模型。從已有的.net文件中加載網(wǎng)絡(luò)，從已有的.weights文件中加載權(quán)重。
從.csv文件中加載數(shù)據(jù)集。在此之前需要對(duì)原始數(shù)據(jù)集(如mnist等）進(jìn)行預(yù)處理組織為一個(gè)二維矩陣。
在SimpleNN中流動(dòng)的所有數(shù)據(jù)都是組織成一列列的并組合成一個(gè)二維矩陣。

例如，大多數(shù)情況下一批batch size為 N N N 的 C C C 通道 H H Hx W W W 圖像會(huì)按通道展開(kāi)成列并組織為一個(gè) ( H ∗ W ) (H*W) (H∗W)x ( C ∗ N ) (C*N) (C∗N)的矩陣。

Dependencies

The core of SimpleNN is completely written with C++11 STL.So to build SimpleNN it just need a C++ compiler surppoting C++11 stantard.

P.S.:Some examples in examplesfolder needs 3rd-party libraries like OpenCV3.So if you want to build them as well you may install the needed libraries first.

Platform

Any os with C++11 compiler.

To Do

豐富layers和nets。
實(shí)現(xiàn)AutoGradient，使之可基于計(jì)算圖構(gòu)造網(wǎng)絡(luò)。
利用并行計(jì)算實(shí)現(xiàn)矩陣運(yùn)算等過(guò)程的加速優(yōu)化（多線程、GPU），目前所有矩陣運(yùn)算都是用for循環(huán)硬堆的，毫無(wú)性能可言。。。
利用自己造的這個(gè)輪子復(fù)現(xiàn)更多的神經(jīng)網(wǎng)絡(luò)模型。
為什么用二維矩陣來(lái)存儲(chǔ)數(shù)據(jù)呢主要是因?yàn)橐婚_(kāi)始只是寫(xiě)了一個(gè)二維矩陣運(yùn)算模板類，然后就想直接用這個(gè)類實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)。一般情況下這種數(shù)據(jù)處理方法應(yīng)該是夠用的，后面看如果有必要的話再實(shí)現(xiàn)一個(gè)四維的Tensor類。

本來(lái)自己想到用C++實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)主要是想強(qiáng)化一下編碼能力并入門(mén)深度學(xué)習(xí)，所以我會(huì)盡力親自從頭實(shí)現(xiàn)以上功能，歡迎各位大佬們批評(píng)指點(diǎn)！

Usage

1.Build

git clone 
cd SimpleNN
mkdir build
cd build
cmake ..
make

2.Run examples(Linux)

examples都在examples目錄下，以例子recognition為例。本例是利用圖像分割和LeNet進(jìn)行數(shù)字識(shí)別。

若目標(biāo)數(shù)字是黑底白字，則在終端輸入（假設(shè)終端在SimpleNN根目錄下打開(kāi)）

examples/mnist/recognition <image_path>

效果：

在這里插入圖片描述

若目標(biāo)數(shù)字是黑底白字，則輸入

examples/mnist/recognition <image_path> --reverse

在mnist目錄下已有訓(xùn)練好的LeNet權(quán)重參數(shù)。若要運(yùn)行examples/mnist/train，需要先在examples/mnist/dataset目錄下運(yùn)行generate_csv.py來(lái)生成數(shù)據(jù)集的csv文件（這個(gè)文件有400多M屬于大文件試了好多種都push不上來(lái)QAQ）。

注：本例依賴OpenCV3，如果要運(yùn)行須事先安裝，不然不會(huì)編譯本例。

3.Coding

Construct network

int input_img_rows1 = 28;
                int input_img_cols1 = 28;
                int input_img_channels1 = 1;

                int conv_output_img_channels1 = 6;
                int conv_filter_rows1 = 5;
                int conv_filter_cols1 = 5;
                int conv_row_pads1 = 0;
                int conv_col_pads1 = 0;
                int conv_row_strides1 = 1;
                int conv_col_strides1 = 1;

                std::shared_ptr<snn::Convolution> conv_layer1(new snn::Convolution(input_img_rows1, input_img_cols1, 
                                                            input_img_channels1, 
                                                            conv_output_img_channels1, 
                                                            conv_filter_rows1, conv_filter_cols1, 
                                                            conv_row_pads1, conv_col_pads1, 
                                                            conv_row_strides1, conv_col_strides1, 
                                                            0, 0.283, 
                                                            0, 0.01));

                int pool_input_img_rows1 = conv_layer1->output_img_rows;
                int pool_input_img_cols1 = conv_layer1->output_img_cols;
                int pool_filter_rows1 = 2;
                int pool_filter_cols1 = 2;
                int pool_pads1 = 0;
                int pool_strides1 = 2;

                std::shared_ptr<snn::MaxPooling> pool_layer1(new snn::MaxPooling(pool_input_img_rows1, pool_input_img_cols1, 
                                                        pool_filter_rows1, pool_filter_cols1, 
                                                        pool_pads1, pool_pads1, 
                                                        pool_strides1, pool_strides1, 
                                                        conv_output_img_channels1, false));

                int input_img_rows2 = pool_layer1->output_img_rows;
                int input_img_cols2 = pool_layer1->output_img_rows;
                int input_img_channels2 = pool_layer1->image_channels;

                int conv_output_img_channels2 = 16;
                int conv_filter_rows2 = 5;
                int conv_filter_cols2 = 5;
                int conv_row_pads2 = 0;
                int conv_col_pads2 = 0;
                int conv_row_strides2 = 1;
                int conv_col_strides2 = 1;

                std::shared_ptr<snn::Convolution> conv_layer2(new snn::Convolution(input_img_rows2, input_img_cols2, 
                                                            input_img_channels2, 
                                                            conv_output_img_channels2, 
                                                            conv_filter_rows2, conv_filter_cols2, 
                                                            conv_row_pads2, conv_col_pads2, 
                                                            conv_row_strides2, conv_col_strides2, 
                                                            0, 0.115, 
                                                            0, 0.01));

                int pool_input_img_rows2 = conv_layer2->output_img_rows;
                int pool_input_img_cols2 = conv_layer2->output_img_cols;
                int pool_filter_rows2 = 2;
                int pool_filter_cols2 = 2;
                int pool_pads2 = 0;
                int pool_strides2 = 2;

                std::shared_ptr<snn::MaxPooling> pool_layer2(new snn::MaxPooling(pool_input_img_rows2, pool_input_img_cols2, 
                                                        pool_filter_rows2, pool_filter_cols2, 
                                                        pool_pads2, pool_pads2, 
                                                        pool_strides2, pool_strides2, 
                                                        conv_output_img_channels2, true));

                int aff1_input_rows = pool_layer2->output_rows * conv_output_img_channels2; // because flatten-flag is true
                int aff1_input_cols = 1;
                int aff1_output_rows = 120;
                int aff1_output_cols = 1;

                std::shared_ptr<snn::Affine> aff1_layer(new snn::Affine(aff1_input_rows, aff1_input_cols, 
                                                aff1_output_rows, aff1_output_cols, 0, 2.0 / double(aff1_input_rows), 
                                                                                    0, 0.01));

                int aff2_input_rows = 120;
                int aff2_input_cols = 1;
                int aff2_output_rows = 84;
                int aff2_output_cols = 1;

                std::shared_ptr<snn::Affine> aff2_layer(new snn::Affine(aff2_input_rows, aff2_input_cols, 
                                                aff2_output_rows, aff2_output_cols, 0, 2.0 / 120.0, 0, 0.01));

                int aff3_input_rows = 84;
                int aff3_input_cols = 1;
                int aff3_output_rows = 10;
                int aff3_output_cols = 1;

                std::shared_ptr<snn::Affine> aff3_layer(new snn::Affine(aff3_input_rows, aff3_input_cols, 
                                                aff3_output_rows, aff3_output_cols, 0, 2.0 / 84.0, 0, 0.01));

                std::shared_ptr<snn::Relu> relu_layer1(new snn::Relu);
                std::shared_ptr<snn::Relu> relu_layer2(new snn::Relu);
                std::shared_ptr<snn::Relu> relu_layer3(new snn::Relu);
                std::shared_ptr<snn::Relu> relu_layer4(new snn::Relu);
                //std::shared_ptr<Softmax> softmax_layer(new Softmax);
				
				snn::Sequential net;
                net << conv_layer1 << relu_layer1 << pool_layer1
                    << conv_layer2 << relu_layer2 << pool_layer2
                    << aff1_layer << relu_layer3
                    << aff2_layer << relu_layer4
                    <<aff3_layer;

也可以直接封裝成一個(gè)類，參考models目錄下各hpp文件：

#include <../include/SimpleNN.hpp>

namespace snn
{
    // Simplified LeNet-5 model
    class LeNet : public Sequential
    {
        public:
            LeNet():Sequential()
            {
               /* ... */

                *this << conv_layer1 << relu_layer1 << pool_layer1
                    << conv_layer2 << relu_layer2 << pool_layer2
                    << aff1_layer << relu_layer3
                    << aff2_layer << relu_layer4
                    <<aff3_layer;
            }
    };
}

Train model

配置優(yōu)化器和loss層：

std::shared_ptr<SoftmaxWithLoss> loss_layer(new SoftmaxWithLoss(true));
net.set_loss_layer(loss_layer);
std::cout << "Loss layer ready!" << std::endl;

std::vector<Matrix_d> init_params = net.get_params();
std::vector<Matrix_d> init_grads = net.get_grads();
 std::shared_ptr<AdaGrad> opt(new AdaGrad(init_params, init_grads, 0.012));
 net.set_optimizer(opt);

加載數(shù)據(jù)

Dataset train_set(true);
Dataset test_set(true);
    
 if (train_set.load_data(train_data_file_path, train_label_file_path))
     std::cout << "Train set loading finished!" << std::endl;
else
     std::cout << "Failed to load train set data!" << std::endl;

if (test_set.load_data(test_data_file_path, test_label_file_path))
     std::cout << "Test set loading finished!" << std::endl;
else
     std::cout << "Failed to load test set data!" << std::endl;

訓(xùn)練并保存模型

net.fit(train_set, test_set, 256, 2);

if (!net.save_net("../../../examples/mnist/LeNet.net"))
{
     std::cout << "Failed to save net!" << std::endl;
     return 0;
}
if (!net.save_weights("../../../examples/mnist/LeNet.weights"))
{
     std::cout << "Failed to save weights!" << std::endl;
     return 0;
}

Load model

if (!net.load_net(net_path))
{
     std::cerr << "Failed to load net!" << std::endl;
     return -1;
    
}
if (!net.load_weights(weight_path))
{
     std::cerr << "Failed to load weights!" << std::endl;
     return -1;
    
}

或者直接

if (!net.load_model(net_path, weight_path))
{
     std::cerr << "Failed to load model!" << std::endl;
     return -1;
    
}

如果網(wǎng)絡(luò)結(jié)構(gòu)和權(quán)重分開(kāi)加載，則先加載結(jié)構(gòu)再加載權(quán)重。

Predict

y = net.predict(x);

到此這篇關(guān)于用C++實(shí)現(xiàn)的簡(jiǎn)易神經(jīng)網(wǎng)絡(luò)框架：SimpleNN的文章就介紹到這了,更多相關(guān)C++實(shí)現(xiàn)神經(jīng)網(wǎng)絡(luò)框架SimpleNN內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！

您可能感興趣的文章: