欧美bbbwbbbw肥妇,免费乱码人妻系列日韩,一级黄片

對(duì)Keras自帶Loss Function的深入研究

 更新時(shí)間:2021年05月25日 09:15:48   作者:Forskamse  
這篇文章主要介紹了對(duì)Keras自帶Loss Function的深入研究,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方,望不吝賜教

本文研究Keras自帶的幾個(gè)常用的Loss Function。

1. categorical_crossentropy VS. sparse_categorical_crossentropy

注意到二者的主要差別在于輸入是否為integer tensor。在文檔中,我們還可以找到關(guān)于二者如何選擇的描述:

解釋一下這里的Integer target 與 Categorical target,實(shí)際上Integer target經(jīng)過(guò)獨(dú)熱編碼就變成了Categorical target,舉例說(shuō)明:

(類別數(shù)5)
Integer target: [1,2,4]
Categorical target: [[0. 1. 0. 0. 0.]
					 [0. 0. 1. 0. 0.]
					 [0. 0. 0. 0. 1.]]

在Keras中提供了to_categorical方法來(lái)實(shí)現(xiàn)二者的轉(zhuǎn)化:

from keras.utils import to_categorical
categorical_labels = to_categorical(int_labels, num_classes=None)

注意categorical_crossentropy和sparse_categorical_crossentropy的輸入?yún)?shù)output,都是softmax輸出的tensor。我們都知道softmax的輸出服從多項(xiàng)分布,

因此categorical_crossentropy和sparse_categorical_crossentropy應(yīng)當(dāng)應(yīng)用于多分類問(wèn)題。

我們?cè)倏纯催@兩個(gè)的源碼,來(lái)驗(yàn)證一下:

https://github.com/tensorflow/tensorflow/blob/r1.13/tensorflow/python/keras/backend.py
--------------------------------------------------------------------------------------------------------------------
def categorical_crossentropy(target, output, from_logits=False, axis=-1):
  """Categorical crossentropy between an output tensor and a target tensor.
  Arguments:
      target: A tensor of the same shape as `output`.
      output: A tensor resulting from a softmax
          (unless `from_logits` is True, in which
          case `output` is expected to be the logits).
      from_logits: Boolean, whether `output` is the
          result of a softmax, or is a tensor of logits.
      axis: Int specifying the channels axis. `axis=-1` corresponds to data
          format `channels_last', and `axis=1` corresponds to data format
          `channels_first`.
  Returns:
      Output tensor.
  Raises:
      ValueError: if `axis` is neither -1 nor one of the axes of `output`.
  """
  rank = len(output.shape)
  axis = axis % rank
  # Note: nn.softmax_cross_entropy_with_logits_v2
  # expects logits, Keras expects probabilities.
  if not from_logits:
    # scale preds so that the class probas of each sample sum to 1
    output = output / math_ops.reduce_sum(output, axis, True)
    # manual computation of crossentropy
    epsilon_ = _to_tensor(epsilon(), output.dtype.base_dtype)
    output = clip_ops.clip_by_value(output, epsilon_, 1. - epsilon_)
    return -math_ops.reduce_sum(target * math_ops.log(output), axis)
  else:
    return nn.softmax_cross_entropy_with_logits_v2(labels=target, logits=output)
--------------------------------------------------------------------------------------------------------------------
def sparse_categorical_crossentropy(target, output, from_logits=False, axis=-1):
  """Categorical crossentropy with integer targets.
  Arguments:
      target: An integer tensor.
      output: A tensor resulting from a softmax
          (unless `from_logits` is True, in which
          case `output` is expected to be the logits).
      from_logits: Boolean, whether `output` is the
          result of a softmax, or is a tensor of logits.
      axis: Int specifying the channels axis. `axis=-1` corresponds to data
          format `channels_last', and `axis=1` corresponds to data format
          `channels_first`.
  Returns:
      Output tensor.
  Raises:
      ValueError: if `axis` is neither -1 nor one of the axes of `output`.
  """
  rank = len(output.shape)
  axis = axis % rank
  if axis != rank - 1:
    permutation = list(range(axis)) + list(range(axis + 1, rank)) + [axis]
    output = array_ops.transpose(output, perm=permutation)
  # Note: nn.sparse_softmax_cross_entropy_with_logits
  # expects logits, Keras expects probabilities.
  if not from_logits:
    epsilon_ = _to_tensor(epsilon(), output.dtype.base_dtype)
    output = clip_ops.clip_by_value(output, epsilon_, 1 - epsilon_)
    output = math_ops.log(output)
  output_shape = output.shape
  targets = cast(flatten(target), 'int64')
  logits = array_ops.reshape(output, [-1, int(output_shape[-1])])
  res = nn.sparse_softmax_cross_entropy_with_logits(
      labels=targets, logits=logits)
  if len(output_shape) >= 3:
    # If our output includes timesteps or spatial dimensions we need to reshape
    return array_ops.reshape(res, array_ops.shape(output)[:-1])
  else:
    return res

categorical_crossentropy計(jì)算交叉熵時(shí)使用的是nn.softmax_cross_entropy_with_logits_v2( labels=targets, logits=logits),而sparse_categorical_crossentropy使用的是nn.sparse_softmax_cross_entropy_with_logits( labels=targets, logits=logits),二者本質(zhì)并無(wú)區(qū)別,只是對(duì)輸入?yún)?shù)logits的要求不同,v2要求的是logits與labels格式相同(即元素也是獨(dú)熱的),而sparse則要求logits的元素是個(gè)數(shù)值,與上面Integer format和Categorical format的對(duì)比含義類似。

綜上所述,categorical_crossentropy和sparse_categorical_crossentropy只不過(guò)是輸入?yún)?shù)target類型上的區(qū)別,其loss的計(jì)算在本質(zhì)上沒(méi)有區(qū)別,就是交叉熵;二者是針對(duì)多分類(Multi-class)任務(wù)的。

2. Binary_crossentropy

二元交叉熵,從名字中我們可以看出,這個(gè)loss function可能是適用于二分類的。文檔中并沒(méi)有詳細(xì)說(shuō)明,那么直接看看源碼吧:

https://github.com/tensorflow/tensorflow/blob/r1.13/tensorflow/python/keras/backend.py
--------------------------------------------------------------------------------------------------------------------
def binary_crossentropy(target, output, from_logits=False):
  """Binary crossentropy between an output tensor and a target tensor.
  Arguments:
      target: A tensor with the same shape as `output`.
      output: A tensor.
      from_logits: Whether `output` is expected to be a logits tensor.
          By default, we consider that `output`
          encodes a probability distribution.
  Returns:
      A tensor.
  """
  # Note: nn.sigmoid_cross_entropy_with_logits
  # expects logits, Keras expects probabilities.
  if not from_logits:
    # transform back to logits
    epsilon_ = _to_tensor(epsilon(), output.dtype.base_dtype)
    output = clip_ops.clip_by_value(output, epsilon_, 1 - epsilon_)
    output = math_ops.log(output / (1 - output))
  return nn.sigmoid_cross_entropy_with_logits(labels=target, logits=output)

可以看到源碼中計(jì)算使用了nn.sigmoid_cross_entropy_with_logits,熟悉tensorflow的應(yīng)該比較熟悉這個(gè)損失函數(shù)了,它可以用于簡(jiǎn)單的二分類,也可以用于多標(biāo)簽任務(wù),而且應(yīng)用廣泛,在樣本合理的情況下(如不存在類別不均衡等問(wèn)題)的情況下,通??梢灾苯邮褂?。

補(bǔ)充:keras自定義loss function的簡(jiǎn)單方法

首先看一下Keras中我們常用到的目標(biāo)函數(shù)(如mse,mae等)是如何定義的

from keras import backend as K
def mean_squared_error(y_true, y_pred):
    return K.mean(K.square(y_pred - y_true), axis=-1)
def mean_absolute_error(y_true, y_pred):
    return K.mean(K.abs(y_pred - y_true), axis=-1)
def mean_absolute_percentage_error(y_true, y_pred):
    diff = K.abs((y_true - y_pred) / K.clip(K.abs(y_true), K.epsilon(), np.inf))
    return 100. * K.mean(diff, axis=-1)
def categorical_crossentropy(y_true, y_pred):
    '''Expects a binary class matrix instead of a vector of scalar classes.
    '''
    return K.categorical_crossentropy(y_pred, y_true)
def sparse_categorical_crossentropy(y_true, y_pred):
    '''expects an array of integer classes.
    Note: labels shape must have the same number of dimensions as output shape.
    If you get a shape error, add a length-1 dimension to labels.
    '''
    return K.sparse_categorical_crossentropy(y_pred, y_true)
def binary_crossentropy(y_true, y_pred):
    return K.mean(K.binary_crossentropy(y_pred, y_true), axis=-1)
def kullback_leibler_divergence(y_true, y_pred):
    y_true = K.clip(y_true, K.epsilon(), 1)
    y_pred = K.clip(y_pred, K.epsilon(), 1)
    return K.sum(y_true * K.log(y_true / y_pred), axis=-1)
def poisson(y_true, y_pred):
    return K.mean(y_pred - y_true * K.log(y_pred + K.epsilon()), axis=-1)
def cosine_proximity(y_true, y_pred):
    y_true = K.l2_normalize(y_true, axis=-1)
    y_pred = K.l2_normalize(y_pred, axis=-1)
    return -K.mean(y_true * y_pred, axis=-1)

所以仿照以上的方法,可以自己定義特定任務(wù)的目標(biāo)函數(shù)。比如:定義預(yù)測(cè)值與真實(shí)值的差

from keras import backend as K
def new_loss(y_true,y_pred):
    return K.mean((y_pred-y_true),axis = -1)

然后,應(yīng)用你自己定義的目標(biāo)函數(shù)進(jìn)行編譯

from keras import backend as K
def my_loss(y_true,y_pred):
    return K.mean((y_pred-y_true),axis = -1)
model.compile(optimizer=optimizers.RMSprop(lr),loss=my_loss,
metrics=['accuracy'])

以上為個(gè)人經(jīng)驗(yàn),希望能給大家一個(gè)參考,也希望大家多多支持腳本之家。

相關(guān)文章

  • python文件特定行插入和替換實(shí)例詳解

    python文件特定行插入和替換實(shí)例詳解

    這篇文章主要介紹了python文件特定行插入和替換實(shí)例詳解的相關(guān)資料,需要的朋友可以參考下
    2017-07-07
  • 解決vscode python print 輸出窗口中文亂碼的問(wèn)題

    解決vscode python print 輸出窗口中文亂碼的問(wèn)題

    今天小編就為大家分享一篇解決vscode python print 輸出窗口中文亂碼的問(wèn)題,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧
    2018-12-12
  • 幫你快速上手Jenkins并實(shí)現(xiàn)自動(dòng)化部署

    幫你快速上手Jenkins并實(shí)現(xiàn)自動(dòng)化部署

    在未學(xué)習(xí)Jenkins之前,只是對(duì)Jenkins有一個(gè)比較模糊的理解,即Jenkins是一個(gè)自動(dòng)化構(gòu)建項(xiàng)目發(fā)布的工具,可以實(shí)現(xiàn)代碼->github或者gitlab庫(kù)->jenkins自動(dòng)部署->訪問(wèn)的整體的過(guò)程,而無(wú)需人為重新打包,今天就帶大家詳細(xì)了解一下,幫你快速上手Jenkins,需要的朋友可以參考下
    2021-06-06
  • python文字轉(zhuǎn)語(yǔ)音的實(shí)例代碼分析

    python文字轉(zhuǎn)語(yǔ)音的實(shí)例代碼分析

    在本篇文章里小編給大家整理的是關(guān)于python文字轉(zhuǎn)語(yǔ)音的實(shí)例代碼分析,有需要的朋友們可以參考下。
    2019-11-11
  • Python break語(yǔ)句詳解

    Python break語(yǔ)句詳解

    這篇文章主要介紹了Python break語(yǔ)句的作用、使用方法,需要的朋友可以參考下
    2014-03-03
  • python寫(xiě)入csv時(shí)writerow()和writerows()函數(shù)簡(jiǎn)單示例

    python寫(xiě)入csv時(shí)writerow()和writerows()函數(shù)簡(jiǎn)單示例

    這篇文章主要給大家介紹了關(guān)于python寫(xiě)入csv時(shí)writerow()和writerows()函數(shù)的相關(guān)資料,writerows和writerow是Python中csv模塊中的兩個(gè)函數(shù),用于將數(shù)據(jù)寫(xiě)入CSV文件,需要的朋友可以參考下
    2023-07-07
  • Pytorch 之修改Tensor部分值方式

    Pytorch 之修改Tensor部分值方式

    今天小編就為大家分享一篇Pytorch 之修改Tensor部分值方式,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧
    2019-12-12
  • 對(duì)python中的argv和argc使用詳解

    對(duì)python中的argv和argc使用詳解

    今天小編就為大家分享一篇對(duì)python中的argv和argc使用詳解,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧
    2018-12-12
  • 用Python代碼來(lái)解圖片迷宮的方法整理

    用Python代碼來(lái)解圖片迷宮的方法整理

    這篇文章主要介紹了用Python代碼來(lái)解圖片迷宮的方法整理,本文精選了StackOverflow相關(guān)人氣問(wèn)題上的幾個(gè)回答,需要的朋友可以參考下
    2015-04-04
  • Python 多線程并行執(zhí)行的實(shí)現(xiàn)示例

    Python 多線程并行執(zhí)行的實(shí)現(xiàn)示例

    本文主要介紹了Python 多線程并行執(zhí)行的實(shí)現(xiàn)示例,通過(guò)使用threading和concurrent.futures模塊可以進(jìn)行實(shí)現(xiàn),具有一定的參考價(jià)值,感興趣的可以了解一下
    2024-07-07

最新評(píng)論