TensorFlow 的 clip_by_value() 函数不同版本间的行为变化

（原创文章，版权所有。转载请注明出处！）

The behavior of tensorflow’s clip_by_value() function since r1.8 has changed, and the documentation has not been mentioned.

PointCNN 训练 S3DIS 分割运行脚本时，提示 clip_by_value() 函数参数错误：最大最小值与 t 不同型。

问题出在 TensorFlow 的 tf.clip_by_value() 函数的行为在不同版本之间发生改变。

r1.6 的代码如下：

def clip_by_value(t, clip_value_min, clip_value_max,
                  name=None):
  """Clips tensor values to a specified min and max.
  Given a tensor `t`, this operation returns a tensor of the same type and
  shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`.
  Any values less than `clip_value_min` are set to `clip_value_min`. Any values
  greater than `clip_value_max` are set to `clip_value_max`.
  Args:
    t: A `Tensor`.
    clip_value_min: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The minimum value to clip by.
    clip_value_max: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The maximum value to clip by.
    name: A name for the operation (optional).
  Returns:
    A clipped `Tensor`.
  Raises:
    ValueError: if the clip tensors would trigger array broadcasting
      that would make the returned tensor larger than the input.
  """
  with ops.name_scope(name, "clip_by_value",
                      [t, clip_value_min, clip_value_max]) as name:
    t = ops.convert_to_tensor(t, name="t")

    # Go through list of tensors, for each value in each tensor clip
    t_min = math_ops.minimum(t, clip_value_max)
    # Assert that the shape is compatible with the initial shape,
    # to prevent unintentional broadcasting.
    _ = t.shape.merge_with(t_min.shape)

    t_max = math_ops.maximum(t_min, clip_value_min, name=name)
    _ = t.shape.merge_with(t_max.shape)

  return t_max

r1.7 版没有变化。

r1.8 的代码如下：

def clip_by_value(t, clip_value_min, clip_value_max,
                  name=None):
  """Clips tensor values to a specified min and max.
  Given a tensor `t`, this operation returns a tensor of the same type and
  shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`.
  Any values less than `clip_value_min` are set to `clip_value_min`. Any values
  greater than `clip_value_max` are set to `clip_value_max`.
  Args:
    t: A `Tensor`.
    clip_value_min: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The minimum value to clip by.
    clip_value_max: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The maximum value to clip by.
    name: A name for the operation (optional).
  Returns:
    A clipped `Tensor`.
  Raises:
    ValueError: if the clip tensors would trigger array broadcasting
      that would make the returned tensor larger than the input.
  """
  with ops.name_scope(name, "clip_by_value",
                      [t, clip_value_min, clip_value_max]) as name:
    return gen_math_ops.clip_by_value(t,
                                      clip_value_min,
                                      clip_value_max,
                                      name=name)

此版的 clip_by_value() 仅作引用，具体功能由 gen_math_ops.clip_by_value() 实现。该函数的描述没有更改，官方文档也只字未动。由于 gen* 类文件是编译时生成，故原码中没有。

而 gen_math_ops.clip_by_value() 的源代码为：

def clip_by_value(t, clip_value_min, clip_value_max, name=None):
  r"""Clips tensor values to a specified min and max.

  Given a tensor `t`, this operation returns a tensor of the same type and
  shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`.
  Any values less than `clip_value_min` are set to `clip_value_min`. Any values
  greater than `clip_value_max` are set to `clip_value_max`.

  Args:
    t: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`.
      A `Tensor`.
    clip_value_min: A `Tensor`. Must have the same type as `t`.
      A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The minimum value to clip by.
    clip_value_max: A `Tensor`. Must have the same type as `t`.
      A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape
      as `t`. The maximum value to clip by.
    name: A name for the operation (optional).

  Returns:
    A `Tensor`. Has the same type as `t`.
  """
  _ctx = _context._context
  if _ctx is None or not _ctx._eager_context.is_eager:
    _, _, _op = _op_def_lib._apply_op_helper(
        "ClipByValue", t=t, clip_value_min=clip_value_min,
        clip_value_max=clip_value_max, name=name)
    _result = _op.outputs[:]
    _inputs_flat = _op.inputs
    _attrs = ("T", _op.get_attr("T"))
    _execute.record_gradient(
      "ClipByValue", _inputs_flat, _attrs, _result, name)
    _result, = _result
    return _result

  else:
    try:
      _result = _pywrap_tensorflow.TFE_Py_FastPathExecute(
        _ctx._context_handle, _ctx._eager_context.device_name, "ClipByValue",
        name, _ctx._post_execution_callbacks, t, clip_value_min,
        clip_value_max)
      return _result
    except _core._FallbackException:
      return clip_by_value_eager_fallback(
          t, clip_value_min, clip_value_max, name=name, ctx=_ctx)
    except _core._NotOkStatusException as e:
      if name is not None:
        message = e.message + " name: " + name
      else:
        message = e.message
      _six.raise_from(_core._status_to_exception(e.code, message), None)

再观察 r1.9 和 master 版本中的注释，此后对于 clip_by_value() 的实现方式会沿用 r1.8 的实现方式。

v1.9.0 已经于 2018年7月10日发布。v1.8.0 中将 clip_by_value() 具体实现移入 gen_math_ops.cc 文件的做法没有沿用，新版又改回 v1.7.0 的版本，即直接实现函数功能。不过 TODO 注释依然存在，其内容要求负责人两周内修改，实际上到现在为止（2018年8月1日，v1.10.0-rc1已发布）依然没有修改。

从 r1.8 开始，PointCNN 便无法运行。之后的版本虽然未发布，但可以预计，PointCNN 如果不做修改则无法在后续版本中运行。

附说明文档：clip_by_value()

该函数的作用是根据值的范围来修剪张量。

第一个参数就是待修剪的张量 t。

第二个参数定义最小值。参数的类型可以是一个标量，表示待修剪张量的所有元素都必须大于等于该最小值，也可以是一个与待修剪张量同尺寸的张量，表示待修剪张量对应元素的最小值。

第三个参数定义最大值。用法与第二个参数相同。

返回值为修剪后的结果，形状与待修剪的张量相同。

例如：

import tensorflow as tf
import numpy as np

A = np.array([[1,2,3,4], [5,6,7,8]])
with tf.Session() as sess:  
    print sess.run(tf.clip_by_value(A, 2, 7))

则输出应该是：

[[2 2 3 4]
 [5 6 7 7]]

又如：

import tensorflow as tf
import numpy as np

A = np.array([[1,2,3,4], [5,6,7,8]])
with tf.Session() as sess:  
    print sess.run(tf.clip_by_value(A, [[1,1,1,1],[6,6,5,6]],[[1,2,1,2],[6,6,7,8]]))

则输出应该是：

[[1 2 1 2]
 [6 6 7 8]]

再如

import tensorflow as tf
import numpy as np

A = np.array([[1,2,3,4], [5,6,7,8]])
with tf.Session() as sess:  
    print sess.run(tf.clip_by_value(A, [1,2,1,2], [5,6,5,6]))

r1.6/1.7 的输出是（the output using r1.6/1.7）：

[[1 2 3 4]
 [5 6 5 6]]

但是从 r1.8 开始，不再允许与张量的分量同型的向量作为参数。如果第二、三个参数是张量，则必须与第一个参数绝对同型，否则将抛出异常。

However, since r1.8 and beyond, the shape of the clip_value parameter must be exactly the same as t, otherwise it will throw an exception if you pass a vector.

注意：你需要自行保证最小值小于等于最大值，否则最大值不生效。