(原创文章,版权所有。转载请注明出处!)
The behavior of tensorflow’s clip_by_value() function since r1.8 has changed, and the documentation has not been mentioned.
PointCNN 训练 S3DIS 分割运行脚本时,提示 clip_by_value() 函数参数错误:最大最小值与 t 不同型。
问题出在 TensorFlow 的 tf.clip_by_value() 函数的行为在不同版本之间发生改变。
r1.6 的代码如下:
def clip_by_value(t, clip_value_min, clip_value_max, name=None): """Clips tensor values to a specified min and max. Given a tensor `t`, this operation returns a tensor of the same type and shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`. Any values less than `clip_value_min` are set to `clip_value_min`. Any values greater than `clip_value_max` are set to `clip_value_max`. Args: t: A `Tensor`. clip_value_min: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The minimum value to clip by. clip_value_max: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The maximum value to clip by. name: A name for the operation (optional). Returns: A clipped `Tensor`. Raises: ValueError: if the clip tensors would trigger array broadcasting that would make the returned tensor larger than the input. """ with ops.name_scope(name, "clip_by_value", [t, clip_value_min, clip_value_max]) as name: t = ops.convert_to_tensor(t, name="t") # Go through list of tensors, for each value in each tensor clip t_min = math_ops.minimum(t, clip_value_max) # Assert that the shape is compatible with the initial shape, # to prevent unintentional broadcasting. _ = t.shape.merge_with(t_min.shape) t_max = math_ops.maximum(t_min, clip_value_min, name=name) _ = t.shape.merge_with(t_max.shape) return t_max
r1.7 版没有变化。
r1.8 的代码如下:
def clip_by_value(t, clip_value_min, clip_value_max, name=None): """Clips tensor values to a specified min and max. Given a tensor `t`, this operation returns a tensor of the same type and shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`. Any values less than `clip_value_min` are set to `clip_value_min`. Any values greater than `clip_value_max` are set to `clip_value_max`. Args: t: A `Tensor`. clip_value_min: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The minimum value to clip by. clip_value_max: A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The maximum value to clip by. name: A name for the operation (optional). Returns: A clipped `Tensor`. Raises: ValueError: if the clip tensors would trigger array broadcasting that would make the returned tensor larger than the input. """ with ops.name_scope(name, "clip_by_value", [t, clip_value_min, clip_value_max]) as name: return gen_math_ops.clip_by_value(t, clip_value_min, clip_value_max, name=name)
此版的 clip_by_value() 仅作引用,具体功能由 gen_math_ops.clip_by_value() 实现。该函数的描述没有更改,官方文档也只字未动。由于 gen* 类文件是编译时生成,故原码中没有。
而 gen_math_ops.clip_by_value() 的源代码为:
def clip_by_value(t, clip_value_min, clip_value_max, name=None): r"""Clips tensor values to a specified min and max. Given a tensor `t`, this operation returns a tensor of the same type and shape as `t` with its values clipped to `clip_value_min` and `clip_value_max`. Any values less than `clip_value_min` are set to `clip_value_min`. Any values greater than `clip_value_max` are set to `clip_value_max`. Args: t: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. A `Tensor`. clip_value_min: A `Tensor`. Must have the same type as `t`. A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The minimum value to clip by. clip_value_max: A `Tensor`. Must have the same type as `t`. A 0-D (scalar) `Tensor`, or a `Tensor` with the same shape as `t`. The maximum value to clip by. name: A name for the operation (optional). Returns: A `Tensor`. Has the same type as `t`. """ _ctx = _context._context if _ctx is None or not _ctx._eager_context.is_eager: _, _, _op = _op_def_lib._apply_op_helper( "ClipByValue", t=t, clip_value_min=clip_value_min, clip_value_max=clip_value_max, name=name) _result = _op.outputs[:] _inputs_flat = _op.inputs _attrs = ("T", _op.get_attr("T")) _execute.record_gradient( "ClipByValue", _inputs_flat, _attrs, _result, name) _result, = _result return _result else: try: _result = _pywrap_tensorflow.TFE_Py_FastPathExecute( _ctx._context_handle, _ctx._eager_context.device_name, "ClipByValue", name, _ctx._post_execution_callbacks, t, clip_value_min, clip_value_max) return _result except _core._FallbackException: return clip_by_value_eager_fallback( t, clip_value_min, clip_value_max, name=name, ctx=_ctx) except _core._NotOkStatusException as e: if name is not None: message = e.message + " name: " + name else: message = e.message _six.raise_from(_core._status_to_exception(e.code, message), None)
再观察 r1.9 和 master 版本中的注释,此后对于 clip_by_value() 的实现方式会沿用 r1.8 的实现方式。
v1.9.0 已经于 2018年7月10日发布。v1.8.0 中将 clip_by_value() 具体实现移入 gen_math_ops.cc 文件的做法没有沿用,新版又改回 v1.7.0 的版本,即直接实现函数功能。不过 TODO 注释依然存在,其内容要求负责人两周内修改,实际上到现在为止(2018年8月1日,v1.10.0-rc1已发布)依然没有修改。
从 r1.8 开始,PointCNN 便无法运行。之后的版本虽然未发布,但可以预计,PointCNN 如果不做修改则无法在后续版本中运行。
附说明文档:clip_by_value()
该函数的作用是根据值的范围来修剪张量。
第一个参数就是待修剪的张量 t。
第二个参数定义最小值。参数的类型可以是一个标量,表示待修剪张量的所有元素都必须大于等于该最小值,也可以是一个与待修剪张量同尺寸的张量,表示待修剪张量对应元素的最小值。
第三个参数定义最大值。用法与第二个参数相同。
返回值为修剪后的结果,形状与待修剪的张量相同。
例如:
import tensorflow as tf import numpy as np A = np.array([[1,2,3,4], [5,6,7,8]]) with tf.Session() as sess: print sess.run(tf.clip_by_value(A, 2, 7))
则输出应该是:
[[2 2 3 4] [5 6 7 7]]
又如:
import tensorflow as tf import numpy as np A = np.array([[1,2,3,4], [5,6,7,8]]) with tf.Session() as sess: print sess.run(tf.clip_by_value(A, [[1,1,1,1],[6,6,5,6]],[[1,2,1,2],[6,6,7,8]]))
则输出应该是:
[[1 2 1 2] [6 6 7 8]]
再如
import tensorflow as tf import numpy as np A = np.array([[1,2,3,4], [5,6,7,8]]) with tf.Session() as sess: print sess.run(tf.clip_by_value(A, [1,2,1,2], [5,6,5,6]))
r1.6/1.7 的输出是(the output using r1.6/1.7):
[[1 2 3 4] [5 6 5 6]]
但是从 r1.8 开始,不再允许与张量的分量同型的向量作为参数。如果第二、三个参数是张量,则必须与第一个参数绝对同型,否则将抛出异常。
However, since r1.8 and beyond, the shape of the clip_value parameter must be exactly the same as t, otherwise it will throw an exception if you pass a vector.
注意:你需要自行保证最小值小于等于最大值,否则最大值不生效。
关于 “TensorFlow 的 clip_by_value() 函数不同版本间的行为变化” 的 1 个意见
您必须登录才能发表评论。